No Options A, B, and C: The Oral AI Examiner

Main idea: a standard test is good at checking the recognition of the right option. An oral format forces the person to explain the requirement themselves. In this setup, AI is needed not to prompt the answer, but to provide a consistent semantic assessment.

Why We Tackled the Oral Exam in the First Place

Classic tests are convenient for mass knowledge assessment, but they have an obvious weakness: the employee sees the answer options. Even with a good question bank, part of the score can be obtained by recognizing the wording, eliminating obviously wrong options, or simply memorizing the test.

An oral exam works differently. The person has to formulate the answer themselves, explain the sequence of actions, and name the hazards, restrictions, and mandatory requirements. This format shows much better whether they actually understood the rule or just saw the correct wording before.

But a standard oral exam has another problem — time and the inevitable difference between examiners. One digs deeper with follow-up questions, another accepts the answer more quickly; one values exact wording, another looks for meaning. That's why in the spring of 2026 we started looking for a way to use AI as a second examiner: not to replace the committee, but as a unified tool that asks the question, listens to the answer, and applies the set criteria equally.

Stage One: Trying to Do Everything Inside a Single AI Agent

The initial idea was quite straightforward: upload the rules into the AI, give it the role of an examiner, and make it simultaneously select a random knowledge area, formulate a question, and assess the answer. In practice, it quickly became clear that a generative model is not the best tool for true randomness. It reasons well, but for it, a "random question" does not always mean a mathematically random choice.

Therefore, we began separating the functions. The regulatory source should be responsible only for the knowledge. Random selection — for the question number. The examiner — only for the sequence of the exam and the assessment of the answer. This was the first major architectural takeaway: the more strictly the functions are separated, the more stable the system is.

Further on, in the appendix, the prompts are deliberately arranged in the same logic as the article: first the historical outline, then the question bank, after that the modern HTML version, assessment, Word/Excel, and data protection.

Stage Two: NotebookLM as the Knowledge Base, Gemini Bot as the Examiner

A working scheme emerged when we separated the knowledge base from the exam itself. The regulatory sources were uploaded into separate Notebooks in NotebookLM: the Law of the Republic of Kazakhstan "On Civil Protection", the Labour Code, and industry regulations — for lifting mechanisms, open-pit mining, pressure vessels, drilling and blasting operations, and other streams.

Gemini Bot played the role of the examiner. It received an instruction: work only within the connected regulatory framework, ask questions one by one, accept the oral answer via the microphone, and assess not the word-for-word match, but the meaning — accuracy, completeness, and logic.

In one of the iterations, we prepared a bank of 300 numbered questions with reference answers and links to regulatory clauses. We needed to eliminate the model picking a question "at its own discretion", which is how our own random number generator came about.

See the appendix with prompts: Prompt 1 — the full historical scenario of the NotebookLM + Gemini examiner, including three questions, the number generator, and answer assessment; Prompt 2 — preparing the question bank based on a regulatory source.

How Our First Random Number Generator Worked

We deliberately created a mechanism that could be described and reproduced without a separate program. After data input, the hours, minutes, seconds, and milliseconds were recorded. Zero values were replaced with one. Then the values were multiplied:

R = H × M × S × MS

If the result was greater than 300, it was consecutively divided by 300 until it became less than or equal to 300. After this, the number was rounded up and became the question number in the bank. For the second question, the time the answer to the first one was received was recorded; for the third — the answer time to the second. If the number repeated an already asked question, one was added to it, looping back to one after 300.

Today this scheme looks cumbersome, but at that stage, it solved an important task: the choice of question no longer depended on the "desire" of the AI. Randomness became a separate function, and the AI examiner received a specific question number.

See appendix: the formula for choosing the number, the recalculation after each answer, and the exclusion of repeats are saved in Prompt 1.

The Pilot: 54 Employees and the First Real Experience

In April 2026, using this scheme, we conducted a pilot with 54 participants — blue-collar workers, office staff, and engineering and technical personnel. The oral exam included three questions, and the pass mark was 70%. The mandatory areas were the Law of the Republic of Kazakhstan "On Civil Protection" and the Labour Code; additionally, the participant underwent a knowledge assessment on industry regulations related to their job.

The answer was given by voice. The AI compared it against the regulatory framework and provided not only a percentage but also the structure of the answer: what was covered correctly, what was left unsaid, where an error was made, and which regulatory source to refer back to.

This is where the main advantage became clear: the system tests not the memory for multiple-choice options, but a person's ability to explain the requirement themselves.

What Proved Inconvenient in the First Version

The pilot confirmed the principle itself but also revealed weak points. The protocol had to be copied and saved separately in Word. The final percentage was then manually transferred to Excel. For different streams, separate notebooks and bots were used, so the specialist had to switch between them.

Furthermore, the first version identified the examinee by their full name, position, and department. For an internal pilot, this was understandable, but as we developed it further, it became obvious: if part of the processing is done by a cloud model, it's better to remove personal data from this flow.

Thus, the next stage emerged — not just another prompt, but our own software shell.

Stage Three: The Portable Browser Examiner

The new version works as a portable HTML/JavaScript application in a separate browser window. It doesn't need Python, Node.js, a database, or administrator rights. The question bank, regulatory sources, exam history, and settings are stored locally on the computer.

It is important to accurately separate the local and cloud parts. The interface itself, local database, question selection, calculation of the final percentage, Word protocol, and Excel export are executed on the user's computer. But speech recognition, semantic assessment of the answer, cloud text-to-speech, and generating draft questions from a regulatory source use the cloud model. That is, it's not a "completely offline AI", but a local secure shell with the minimum necessary cloud footprint.

In a few days, this architecture gave us what we lacked in the first pilot: a single interface instead of a set of bots, a unified bank of approved questions, an automatic protocol, and a cumulative history of results.

If the reader wants to recreate the modern version from scratch, the main starting request is Prompt 3. It is specifically designed for vibe coding: the AI guides a non-programmer step by step, one action at a time, checking each result.

Personal Data: RSS Code Instead of Full Name

In the new version, the employee does not enter their full name and personnel number. Before the exam, a pseudonymous RSS code like RSS 1256 is used. The RSS ↔ full name mapping is stored separately inside the corporate perimeter and can be merged with the results locally.

There is no need to transmit the full name or personnel number into the cloud stream. For the semantic assessment by the model, the question itself, the reference answer, the criteria, and the employee's voice answer are sufficient. The interface separately indicates that personal data must not be spoken into the microphone either.

The Word protocol saves the RSS code, and leaves the full name and knowledge assessment subject fields blank to be filled in after downloading. The daily Excel export also contains the RSS code, position, department, language, number of questions, and the result — without the full name and personnel number.

See appendix: Prompt 8 — personal data audit and fair separation of local and cloud boundaries. See Prompt 7 for the logic of automatic Word/Excel results.

How the Question Bank Is Generated Now

This is another fundamental difference from the first version. Now there is no need to manually prepare exactly 300 questions in advance. A regulatory source can be uploaded to the program: Word, PDF, RTF, ODT, TXT, or Markdown. The AI then selects meaningful excerpts distributed throughout the document and creates draft questions, reference answers, and 2–8 assessment criteria with weights.

But an automatically generated question does not make it into the official exam straight away. It gets a draft status. The specialist must check the wording, reference answer, criteria, and source link, and only after that approve the question. Only approved questions are used for the exam.

You can also import a pre-prepared bank from CSV or XLSX. A question stores the topic, language, source, difficulty, reference answer, and criteria. Therefore, different assessment streams can be managed in a single application, rather than in separate bots.

The random selection has changed too. Instead of our old time-based formula, the program uses the system's cryptographic random number generator to select the required number of approved questions. This is simpler, more reliable, and does not require the AI to simulate randomness.

See appendix: Prompt 4 — generating draft questions from a regulatory source with a mandatory methodological review; Prompt 9 — combining multiple exam streams in a single interface.

The Exam Itself: Voice Question, Voice Answer

The examinee chooses the Russian or Kazakh language. The question can be read on the screen or listened to. Separate voice instructions are used for each language so that the speech is natural and normative, without any unnecessary foreign accent.

Microphone access is requested once for the entire exam. The answer can be started manually with a button or automatically after 30 seconds, one, two, or three minutes of preparation. After recording, the system recognizes the speech, displays the transcript, and allows checking the text before assessment.

If the answer is given in one's own words, this is not a drawback. The examiner looks at the meaning, not the literal wording. For each criterion, the following states are possible: fully covered, partially covered, missing, or contradicting the requirement. Then, the programmatic algorithm calculates the percentage based on pre-set weights. In case of a critical dangerous error, a separate cap on the maximum score can be applied.

This way, the generative model is responsible for understanding the meaning, while the final percentage is calculated deterministically based on the structure of the criteria. This is an important separation: the AI must not freely "invent a score."

See appendix: Prompt 5 — semantic assessment of the answer based on criteria; Prompt 6 — separate programmatic percentage calculation. Thus, the model determines the content of the answer, and the final score is calculated deterministically.

The video "Oral Exam in the Browser Version" is at the end of the article.

The video shows the finished result of the work: launching the exam, the voice answer, assessment, generation and downloading of the protocol, history, and subsequent export. The step-by-step logic for creating such an application begins with Prompt 3.

The Protocol and History Are Now Generated Automatically

After completing the exam, the specialist receives a ready-made Word protocol. It saves the questions, answer transcripts, assessments, regulatory sources, strengths, missing elements, and errors. The full name remains blank, and the RSS code is saved.

In parallel, each exam is recorded in the local history. At the end of the day, the specialist selects a date and clicks "Export Excel for Date." A single table is generated for all exams: date, time, RSS, position, department, language, number of questions, pass mark, final percentage, result, and status.

This Excel export can already go into a cumulative form and further — into an interactive dashboard. That is, we got the same principle that we used in previous publications: first a separate operation, then the accumulation of structured data, then management analytics.

See appendix: Prompt 7 — structure of the Word protocol and daily Excel export without full name and personnel number.

Where This Can Be Used

Such an examiner is not tied only to industrial safety. By the same logic, you can conduct a knowledge assessment of occupational health and safety requirements, industry rules, instructions, technological regulations, internal standards, and other documents where it is important not to choose the correct letter, but to explain the requirement in your own words.

It can be used as a preliminary check before the commission, as a training element, as an oral exam in specific areas, or as a tool for selective control of understanding requirements. But the final decision where legislation or internal procedures require a commission and authorized persons remains with humans.

The main change compared to the first pilot is that we moved away from a set of separate AI bots and manual transfer of results to a single repeatable process: source → approved question → voice answer → semantic assessment → protocol → Excel → analytics.

What I Think Is the Most Important Thing Here

The most valuable part of the project is not the voice and not the beautiful interface. It is the change in the very logic of the knowledge assessment.

A test asks: "Can you recognize the correct option?" An oral exam asks: "Can you explain yourself what needs to be done and why?"

AI makes this format scalable. It does not cancel the responsibility of the commission and does not replace regulatory requirements. It allows asking questions in the same way, analyzing the meaning of the answer in the same way, and immediately showing the specific gaps that need to be closed.

Appendix: Prompts for the Publication

The historical logic of the first version + updated prompts for the portable browser examiner

The appendix deliberately separates the old and new approaches. The first prompts show the evolution. The subsequent ones are a more practical modern architecture, where the source, bank generation, methodological review, voice answer, assessment, Word, and Excel are separated.

Prompt 1. The Historical NotebookLM + Gemini Examiner With a 300-Question Bank

Connection to the article: sections "Stage Two: NotebookLM as the Knowledge Base, Gemini Bot as the Examiner" and "How Our First Random Number Generator Worked".

ROLE
You are a strict examiner for the knowledge assessment of industrial safety, civil protection and occupational health and safety requirements.
Work only based on the connected regulatory framework and the approved question bank. Do not use the internet and external knowledge. Do not invent requirements.

LANGUAGE
The exam is conducted in the selected language: RU or KZ. Do not mix languages.

STRUCTURE
The exam consists of 3 questions. Ask only one question at a time. The next question — only after the answer and assessment of the previous one.
Pass mark: 70%.

QUESTION 1 SELECTION
After entering data, record the local time:
H — hours; M — minutes; S — seconds; MS — milliseconds.
If any value is 0, replace it with 1.
R1 = H × M × S × MS.
While R1 > 300: R1 = R1 / 300.
INDEX1 = ceil(R1). If INDEX1 = 0, use 1.
Ask question No. INDEX1 from the approved bank of 300 questions.

QUESTION 2
After receiving the answer to question 1, record H, M, S, MS again and repeat the calculation.
If the new number has already been used, increase it by 1. If the number becomes >300, use 1.

QUESTION 3
After receiving the answer to question 2, repeat the same procedure. Do not allow repetition of previously asked questions.

ANSWER ASSESSMENT
Assess the meaning, not verbatim accuracy.
1. Accuracy — up to 60%.
2. Completeness — up to 25%.
3. Logic — up to 15%.
In case of a dangerous statement, a prohibited action or a critical contradiction to a regulatory requirement, the result for the question must not exceed 30%.
For each question show:
- number and wording;
- regulatory source;
- what is covered correctly;
- what is not mentioned;
- what is stated incorrectly;
- percentage for the question.

FINAL
RESULT = (Q1 + Q2 + Q3) / 3.
Round the result to a whole number.
≥70% — PASSED.
<70% — FAILED.
After the third answer, generate the final protocol.

Prompt 2. Preparing the Exam Question Bank Based on a Regulatory Source

Connection to the article: the historical bank of 300 questions in the second stage. This prompt is needed if the bank is prepared in advance based on a regulatory source.

I am uploading a regulatory document.
We need to prepare a question bank for an oral exam.

Rules:
1. Work only with the uploaded document.
2. Do not use external knowledge.
3. Do not ask for article and clause numbers just to test memory — check practical understanding of the requirements.
4. Distribute the questions throughout the entire document, not just the first sections.
5. Exclude duplicates and nearly identical questions.
6. For each question, prepare a reference answer and a link to the source.
7. In the reference answer, highlight the mandatory elements without which the answer cannot be considered complete.
8. Separately mark critical errors — statements that could lead to dangerous action.

Table format:
No. | Topic | Question | Reference answer | Mandatory elements | Critical error | Document | Article/clause | Page | Language

Prepare up to 300 questions. If the regulatory document does not contain enough material for 300 substantially different questions, do not create artificial repetitions: stop at a factually justified number.

Prompt 3. Master Prompt for Vibe Coding the Portable HTML Examiner

Connection to the article: section "Stage Three: The Portable Browser Examiner". This is the main prompt for the reader who wants to replicate the modern version.

I am not a programmer. Walk me through creating a portable oral AI examiner step-by-step.
Do not give me 20 steps at once. For each step:
1. Explain what we are doing and why.
2. Give one specific action or one ready-made block of code/file.
3. Wait for my screenshot or result.
4. Check the screenshot and only then move on.

TARGET ARCHITECTURE
- portable HTML/JavaScript application;
- runs on Windows without Python, Node.js or administrator rights;
- local storage of the question bank, sources and history;
- without full name and personnel number in the exam stream;
- mandatory pseudonymous RSS code in the format RSS 1256;
- Russian and Kazakh language;
- question bank import from XLSX/CSV;
- uploading regulatory sources DOC/DOCX/PDF/RTF/ODT/TXT/MD;
- generating draft questions based on randomly distributed source sectors;
- each generated question must be checked and approved by a specialist before the exam;
- a question stores the topic, wording, reference answer, 2–8 weighted criteria, source and link to the clause;
- only approved questions are used for the exam;
- do random selection of questions programmatically via a cryptographic generator, not through the model's "randomness";
- number of questions and pass mark are customizable;
- voice answer via microphone;
- answer start mode: manually or automatically after 30/60/120/180 seconds of preparation;
- request the microphone once for the entire exam;
- speech recognition using a cloud model;
- semantic assessment of the answer based on criteria;
- percentage calculated deterministically by criteria weights;
- in case of a critical error, apply a customizable score ceiling;
- natural voiceover of the question RU/KZ, system voice as a backup;
- automatic Word protocol;
- local history;
- Excel export of all exams for a selected date;
- backup of the local database without the API key.

SECURITY
Strictly separate the local and cloud contours. Explain what data goes into the cloud. Do not transmit full name and personnel number there. Display a direct warning in the voice interface not to say personal data out loud.

Start with the architecture and file structure. Write code only after my confirmation.

Prompt 4. Generating Draft Questions From a Regulatory Source for the New Version

Connection to the article: section "How the Question Bank Is Generated Now" — generating draft questions from uploaded regulatory sources.

You only generate DRAFT exam questions. No created question is considered automatically approved.

Input:
- regulatory document;
- RU or KZ language;
- required number of draft questions;
- random number/seed generated by the program.

Task:
1. Using the seed, choose meaningful sectors distributed throughout the document, preferably non-consecutive.
2. Work only with the document.
3. Formulate one practically significant question from each sector.
4. Do not ask for the clause number as an end in itself.
5. Prepare a brief reference answer.
6. Prepare from 2 to 8 assessment criteria. Each criterion must check a separate semantic element.
7. Assign weights to the criteria so that the sum is exactly 100.
8. Specify the topic, difficulty, page/article/clause or other link to the source.
9. Do not duplicate questions from adjacent fragments.
10. If a sector does not contain a sufficient requirement for a meaningful question — skip it and choose another.

Return a structured result:
Topic | Question | Reference answer | Criteria with weights | Source | Link | Difficulty | Language

Status of each result: DRAFT, REQUIRES METHODOLOGICAL REVIEW.

Prompt 5. Semantic Assessment of the Oral Answer Based on Criteria

Connection to the article: section "The Exam Itself: Voice Question, Voice Answer" — the model determines how well the meaning of each criterion has been covered.

You are a strict but objective examiner in occupational health and safety and industrial safety.

Assess the meaning, not the word-for-word match.
Accept synonyms and conversational phrasing if they unambiguously convey the regulatory requirement.
Work only based on the provided question, reference answer, criteria, and the regulatory source excerpt.
Do not add requirements from memory or the internet.

For EACH criterion, return exactly one status:
- FULL — fully covered;
- PARTIAL — a substantial part is covered, but there is a gap;
- MISSING — element is missing;
- INCORRECT — the answer contradicts the requirement.

For each criterion, provide a brief proof from the employee's answer and an explanation.

CRITICAL_ERROR = true only when the answer contains a dangerous statement or action that creates an unacceptable risk and contradicts a regulatory requirement.
Do not use critical_error simply for an incomplete answer.

Separately generate:
- WELL_SAID — what is covered well;
- PARTLY_SAID — what is partially covered;
- MISSING — what is not said;
- INCORRECT — what is said incorrectly;
- SUMMARY — a short conclusion in the language of the exam.

Do not calculate the total percentage "by eye". The percentage must be calculated by a programmatic algorithm based on the criteria weights after receiving your statuses.

Prompt 6. Programmatic Logic for Deterministic Score Calculation

Connection to the article: the same section "The Exam Itself: Voice Question, Voice Answer" — the percentage is calculated programmatically, not assigned "by eye" by the model.

Implement the score calculation NOT through the free assessment of the model, but programmatically.

For each criterion there is a weight in percentage.
Coefficients:
FULL = 1.0
PARTIAL = 0.5
MISSING = 0
INCORRECT = 0

SCORE = sum(weight × coefficient) for all criteria.
Limit the result to the 0–100 range and round to 0.1.

If the AI assessor returned critical_error=true:
SCORE = min(SCORE, CRITICAL_CAP), where CRITICAL_CAP is set in the settings (for example, 40%).

After all questions:
TOTAL_SCORE = arithmetic mean of the scores for the questions.
PASSED = TOTAL_SCORE >= the established pass mark.

Important:
- the model determines the semantic status of the criterion;
- the program calculates the percentage;
- the model must not independently assign an arbitrary final score.

Prompt 7. Word Protocol and Daily Excel Export Without Personal Data

Connection to the article: section "The Protocol and History Are Now Generated Automatically" — the Word protocol and daily Excel export.

Enhance the oral examiner.

WORD PROTOCOL
After the exam is completed, automatically generate a .docx.
Include in the protocol:
- date;
- RSS code;
- position/profession;
- department;
- language;
- pass mark;
- final result PASSED/FAILED;
- each question;
- transcript of the answer;
- percentage for the question;
- regulatory source and link;
- what is covered well;
- what is not said;
- errors.

Leave the "Full name" and "Subject of knowledge assessment" fields blank for local filling after downloading.
File name: Protocol_RSS_1256_YYYY-MM-DD.docx.

EXCEL FOR THE DATE
In the "History" section, add a date selection and an "Export Excel for date" button.
Columns:
№ | Exam date | Start time | RSS code | Position/profession | Department | Language | Number of questions | Pass mark % | Total score % | Result | Status | End time | Exam ID.

Do not include the full name and personnel number.
Generate the Excel only from the local history of the current computer.
The mapping of RSS <-> full name must be merged with the results only offline within the corporate perimeter.

Prompt 8. Personal Data Audit and the Offline/Cloud Boundaries

Connection to the article: sections "Stage Three: The Portable Browser Examiner" and "Personal Data: RSS Code Instead of Full Name".

Conduct a technical audit of the oral examiner app.
You need to honestly separate local and cloud operations.

Check and describe:
1. What data is stored in IndexedDB/localStorage.
2. What data is sent to the cloud model during transcription.
3. What data is sent during the semantic assessment.
4. What data is sent when generating draft questions.
5. Whether the full name, personnel number, employee list, or the RSS <-> full name mapping table is transmitted.
6. Where the API key is stored.
7. Whether the backup copy contains the API key.
8. What remains after completely closing the app.
9. How to delete the local history and sources.
10. What risks remain during pilot operation.

Requirement for phrasing the result:
do not call the app "completely offline" if speech recognition, assessment, TTS, or question generation use a cloud API.
The correct phrasing is: "a local portable shell with a local database and a controlled cloud AI perimeter".

Prompt 9. Adding Multiple Exam Streams in a Single Interface

Connection to the article: section "How the Question Bank Is Generated Now" — multiple rules and streams in one examiner without switching between bots.

Refine the question bank and the exam start screen so that it is no longer necessary to create a separate bot for each rule.

Every question already has a "Theme/stream" field.
Add a stream selection to the exam screen:
- a single thematic block;
- multiple thematic blocks simultaneously;
- the "All selected streams" mode.

When launching the exam:
1. filter only the approved questions;
2. take into account the selected language;
3. take into account the selected streams;
4. randomly select the specified number of questions using a programmatic cryptographic generator;
5. distribute the questions among the selected streams as evenly as possible;
6. do not allow duplicates;
7. if there are not enough questions for the selected set — stop the launch and show a clear message.

In the Word protocol and Excel, additionally save the selected exam streams.

How to Use the Appendix to the Article

The numbering below is synchronized with the text of the publication. In the article, next to the corresponding stage, there are direct links "See Prompt ...", and here, under each prompt, it is indicated which section of the article it refers to.

• Prompt 1 — historical version: shows how the first examiner and our own number generator from 300 questions worked.
• Prompt 2 — preparation of a large question bank based on a regulatory source.
• Prompt 3 — the main request for the reader who wants to replicate the modern browser version using vibe coding.
• Prompt 4 — generation of draft questions from the uploaded source.
• Prompt 5–6 — separation of the AI semantic assessment and the programmatic percentage calculation.
• Prompt 7 — automatic Word and Excel.
• Prompt 8 — mandatory check of the local and cloud boundaries.
• Prompt 9 — combining multiple exam streams in a single interface.