A Guide to building ambient scribing that handles the physical exam

Most information is spoken, but not all of it

In somatic medicine, a lot of the clinically relevant information enters the room through speech. The patient describes their symptoms, the clinician asks follow-up questions, reads measurements aloud and explains the plan. That content enters the audio and can be captured.

The examination happens in the hands, not in the audio

During a physical examination the audio is often fragmented. The clinician says "breathe in", "does this hurt?" or "and here?", and the patient answers yes or no. The finding itself, for example tenderness in the right lower quadrant, may never be stated. Some clinicians narrate findings as they go, while others examine in silence and write the findings down afterwards.

An ambient system has no way of knowing what the clinician's hands are doing. It only knows what is said, and it has to infer from the words which examination is being performed and what was found. When the words are few, or when they are about something else entirely, this inference becomes unreliable.

Would a human understand the transcript?

This is a key thing to keep in mind when using an ambient scribe: If a human reading the transcript cannot tell whether something was an examination finding or reported history, the system cannot reliably tell either. In those cases the solution is not in the prompt but in how the consultation is conducted, and that is why ambient best practices matter as much as template and prompt design.

What we learned in practice

1. Stating findings and naming the examination

The most reliable examination sections come from consultations where the clinician states the findings and which examination they belong to, for example "abdomen soft, non-tender, no masses". It doesn't need to be long or formal, but it needs to be said.

Be specific: Stating the finding isn't always enough on its own. The specific details within it need to be said too. Laterality is a common example: "tender here" names the finding, but the side often comes from a gesture or where the clinician is standing, not from the words. The same goes for other localising details, such as which quadrant, how many centimetres or how far from a landmark. If it's not said, it's not captured, even when the finding itself was clearly stated.

Post-speak: For clinicians who prefer to examine in silence, or who don't want to narrate findings in front of the patient, a short verbal summary of the findings after the examination or after the patient has left works well. The same approach is used in psychiatry for mental status observations. Similarly, if photodocumentation (e.g. of rashes or wounds) is used, what the photos show must also be verbalised to be captured by the scribe.

More generally, mixing fluent conversation with dictation-like passages, as if speaking to a stenographer or secretary in the room, typically results in very high-quality documentation. This can be done with any passage, not just examination findings.

In practice

State the finding and the examination it belongs to, including details such as side and location. If you prefer to examine in silence, give a short verbal summary afterwards, and remember to say out loud what any photos show.

2. History taken during the examination

A typical example is an abdominal examination. While the patient is lying down, the clinician asks a few questions:

"If you could just lie back for me and loosen your belt a little. Lovely. No problems with your stomach lately, constipation or diarrhoea [the patient shakes their head]. And no blood in the stool [the patient shakes head]. That's fine."

The transcript often doesn't show that these were questions, because the inflection is lost, which is why the example above has no question marks. The patient's answers are either not spoken or not captured, and as a result the note states "Digital Rectal Examination: No blood", although no rectal examination was performed.

From a clinical point of view, what happened is fairly clear. The clinician was most likely palpating the abdomen while talking about something else. This is a good tactic in general, because it distracts the patient and makes tenderness easier to assess, but the transcript does not establish which examination manoeuvres were performed or what they showed. The question about blood in the stool was history, not an examination finding, and to conclude "no blood" as a finding would require a rectal examination.

For an LLM, however, a symptom question asked in the middle of an examination looks a lot like an examination finding. Taking history during the physical examination is one of the most common sources of misattribution in somatic ambient scribing. The simplest solution is to ask history questions before or after the examination, not during it. If that isn't practical, the clinician can make the distinction clear in what they say, by stating the findings themselves, for example "abdomen soft, non-tender, no masses", and by repeating the patient's answers back to them.

In Ppractice
‍
‍Ask history questions before or after the examination where possible. If you do ask them during the examination, state the findings explicitly and repeat the patient's answers back to them.

3. Answers that are never spoken

In the same example, the transcript contains the clinician's questions but not the patient's answers. The patient most likely nodded, shook their head or said "mmhm". In the room this is a perfectly clear answer, but in the transcript it is either missing or meaningless.

When the patient answers non-verbally, the clinician should repeat the answer out loud, for example "so no history of bloody stool". Many clinicians already do this naturally, and it makes a large difference to how reliably the answer is captured and attributed.

In practice

When the patient nods, shakes their head or says "mmhm", repeat the answer out loud.

4. Information read from the screen

Lab results, imaging reports, vitals and previous notes are often read silently on the screen and only referred to in passing, for example "your values look fine". If specific results should be in the note, they need to be mentioned out loud or added afterwards. The system cannot document what it never hears.

In practice

Mention the results you want in the note out loud, or add them to the note afterwards.

5. Microphone positioning

In general practice settings, the conversation between patient and clinician usually takes place at the clinician's desk, with a microphone positioned there. For the physical examination, both usually move to the examination table, which might be at a distance from the microphone on the desk. Sometimes a single microphone captures that well, but if this keeps causing problems, review microphone placement and the supported audio setup as part of implementation.

In practice

If the examination table is far from the desk and the examination is often poorly captured, consider a second microphone near the table.

6. Templates and prompts

For implementation teams, the template and prompt should support the same distinction between history and examination. It helps to keep the two clearly separated in the template, and to instruct the model not to treat questions asked during the examination as findings. Examination fields should be left empty when nothing was said about them, rather than filled with assumed normal findings.

In practice

Keep history and examination separate in the template, instruct the model not to treat questions as findings, and leave examination fields empty when nothing was said.

7. Reviewing the note

Whatever the setup, the clinician should read through the examination section before signing the note. Normal findings for systems that were never examined are one of the errors that are easiest to miss.

In practice

Read through the examination section before signing, and look out for normal findings in systems you didn't examine.

‍