Fictional Visit
Fictional practice data
This visit is invented and simplified for teaching. It is not clinical guidance. Never paste real patient information into an unapproved model.
Dr. Patel: What brings you in today?
Ms. Rivera: Follow-up for my blood pressure. Honestly, I stopped the lisinopril about two weeks ago. It made me dizzy when I stood up.
Dr. Patel: Thanks for telling me. Any dizziness since you stopped?
Ms. Rivera: No, it went away.
Dr. Patel: Your blood pressure today is 148 over 92. Let's try a different medicine. I'll start amlodipine, 5 milligrams once a day.
Ana (daughter): Should she keep checking it at home?
Dr. Patel: Yes. Please log it each morning and send me the readings through the portal in two weeks.
Ms. Rivera: Can I get my flu shot today?
Dr. Patel: Yes, the nurse will give it before you leave. We'll decide at your next visit whether to adjust the dose.
Two Draft Notes
Note 1
Subjective: Hypertension follow-up. Stopped lisinopril about two weeks ago because of dizziness on standing; dizziness has resolved since stopping.
Objective: BP 148/92.
Plan:
- Lisinopril discontinued (dizziness).
- Start amlodipine 5 mg once daily.
- Home BP log each morning; send readings through the portal in two weeks.
- Decide at next visit whether to adjust the dose.
Note 2
Subjective: Hypertension follow-up. Reports dizziness on standing. Daughter present and asked about home monitoring.
Objective: BP 148/92.
Medications:
- Continue lisinopril 10 mg daily.
- Start amlodipine 5 mg once daily.
Plan:
- Home BP log each morning; send readings through the portal in two weeks.
- If home BP remains above 140/90, increase amlodipine to 10 mg.
Immunizations: Influenza vaccine given today.
What Counts as Critical
A critical error blocks the note, however complete the rest is: anything invented or contradicted about a medication, dose, plan, or decision, or a medication change recorded without its direction. A missing item alone is major, not critical.
As a group, decide whether each note is acceptable to sign.
Ask a Model
Open Stanford AI Playground, pick a model, and write down its name and version. Start a new chat for every run, so no one's chat history influences the answer. Paste the fictional visit and one note, followed by:
You are checking one draft clinical note against the visit transcript before a physician signs it. A critical error is anything invented or contradicted about a medication, dose, plan, or decision, or a medication change recorded without its direction. A missing item alone is not critical. Reply in exactly two lines:
ISSUE: quote the critical error from the note, or "none"
VERDICT: PASS (no critical error) or FAIL
Run it once for each note, each in a new chat. Expected: the shorter note PASS, the longer note FAIL.
Break the Judge
If the judge got both notes right, try to break it. Start from a copy of the note it passed and change one phrase at a time, rerunning in a new chat after each change:
- A miss: add a critical error and see whether the judge still says PASS.
- A false alarm: make a harmless change and see whether the judge now says FAIL.
Ideas to try: change a dose or a frequency; credit the daughter's words to Ms. Rivera; turn "decide at the next visit" into a firm plan; change when something happened; bury the error in longer, more polished text; swap a lay term for a technical one, or the reverse.
If the judge got a note wrong, fix it instead: change the prompt, not the note, until it gets both notes right, and write down what you changed.
Record:
- the model and version;
- your edit (before → after);
- the verdict before and after; and
- whether it was a miss, a false alarm, or a fix.
Then report it in the debrief.
If you finish early: rerun your best break with "Answer with one word: PASS or FAIL." in place of the two-line format. Did the verdict change?