By H. Omer Aktas
Editor, AIUpdateWatch.com
Published July 27, 2026 · Approximately 21 minutes
A quiet risk
The reassuring answer at the wrong time
It is after midnight. A man has had a cough and fever for two days. He asks a chatbot whether he can wait until morning.
The chatbot gives a calm answer. It says many respiratory infections improve with rest, fluids and time. The reply is clearly written and sounds reasonable.
But the man had also mentioned that breathing had become harder during the past hour. That detail appeared in the middle of a long paragraph. The chatbot did not give it enough weight.
The answer it gave
General information
Many ordinary infections can be managed at home.
The decision that mattered
Urgency
New or worsening difficulty breathing can require immediate medical assessment.
The first statement may be broadly true. It can still be the wrong answer for the person asking.
A health answer is not safe merely because its facts are generally correct. It must also fit the person, the timing and the level of risk.
“A health question” can mean five very different jobs
People often speak about AI health advice as though it were one activity. It is not. Asking what a word means is very different from deciding whether chest pain can wait.
Explain
Translate a medical term or explain the general purpose of a test.
Organize
Turn symptom notes into a clear timeline for an appointment.
Compare
Describe general differences between treatment approaches or test types.
Diagnose
Decide which disease is causing a person’s symptoms.
Direct action
Tell someone whether to delay care, change a dose or stop a medicine.
The risk rises sharply when the answer changes what someone does next. A weak explanation may confuse. A weak triage decision may delay urgent care.
The World Health Organization recognizes possible uses for large multimodal models in patient questions, clinical care, education and research. It also warns that these systems can produce false, inaccurate, biased or incomplete statements, and that people may overlook errors because the system sounds authoritative.
Where AI can genuinely help
A cautious health workflow does not require avoiding AI completely. The technology is often useful when it supports understanding rather than replacing medical judgment.
Translate medical language
Ask for a plain-English explanation of a term in a report, then confirm the meaning with the clinician or official source that produced the document.
Prepare for an appointment
Turn scattered notes into a timeline: when symptoms started, what changed, what makes them better or worse, and which medicines are being taken.
Generate questions
Ask what questions a patient might discuss with a doctor after receiving a diagnosis, test result or new prescription.
Explain general concepts
Learn the broad difference between a screening test and a diagnostic test, or between a side effect and an allergic reaction.
Improve instructions
Rewrite a complicated set of non-emergency instructions into a checklist without changing the facts or adding new advice.
Support record keeping
Create a symptom diary template, medication list or blood-pressure log that can later be reviewed by a professional.
A safer kind of request
Explain this term in plain language. Do not diagnose me or recommend a treatment. Tell me what parts of the explanation I should confirm with the clinician who ordered the test.
This keeps the model in an educational role. It asks for clarity while preserving the professional decision.
What the chatbot cannot see
A person may describe a symptom accurately and still leave out the detail that changes the decision. A clinician gathers information actively. A general chatbot usually receives only what the user chooses to type.
Skin colour, distress, confusion, swelling, posture and how ill someone looks.
Temperature, oxygen level, blood pressure, pulse, glucose and other vital signs.
Breath sounds, tenderness, strength, reflexes and findings that require trained examination.
Pregnancy, age, allergies, previous conditions, recent surgery, travel and family history.
Prescription drugs, non-prescription medicines, supplements, alcohol and missed doses.
Whether the person is stable, improving gradually or becoming worse by the hour.
Even a model that asks follow-up questions cannot perform an examination or guarantee that the answers are complete.
The chatbot sees a description of the patient. It does not see the patient.
A symptom checker is not a diagnosis
Symptoms rarely point to only one condition. Headache, fatigue, nausea and dizziness appear in many illnesses. The same symptom can be minor in one setting and urgent in another.
A chatbot often generates a list of possibilities. That list can be useful for learning vocabulary, but it does not establish which condition is present.
A condition could produce these symptoms.
The symptoms appear to fit part of that condition.
A professional considers it after history and examination.
Evidence is weighed, alternatives are considered and tests may be used.
AI can move quickly through possibilities. It cannot safely skip the evidence between possibility and diagnosis.
This distinction also protects against “diagnosis shopping.” A person can keep rephrasing a question until the model returns the condition they already fear or hope to see.
The two most dangerous answer patterns
Error one
False reassurance
The model notices a common harmless explanation and underweights a warning sign. The person delays care because the answer sounds calm.
- “It is probably stress.”
- “This usually goes away.”
- “You can monitor it at home.”
These phrases become dangerous when the system has not ruled out urgent causes.
Error two
Worst-case escalation
The model lists rare serious diseases without explaining likelihood or context. The person becomes frightened and may pursue unnecessary tests or treatment.
- A common symptom is linked to a rare cancer.
- A normal variation is described as a warning sign.
- A long list of possibilities is mistaken for a ranked diagnosis.
More alarming possibilities do not make an answer more thorough.
A safe answer should manage both risks: it should not dismiss important warning signs, and it should not present remote possibilities as probable facts.
Medicines and dosage questions need a harder boundary
Medication advice is especially sensitive because the correct action can depend on age, weight, kidney or liver function, pregnancy, allergies, timing, formulation and other medicines.
Understanding the label
Ask what “take with food,” “extended release” or “as needed” generally means, then follow the actual prescription and pharmacist’s instructions.
Possible interaction
Use AI to create a question for a pharmacist, not to decide that two medicines are safe together.
Changing treatment
Do not rely on a chatbot to increase, reduce, split, combine or stop prescribed medicine.
Overdose or serious reaction
Contact local emergency or poison-control services rather than waiting for a chatbot conversation.
A chatbot may confuse two medicines with similar names, overlook a formulation difference or apply an adult rule to a child. It may also give an apparently precise dose without knowing the clinical facts needed to calculate it safely.
Never treat a confident dosage number as safe merely because the calculation looks exact.
When to stop asking the chatbot and seek help
AI should not become an extra step between a person and urgent care. When the situation may be time-sensitive, stop refining the prompt.
Seek immediate help
Contact local emergency services when there may be an emergency.
Examples include, but are not limited to:
- severe or worsening difficulty breathing;
- new chest pressure or severe chest pain;
- sudden facial droop, weakness, confusion or difficulty speaking;
- loss of consciousness, seizure or inability to wake normally;
- severe bleeding;
- signs of a severe allergic reaction, such as swelling with breathing difficulty;
- a suspected overdose or dangerous poisoning;
- immediate risk of self-harm or harm to another person.
This list is not complete. A person who feels seriously unwell, is rapidly worsening or is uncertain about an urgent situation should seek professional help rather than waiting for certainty from a chatbot.
Immediate danger, severe symptoms or rapid deterioration.
Prompt same-day assessment may be needed even when the situation is not clearly life-threatening.
Stable questions can be prepared for a normal appointment, pharmacist or official health service.
One of the hardest medical decisions is choosing the correct level of care. That is precisely where a general-purpose chatbot should not be the only decision-maker.
How to ask a health question more safely
The wording of the request cannot turn a chatbot into a doctor. It can, however, reduce some common mistakes and keep the answer in a supporting role.
Name the task
Ask for explanation, organization or questions—not a definitive diagnosis.
Separate facts from assumptions
Tell the model not to invent missing details and to list what information is absent.
Ask for warning signs
Request general red flags that would make professional or emergency assessment more important.
Require source types
Ask for official public-health, hospital, medicine-label or professional guidance rather than anonymous articles.
Preserve the decision boundary
State that medication changes and care decisions will be confirmed with a qualified professional.
Example
Help me prepare for a medical appointment. Organize the symptoms I provide into a timeline and suggest questions I can ask. Do not diagnose the condition, tell me to change medication or assume missing facts. Highlight any details that may require prompt professional attention.
This produces a useful document without asking the model to make the final clinical decision.
How to check a health answer
Health claims should be checked against a source suited to the question. A long list of links is not enough. The source must actually support the claim being made.
- 01
Identify who is responsible
Prefer official health agencies, recognized medical institutions, medicine regulators and professional organizations with named accountability.
- 02
Check the date
Guidance, recalls, approved uses and treatment recommendations can change. Old information may still look polished.
- 03
Open the original source
Confirm that the source exists and says what the chatbot claims. Do not rely on a generated quotation or summary alone.
- 04
Look for the intended audience
Research findings, professional guidance and patient instructions serve different purposes and cannot always be substituted for one another.
- 05
Ask whether it applies to this person
General information may not apply because of age, pregnancy, other conditions, medicines or test results.
The U.S. National Library of Medicine recommends checking who runs a health site, why it exists, where its information comes from, whether experts review it, whether it is current and how it handles privacy.
Your health question is also a privacy decision
People often type information into a chatbot that they would hesitate to post publicly: symptoms, diagnoses, fertility history, medication lists, mental-health concerns, insurance details and photographs.
Do not assume that a consumer chatbot has the same privacy obligations as a doctor, hospital or health plan. In the United States, the Federal Trade Commission notes that many consumer health apps are not covered by HIPAA, even when they collect highly sensitive information.
Remove or generalize
- full name and exact date of birth;
- address, phone number and email;
- insurance, patient or national identification numbers;
- faces and identifying details in photographs;
- names of clinicians, employers or family members;
- complete unredacted medical records.
Check before sharing
- who operates the service;
- whether conversations are stored;
- whether data may be used for training;
- whether information is shared with third parties;
- how deletion works;
- whether the tool is provided by your health organization.
A health detail can be useful to the answer and still be unnecessary to identify you.
A general chatbot is not the same as an authorized medical device
The phrase “medical AI” covers very different products.
General-purpose chatbot
Broad conversation
Designed to answer many kinds of questions. Its health responses may not have been reviewed for a specific medical use.
Clinical support tool
Defined task
May help a professional perform a particular workflow, using specified data and controls.
Authorized medical device
Reviewed intended use
In the United States, an FDA-authorized AI-enabled medical device has met applicable premarket requirements for its specific intended use.
Authorization does not mean a device is perfect or suitable for every purpose. It means the product, intended use and evidence were reviewed through an applicable regulatory pathway. A general chatbot should not borrow that credibility merely because both products use AI.
Ask a simple question: What exact medical task was this tool designed and evaluated to perform?
A practical seven-step workflow
- 1
Check urgency first
When symptoms may be severe, sudden or rapidly worsening, contact professional or emergency services before using AI.
- 2
Choose a supporting task
Use the chatbot to explain, organize or prepare questions—not to replace diagnosis or treatment.
- 3
Minimize personal data
Share only the information needed for the educational task and remove identifying details.
- 4
Expose uncertainty
Ask what information is missing, what assumptions were made and which parts should not be relied upon.
- 5
Verify the source
Open the official guidance, label or original publication and check the date and wording.
- 6
Confirm the personal decision
Discuss diagnosis, medication, treatment and care level with the appropriate qualified professional.
- 7
Watch for change
A previous answer becomes irrelevant when symptoms worsen, new warning signs appear or the person’s condition changes.
The bottom line
AI can help you prepare for care. It should not become a reason to postpone care.
The strongest health uses are often modest: understanding a word, organizing a timeline, preparing questions and checking whether an explanation is clear.
The weakest uses are often the most consequential: deciding that a serious symptom is harmless, selecting a diagnosis, changing medication or calculating a dose without the full medical picture.
When an answer affects whether someone seeks care, takes medicine or waits, the final check must come from a source that can see the person, accept responsibility and act when the situation changes.
Sources and further reading
- Google Trends — Artificial Intelligence Search Trends
- World Health Organization — Ethics and governance of artificial intelligence for health: guidance on large multimodal models
- World Health Organization — Potential benefits, risks and governance recommendations for health AI
- U.S. National Library of Medicine — Evaluating Health Information
- U.S. Food and Drug Administration — Artificial Intelligence-Enabled Medical Devices
- U.S. Federal Trade Commission — Mobile Health App Interactive Tool and health privacy overview