Analysis limits. This is a commissioned narrative review of 82 studies with 22 authors and 80 references. It reports no new data, so there is no risk of bias table for the review itself. I read the full open access text. I checked the claims that carry its conclusions, not all 80 references. Author overlap was traced from the published author lists of the cited papers. Findings were put to two outside models (GPT‑5, Grok 4.3) with authorship removed; both rated nine of ten fair, and GPT‑5 asked for a closer check of the harm figure, which is below.
The review’s conclusion is sensible and hard to argue with. Models are improving faster than the evidence, the evaluations and the governance needed to deploy them. The most credible progress is on narrow tasks with clear endpoints. Broad clinical autonomy is premature.
It is a written version of the ARISE network’s January 2026 “State of Clinical AI Report,” a Stanford and Harvard collaboration, and reference 1 is that report’s Google Slides deck. Many of the studies it summarizes were written by its own authors. That is normal for people who lead a field. It also means the review is grading work its authors produced, and several of the errors below sit in exactly those places.
| Element | Detail |
|---|---|
| Citation | Worth JE, Perez A, Wu DJH, et al. State of clinical AI in 2026. BMJ Digit Health 2026;2:e000113. doi:10.1136/bmjdh‑2026‑000113. Received May 9, accepted August 31, published September 29, 2026. Open access. |
| Type | Narrative review. Commissioned, externally peer reviewed. No protocol, no registration, no quality appraisal of included studies. |
| Authors | 22, mostly Stanford, Beth Israel Deaconess and Harvard. Joint senior authors Rodman, Goh (corresponding) and Chen. |
| Study selection | Search of PubMed, medRxiv and arXiv, October 2025 to January 2026. Final set of 82 studies: 40 (49%) from the search, 31 (38%) nominated by the authors, 11 (13%) submitted by outside researchers solicited by email and social media. Inclusion and themes set by author consensus (“Delphi process”) over three meetings. |
| Scope | Six domains: model performance, benchmarks and evaluation, foundational methods, clinical workflows, patient facing AI, applied AI. Equity, regulation, privacy and implementation science explicitly excluded. |
| Reference mix | 80 references. 16 are preprints. 5 are company product announcements (OpenAI, Perplexity, Microsoft, Anthropic) or web pages. Reference 1 is a Google Slides deck. |
| Funding | No specific grant declared. The ARISE report was supported by Stanford, Harvard Medical School, BIDMC, the Shapiro Institute and the Blavatnik Institute. |
| Competing interests | Listed for 3 of 22 authors (Goh, Rodman, Chen). The statement is silent on the other 19. |
| The review says | The source shows | Verdict |
|---|---|---|
| “Across multiple randomised trials, clinician‑AI teams failed to outperform the AI model alone, despite consistently outperforming clinicians working without AI” (ref 40) | The first half holds. The second does not. A 2024 RCT by Goh, Hom, Rodman and Chen (four of this review’s authors) found LLM access did not improve diagnostic reasoning over conventional resources: 76% vs 74%, adjusted difference 2 points (95% CI −4 to 8). It is not cited. The cited Qazi trial found erroneous AI advice lowered accuracy by 14 points. | Embellished |
| Frontier models produced potentially harmful responses in “approximately 22% of primary care consultation cases,” later “approximately one in five” (ref 27, NOHARM) | The NOHARM abstract (v2 and v4) says severe harm “in up to 24.6% of cases.” Across systems the range was 2.9% (AMBOSS LiSA) to 24.6% (Llama 4 Maverick); the top general purpose frontier models ran 8.9% to 13.6% in v4. The cases are primary care to specialist eConsults. A worst case figure is presented as typical. Thirteen of this review’s authors are NOHARM authors. | Embellished |
| AMIE was non‑inferior to 20 primary care physicians in management reasoning and outperformed them on medication reasoning (ref 15) | Reference 15 (Tu et al., Nature 2025) is the diagnostic dialogue study: 20 PCPs, 159 scenarios, text chat. The management findings come from a separate Google preprint (Palepu et al., arXiv 2503.06074): 21 PCPs, 100 multivisit scenarios, the RxQA medication benchmark. The management paper is not cited. Tu et al. list text chat, “unfamiliar in clinical practice,” as a limitation; the review omits it. | Embellished |
| Physicians exposed to erroneous LLM advice scored 14 points lower (ref 51); this was the “first RCT‑grade” evidence of automation bias (refs 51, 52) | The number is correct (−14.0 points, 95% CI −18.9 to −9.1). It comes from Qazi et al., NEJM AI 2026, which is the review’s reference 79. Reference 51 is the same group’s separate AI literacy preprint. | Miscited |
| o3 reached 60% top‑1 and 84% top‑10 on difficult CPCs; physicians with unrestricted search reached 24% and 45% on 302 CPCs (ref 14) | Ref 14 is the authors’ own CaBot preprint (Rodman, Manrai, Brodeur). The physician figures match McDuff et al. (Google, 2023), 302 CPC cases, clinicians with search at 44.4% top‑10. Different case sets, different years, different conditions. A cross study comparison presented as head to head. | Mostly Solid |
| Table 2, “selected randomised controlled trials from 2025” | Includes TrialMatchAI (Abdallah et al., Nat Commun 2026), a development and validation study of a matching system. It is not an RCT. Two of the nine entries are 2026 publications. | Embellished |
| Fewer than 5% of 1,200+ FDA‑cleared AI/ML devices have undergone rigorous peer‑reviewed evaluation (ref 13, JAMA Summit) | Not located in the JAMA Summit abstract. Other estimates depend on the definition: 39.4% of 691 devices had peer‑reviewed data in premarket summaries (Lin et al., JAMA Health Forum 2025); 2.5% of 1,357 linked to registered trials and 3 tested patient outcomes (Abulibdeh et al., PLOS Digital Health 2026). | Not Verified |
| Penda Health and OpenAI study, 15 Kenyan clinics, 39,849 visits, 16% and 13% fewer diagnostic and treatment errors; later called “a large‑scale prospective trial” (ref 41) | Figures correct. A quality improvement study in which half of clinicians were randomly given the tool; errors were physician rated; patient reported outcomes did not differ. OpenAI funded the study. The review does not say so. | Mostly Solid |
| NHS stroke AI across 26 hospitals: 100% increase in thrombectomy, greater than at non‑AI sites (ref 72) | Correct: 2.3% to 4.6% at AI sites, against 1.6% to 2.6% (+62.5%) elsewhere; interaction OR 1.24 (1.08 to 1.43). Observational. Four authors, including the corresponding author, are Brainomix employees. Not noted. The functional outcome OR of 1.16 is not in the abstract and was not checked. | Mostly Solid |
| German mammography: AI users detected 6.7 vs 5.7 cancers per 1,000, +17.6% (CI 5.7% to 30.8%), no rise in recalls (ref 42) | Correct. Observational, 463,094 women, radiologists chose whether to use AI. Three authors are employees of Vara, the vendor. Not noted. | Solid |
| MAI Diagnostic Orchestrator reached about 80% on NEJM cases while cutting costs up to 70% versus o3 (ref 31) | Consistent with the preprint. The physician comparators worked without colleagues, books or internet, which the review does not mention. | Mostly Solid |
| Ambient scribe time savings were modest, “around 20 s per note” (refs 45, 46) | Direction right. Lukac et al.: Nabla cut time in note 9.5%, DAX showed no significant change. The 20 second figure is not in either abstract and was not checked further. | Mostly Solid |
| Delphi‑2M averaged an AUC of 0.76 across 1,000+ diseases (ref 29) | Correct, and the review repeats the developers’ caution against causal reading. | Solid |
| Endoscopist adenoma detection fell after routine AI exposure (ref 52) | Correct: 28.4% to 22.4% (−6.0 points, 95% CI −10.5 to −1.6), observational, four Polish centres. | Solid |
At least seven references that carry headline claims include review authors: the “superhuman performance” preprint (ref 2, nine review authors), the CaBot preprint (ref 14), the script concordance benchmark (ref 21), MedAgentBench (ref 26), NOHARM (ref 27, thirteen review authors), the GPT‑4 management RCT (ref 40) and the collaboration RCT preprint (ref 50). Reference 1 is the authors’ own slide deck. Thirty one of the 82 studies (38%) were nominated by the author team.
Why it mattersThe review does not grade the quality of any included study, so there is no external check on how its own papers are weighed. The two largest numerical errors above (the harm rate and the “consistently outperforming” claim) both sit in work the authors produced or ran. The limitations section acknowledges bias toward “well‑networked academic institutions.” It does not mention that the network is the authors’.
The review’s central discussion point is that human‑AI teams beat clinicians alone but not the AI alone. The authors’ own 2024 diagnostic reasoning RCT is the best known study showing that clinicians with an LLM did not beat clinicians without one. Including it would not change the conclusion that collaboration is the unsolved problem. It would remove the word “consistently,” which is the part a reader is likely to repeat.
Goh consults for Google, OpenAI, Samsung Research America, Roche Diagnostics, Novartis and others. Rodman reports funding from Google and Google DeepMind (with Moore, Macy, NIH, ARPA‑H). Chen reports an unrelated education software company and expert witness work. The other 19 authors have no statement.
What the text does not sayGoogle systems (AMIE twice, the Personal Health LLM) and OpenAI systems (o1, o3, HealthBench, the Penda study) are among the review’s most prominent examples. The review does not mention that OpenAI funded the Penda study, that Brainomix employees coauthored the stroke study, or that Vara employees coauthored the mammography study. None of this makes the summaries wrong. A reader weighing “100% increase in thrombectomy” would want to know who wrote it.
Four separate citation errors turned up in fourteen claims checked: AMIE management results attributed to the diagnostic paper, the automation bias result attributed to the wrong Qazi paper, a non‑randomized study in the RCT table, and an unlocated FDA statistic. The discussion also ends with “As that report argues” without saying which report. That is a high rate for a peer reviewed synthesis whose stated purpose is to be a reliable reference for clinicians and policymakers.
Half the included studies were chosen by the authors or by people who answered a social media request. One in five references is a preprint. No study was rated for quality, and simulated vignette studies sit next to multicentre trials with similar weight in the prose. The authors say plainly that this is a narrative review and not a systematic one. Readers should take its numbers as a reading list with commentary, not as pooled evidence.
- The benchmark to bedside gap is real and well supported: answers collapse when “none of the above” replaces the right option, accuracy drops when vignettes become dialogue (CRAFT‑MD, GPT‑4 from 0.82 to 0.63), and models do not know when they are wrong.
- The “optimisation paradox” finding (best individual agents assembled into a worse system) is a useful caution for anyone buying multiagent products.
- The deskilling and automation bias evidence is correctly flagged as early and important, even with the citation slip.
- The call for prospective, multisite trials with patient outcomes, and for postdeployment monitoring, is right.
- The review is candid that patients rate low accuracy AI advice as trustworthy, so safety cannot rest on patient judgment.
- How strong the evidence is for any single claim, since no study was quality rated.
- What the field looks like outside a Stanford and Harvard reading list, or in equity, regulation and implementation, which were excluded.
- How much human‑AI teaming helps in real care, as opposed to vignettes, since nearly all teaming evidence cited is simulated.
Medical ethics and privacy check. Clean. No patient, family member or clinical encounter appears in the review or this appraisal. No permission note required.
Related WAiR pieces. First, Do NOHARM (October 19) appraised the harm benchmark this review cites. The Signature Is the Loop (October 27) covers the same automation bias trial from the oversight side.
The conclusion is right. The best progress is narrow, collaboration is the unsolved problem, and autonomy is premature. The numbers on the way there lean in the direction of the authors’ own papers. A review that warns models cannot recognize their own errors could have used an outside reader for the same reason.
Sources
The review: Worth JE, Perez A, Wu DJH, et al. State of clinical AI in 2026. BMJ Digit Health 2026;2:e000113. doi:10.1136/bmjdh‑2026‑000113
ARISE report: “Clinical AI has boomed.” Stanford Medicine, January 15, 2026. medicine.stanford.edu
Omitted RCT: Goh E, Gallo R, Hom J, et al. Large language model influence on diagnostic reasoning: a randomized clinical trial. JAMA Netw Open 2024;7:e2440969. doi
NOHARM: Wu D, Nateghi Haredasht F, et al. First, do NOHARM. arXiv 2512.01241 (v2 and v4). arxiv.org. Range by system from WAiR’s October 19 appraisal.
AMIE: Tu T, et al. Towards conversational diagnostic artificial intelligence. Nature 2025;642:442‑450. doi. Palepu A, et al. Towards conversational AI for disease management. arXiv 2503.06074. arxiv.org
Automation bias: Qazi IA, et al. NEJM AI 2026;3(5):AIoa2501001. doi
CPC comparisons: Buckley TA, et al. Teaching large language models to reason like expert diagnosticians. arXiv 2509.12194. arxiv.org. McDuff D, et al. Towards accurate differential diagnosis with large language models. arXiv 2312.00164. arxiv.org. Brodeur PG, et al. Superhuman performance of a large language model on the reasoning tasks of a physician. arXiv 2412.10849. arxiv.org
TrialMatchAI: Abdallah M, et al. Nat Commun 2026;17. doi
FDA device evidence: Angus DC, et al. JAMA Summit Report on AI. JAMA 2025;334:1650‑1664. doi. Lin JC, et al. JAMA Health Forum 2025;6:e253351. jamanetwork.com. Abulibdeh R, et al. PLOS Digit Health 2026;5:e0001597. doi
Penda Health: Korom R, et al. AI‑based clinical decision support for primary care. arXiv 2507.16947. arxiv.org. OpenAI, “Pioneering an AI clinical copilot with Penda Health” (funding statement). openai.com
Stroke AI: Nagaratnam K, et al. Lancet Digit Health 2025;7:100927. doi
Mammography: Eisemann N, et al. Nat Med 2025;31:917‑924. doi
Ambient scribes: Lukac PJ, et al. NEJM AI 2025;2(12). doi
Deskilling: Budzyń K, et al. Lancet Gastroenterol Hepatol 2025;10:896‑903. doi
Optimisation paradox: Bedi S, et al. arXiv 2025. arxiv.org