Clinical Evidence is Growing Rapidly, and AI May Make It Even Harder to Interpret, New MedINT Research Shows

Real World Human-LLM Interactions – Prospective Blinded versus Unblinded Expert Physician Assessments of LLM Responses to Complex Medical Dilemmas

TEL AVIV, Israel, March 17, 2026 /PRNewswire/ -- A new peer-reviewed study published in the March edition of PLOS Digital Health offers one of the first real-world evaluations of how physicians assess AI-generated clinical content in everyday practice. The findings reveal that large language models (LLMs) frequently miss critical clinical nuances when addressing complex medical queries, sometimes sounding convincing while providing incomplete, misaligned, or irrelevant evidence.

The results emphasize the need for a new approach in clinical AI development centered on transparency, verifiable literature, and human oversight. The study, titled "Real World Human–LLM Interactions – Prospective Blinded versus Unblinded Expert Physician Assessments of LLM Responses to Complex Medical Dilemmas," was conducted by researchers at Soroka University Medical Center in Be'er Sheva, Israel, in collaboration with the clinical team at MedINT. It compared leading AI models with trained human researchers as they analyzed real-world complex clinical dilemmas.

When AI Sounds Like an Expert but Misses the Evidence
While AI tools can provide accurate advice in simple cases, such as managing a sore throat, they struggle when clinical complexity increases. In one case, a pregnant woman with a rare blood-clotting disorder faced anesthesia risks during a scheduled cesarean section. Determining whether to administer medication before proceeding required synthesizing data across multiple medical domains, a task LLMs struggled to perform effectively.

In this and other cases, LLMs produced responses that sounded authoritative but cited literature unrelated to the clinical question or misinterpreted key laboratory values. Based on this study, physicians are not always effective gatekeepers for data quality.

"LLMs can produce fluent, confident answers that feel reassuring, but confidence is not a marker of correctness," said Dr. Itamar Ben-Shitrit, the study's lead author. "In complex clinical scenarios, small details matter. When those details are missed or misinterpreted, the entire recommendation can shift in the wrong direction. That's exactly why we need transparent systems that enable human validation, not blind trust."

The Gap Between Confidence and Quality 
Researchers identified a critical disconnect between perceived and actual quality. Physician satisfaction with AI outputs did not correlate with factual accuracy or clinical appropriateness. In some cases, AI-generated citations were fabricated or misaligned with the question.

"AI systems can sound confident and convincing, but that doesn't always mean they're correct," said Sigal Ben-Ari, PhD, Vice President of Product at MedINT. "For clinicians to truly trust AI, they must be able to see where the information comes from. Transparency about sources allows physicians to validate, authenticate, and understand the evidence behind every recommendation."

Building Decision-Support Tools to Elevate Clinicians 
The findings reinforce MedINT's philosophy that AI should enhance, not replace, clinical reasoning. MedINT's platform integrates AI with transparent, human-centered validation tools that help clinicians verify sources and patient-specific factors in real time, ensuring that every recommendation supports expert judgment rather than shortcutting it.

About MedINT 
MedINT helps clinicians manage complex, multidisciplinary cases by embedding transparent, human-centered AI into clinical workflows. Its solution ensures that physicians remain fully informed and engaged throughout treatment planning, reinforcing clinical judgment and contextual relevance.

Read the full study: PLOS Digital Health, March 2026 

Media Contact: 
Itamar Ben-Shitrit, Hadas Sasson-Zitomer
Email: [email protected]
+1-866-568-4040
Website: www.medint.ai

SOURCE MedINT