First Peer-Reviewed Study to Evaluate AI-Generated Impressions Demonstrates Radiologist Preference for Domain-Specific Versus General Models

Multi-stakeholder evaluation finds domain-specific AI better aligns with clinical workflows, while general-purpose LLMs fall short in usability and clarity

SAN FRANCISCO, Calif., April 21, 2026 /PRNewswire/ -- Rad AI, the leader in AI-powered radiology workflow solutions, announced the publication of new peer-reviewed research in Nature Portfolio's npj Digital Medicine, demonstrating that domain-specific AI models better meet radiologist expectations than general-purpose large language models (LLMs) when generating radiology impressions, the most critical component of the report guiding patient care. Conducted in collaboration with researchers at Moffitt Cancer Center, the study represents the first comprehensive, multi-stakeholder evaluations of AI-generated impressions and highlights a broader shift in how healthcare organizations assess AI: not just on technical performance but on how well it supports real-world clinical workflows, communication and decision-making.

The analysis included 200 oncologic CT reports, comparing radiologist-authored impressions with outputs from a radiology-specific AI model fine-tuned on institutional data from radiologists and oncologists, and a general-purpose LLM. The domain-specific AI model performed in close alignment with human radiologists across key quality measures, including completeness, correctness and conciseness. Generic LLMs, however, were consistently deprioritized by radiologists, with differences driven by conciseness and overall clinical usability ranging from approximately 28% to nearly 50%.

The findings showed the domain-specific AI model generated high-quality radiology impressions significantly faster than general-purpose LLMs while maintaining the concise, high-signal structure radiologists rely on. The results also reinforced that purpose-built AI can meet clinician expectations in real-world settings when tailored to each radiologist's workflow. Patient harm ratings remained low across all impression types, with scores ranging from approximately 1.0 to 1.2 on a 3-point scale, where 1 represents no-to-minimal risk of harm.

"Impressions are the most critical part of the radiology report. This study demonstrated that, in addition to accuracy, customization matters to radiologists in order for them to feel confident that AI clearly and effectively communicates findings from impressions — and does so in a way that aligns with their clinical workflows," said Andrew Del Gaizo, Chief Medical Information Officer at Rad AI and co-author of the study. "These findings highlight the importance of AI that's purpose-built for radiology and designed to support how clinicians actually make decisions in practice."

In radiology, impressions serve as the critical summary that informs downstream care decisions, where clarity and prioritization in reporting are essential. This study provides a novel look at how both radiologists and oncologists evaluate the quality of those impressions in practice. By taking on the time-intensive work of drafting impressions, domain-specific AI allows radiologists to focus on the clinical judgment and decision-making that are central to patient care. These results reinforce that generating accurate text alone isn't enough, and the clinical usefulness of AI models depends on how effectively the information is communicated in practice. Radiologists consistently favored concise, high-signal summaries, while oncologists were more receptive to more detailed, explanatory outputs, underscoring the need for AI systems that can support different users across specialties.

"We saw meaningful variability in how radiologists and oncologists evaluated the same outputs, which has important implications for how health systems think about deploying AI," said Trevor Rose, MD, MPH, diagnostic radiologist at Moffitt Cancer Center. "The study shows that impression quality can be inherently subjective even with clinicians in the same specialty. Rather than optimizing for a single 'best' output, organizations should be prioritizing tools that can adapt to different users, workflows and clinical preferences."

As healthcare organizations continue evaluating AI solutions, the distinction between general-purpose models and clinically tuned systems is becoming increasingly important. Tools that align with real workflows and clinician expectations are more likely to deliver meaningful impact in practice, enabling radiologists to keep pace with growing demand while maintaining high standards of patient care.

See the full study.

About Rad AI

Rad AI is the leader in generative AI solutions for radiology, transforming the way radiologists work and improving patient care. The company's flagship product, Rad AI Reporting, streamlines radiology reporting with AI-driven technology and empowers radiologists to achieve greater efficiency, accuracy and satisfaction. Rad AI Impressions, the company's pioneering product, established Rad AI as an innovator in generative AI for radiology; as the first generative AI application to achieve widespread adoption in radiology, it's now trusted by thousands of U.S. radiologists. Rad AI Continuity, the most comprehensive solution of its kind, uses AI to help ensure patient follow-up for potential new cancers.

The company has been recognized as one of the most promising healthcare AI companies by CB Insights (Digital Health 50, AI 100) and AuntMinnie (Best New Radiology Software 2023, Best New Radiology Vendor 2021). In November of 2025, Rad AI was named the 36th fastest-growing company in North America on the Deloitte Fast 500 and the company was also named to CNBC's Disruptor 50. In March 2026, the company was selected as one of Fast Company's Most Innovative Companies.

Learn more about Rad AI at www.radai.com or on LinkedIn at www.linkedin.com/company/radai/.

Press Contacts

Alex Jenkins
120/80 MKTG for Rad AI
[email protected]

Heather Cmiel, APR
VP, Marketing and Communications for Rad AI
[email protected]

SOURCE Rad AI