A Safety Proposal for AI-Enabled Medical Devices

WASHINGTON, May 20, 2026 /PRNewswire/ -- Paragon Health Institute, a leader in health care research and market-based policy reforms, has published Generalization Uncertainty in AI-Enabled Medical Devices: A Safer Way Forward, a major research paper proposing a new framework for one of the key issues in the safety debate over artificial intelligence (AI) in health care.

AI medical devices are increasingly important to patient care as well as U.S. global leadership. These devices often perform well in testing but less predictably when used on real patients whose information (e.g., X-rays, CT scans, mammograms, etc.) differs from the data used to teach those devices. The ability of an AI device to work as well in the real world as it did during development is called generalization.

However, generalization is not a foregone conclusion for AI devices. The central policy challenge is to avoid mandating an ineffective remedy to generalization uncertainty in the pursuit of improved safety. Failing to solve this challenge would prove detrimental to the advance of life-saving technologies, the lives of patients, and the future of American health care.

The limitations of current remedies include the potential for high consultative costs as well as risk assessments that are not personalized for individual patients. High consultative costs are a particular concern, because they encourage a divide between well-financed health systems that can afford consultation versus rural health systems and safety net providers that cannot.

Paragon's paper proposes a solution—a voluntary framework called Digital Similarity Analysis (DSA). DSA will evaluate the similarity of an individual patient's information to a device's training and testing information. The purpose of DSA is to determine if a patient's information is an outlier prior to the use of an AI device. The physician, when alerted by DSA to an outlier scenario, can decide to:

• forgo device use because of the perceived risk,
• require supplemental validation of the medical device's output, or
• use the device but treat its output with lower confidence.

Although the DSA proposal would not eliminate generalization uncertainty, it could provide valuable guidance to physicians and guardrails for AI medical device safety, while avoiding alternatives that inadequately address the problem. The approach, when implemented by device manufacturers, would preserve the confidentiality of the manufacturers' training data, a key resource in AI development. Furthermore, DSA would expand the discussion of algorithmic bias beyond broad demographic categories to the specific characteristics of each patient. By shifting evaluation from population groups to individuals, the DSA approach may enhance safety across demographic segments.

"Generalization uncertainty is a critical issue for health care AI," remarked Kev Coleman, Director of Paragon's Health Care AI Initiative. "The DSA proposal is a contribution toward that need and, when complemented with targeted postmarket surveillance, could provide a powerful architecture to evaluate AI safety while preserving the technology's potential to improve patient lives and enhance the effectiveness of our entire health care system."

This paper is the latest work associated with Paragon Health Institute's Health Care AI Initiative. This initiative explores ways health care AI can be used to accelerate life-saving innovations, fight waste, empower patients, and reduce costs. Paragon's other notable AI papers include Targeted Postmarket Surveillance: The Way Toward Responsible AI Innovation in Health Care, Healthcare AI Regulation: Guidelines for Maintaining Public Safety and Innovation, and Lowering Health Care Costs Through AI: The Possibilities and Barriers.

Launched in late 2021 by Brian Blase, Paragon Health Institute provides health policy research as well as market-based policy proposals for improved outcomes in the public and private sectors. A 501(c)(3) non-profit, the organization is funded by donations from foundations and individuals. Paragon does not accept any funding from industry and does not engage in lobbying efforts. Journalists and health care analysts can review Paragon's latest studies and commentary at paragoninstitute.org/research/.

Contact:
Anthony Wojtkowiak
[email protected]
703.527.2734

SOURCE Paragon Health Institute