Study finds retrieval-augmented AI can eliminate hallucinations in clinical LLMs while protecting patient privacy

A new study from Japanese researchers shows that retrieval-augmented generation can remove hallucinations in Large Language Models (LLMs), while still protecting patient privacy.

Published in npj Digital Medicine, the study considered whether retrieval-augmented generation (RAG) can enhance performance in radiology contrast media consultations.

A team of researchers led by Associate Professor Akihiko Wada from Juntendo University, Japan, developed a RAG-enhanced LLM and tested it on 100 synthetic cases involving iodinated contrast media, a common component in computed tomography imaging.

The consultations typically require real-time assessments based on factors including kidney function, allergies and medication history. The enhanced model was compared to three cloud-based AI and a standard LLM.

The RAG-enhanced model was found to completely remove dangerous hallucinations and responded faster than the cloud-based systems. 

“For clinical use, reducing hallucinations to zero is a safety breakthrough,” explains Dr. Wada. “These hallucinations can lead to incorrect recommendations about contrast dosage or missed contraindications. Our system generated accurate, guideline-based responses without making those mistakes.”  

“We frequently encounter complex contrast media decisions that require consulting multiple guidelines under time pressure," Dr. Wada adds. "For example, cases involving patients with multiple risk factors - reduced kidney function, medication interactions, or allergy histories. We realized that AI could streamline this process, but only if we could keep sensitive patient data within our institution.” 

The RAG-enhanced model works by retrieving relevant information from a curated knowledge base, including international radiology guidelines and protocols to ensure each response is grounded in up-to-date medical knowledge and not just pre-trained data.

The researchers say the technology could be applied to emergency medicine, cardiology, internal medicine, and medical education and could be used to support rural hospitals and other healthcare providers in low-resource settings.

“We believe this represents a new era of AI-assisted medicine,” Dr. Wada adds. “One where clinical excellence and patient privacy go hand-in-hand.”