This is Naomi Diaz with the Becker's Digital
Health podcast. Here's your daily industry news briefing
for today, August 6.
First,
researchers from the Icahn School of Medicine at
Mount Sinai found that widely used AI chatbots
designed for medical support
are vulnerable to repeating and elaborating on false
health information.
Published August 2 in communications medicine,
the study tested how large language models respond
to fictional patient scenarios containing made up medical
terms.
The chat bots often accepted the false information
without question,
generating detailed responses about nonexistent
conditions, symptoms, or treatments.
Researchers then introduce a simple safeguard,
a one sentence warning at the start of
the prompt reminding the AI that the information
provided may be inaccurate.
With this added prompt, the rate of errors
drops significantly,
suggesting that even minimal guardrails
can help reduce misinformation
generated by AI systems.
The findings highlight the risk of relying on
AI chat box for medical decision making and
underscore the importance of prompt engineering,
system design, and human oversight.
According to the research team, the results support
the need to test AI tools using fabricated
inputs before they are deployed in clinical settings.
Researchers plan to apply the method to de
identified patient data and explore additional safety props
and retrieval tools as AI technology
continues to evolve.
Second,
the venture capital arms of UPMC and UChicago
Medicine have invested in a health care AI
startup as a part of a $55,000,000
series c funding round.
UPMC
enterprises and UCM ventures back all Ultramix,
which says it offers its first FDA cleared
Medicare reimbursed
AI technology
to aid in detecting cardiac amyloidosis
and heart failure with preserved ejection fraction.
The company developed the tool in collaboration with
Mayo Clinic.
UltraMix other health system partners include university hospitals
and Northwestern
Medicine.
The startup says this platform helps hospitals cut
unnecessary
tests, enhance workflows, and start treatments earlier.
Third,
health systems are increasing their adoption of Microsoft's
Ambient AI for clinical documentation.
Here are four things to know from a
July 30 Microsoft earnings call.
First, Microsoft had a breakout physical 2025
for the use of its Dragon co pilot,
AKA DAX, which ambiently listens to patient visits
and drafts clinical notes for the EHR, said
Microsoft chair and CEO, Sato Nadala.
Second,
health care customers employ the tool to document
over 13,000,000
patient encounters in the most recent quarter, up
nearly seven times year over year.
Third,
at Mercy, more than 1,000 physicians are now
using Dragon Co pilot.
Four, mister Nadella said of Mercy, quote, they
have saved more than a 100,000
to date and plan to expand to all
5,000 providers.
As one physician put it, the best thing
to happen to my practice in ten years,
end quote.
If you would like the latest health IT
and digital health industry news delivered to your
inbox every morning,
subscribe to the Becker CIO and health IT
review e newsletter through our website at ww.beckershospitalreview.com.