Now we're starting with a really general solution, this model that is pretty good
at everything and the trick is how do we get this general model to focus up, right?
Like, focus in on medicine.
Be better at that specific task.
Hi everyone, and welcome to another episode of EMPlify
I'm your host, Sam Ashoo.
Before we dive into this month's episode, I want to say thank you for joining us.
I sincerely hope that you find it to be helpful and informative for your
clinical practice, and I want to remind you that you can go to ebmedicine.net
where you will find our three journals, Emergency Medicine Practice, Pediatric
Emergency Medicine Practice, and Evidence Based Urgent Care, and a multitude of
other resources, like the EKG course, the laceration course, interactive
clinical pathways, just tons of information to support your practice
and help you in your patient care.
And now, let's jump into this month's episode.
Hey, I'm Jack Teitel.
Great.
Thanks Jack for joining us on the podcast today.
I asked you to come on as a special guest because we've had a lot of interest
recently on artificial intelligence in medicine, and I cannot think of
someone with more expertise than you.
You have a pretty significant background in healthcare and artificial intelligence.
Tell me where that journey started for you.
Totally, well started all the way back in college for me.
I actually tripped and fell into an AI lab where we were doing a lot
of population health level stuff.
So looking at social media, tracking flu trends through Twitter, analyzing
real time outbreaks of food poisoning, again via Twitter, all that kind of
stuff and then that kind of, led to my passion in AI a nd also for healthcare.
From there, I went and actually started leading a AI lab at a hospital
network up in upstate New York.
This was originally a software lab that they called a innovation team.
Basically different department heads or researchers who wanted custom
software built and had a little bit of budget to spare, could come
to us, we would build it for them.
And they brought me in to see what AI was all about.
This is about 10 years ago now.
Mm-hmm.
Wow.
Yeah, this is about 10 years ago.
We started working AI, obviously at that time it looked very different than
it does now, a lot of machine learning algorithms, a lot of computer vision,
and so ran that team for about five years, grew it to about five of us,
worked in everything from predicting surgical outcomes, using chart data
to image analysis, helping providers read x-rays and ultrasounds better.
Doing some things like flow analysis of patients in the hospital, helping
doing analyses of how physicians interact with the medical record
to help optimize that process.
All sorts of things.
After doing that for five years, I switched over to Blue Cross Blue Shield
moved down south where it's a little bit warmer than upstate New York.
Yes.
Started working for Blue Cross Blue Shield doing things like fraud detection,
member profiling for claims overpayment.
And for the past five years, I've been working in just AI consulting.
General custom AI work for various companies, including a lot of health
tech companies, sports medicine, medical bill coding that kind of stuff.
The past year of that has been actually as my own company, Title AI.
So yeah, quite a long track where good in healthcare and to AI specifically
touching a lot of different points along the way also I used to teach as well.
Back when I was working in New York, I was adjunct at the local
university where I taught a course mostly based on my own experience for
PhD students on how to apply machine learning, deep learning, AI concepts
in the healthcare space effectively.
So not just how do you build good models, but how do you deploy them?
How do you validate them?
How do you make sure people are actually using them?
All that stuff that can slip through the cracks a lot of the time.
Now this is fascinating because I think most of the people who are listening
think of artificial intelligence as something that, was born like
in 2023, maybe the end of 2024.
But this has been a work in progress for decades.
Is that right?
Yeah I could talk about the history of AI for the whole podcast,
but I'll contain myself here and just do a little quick summary.
AI kinda started in the fifties and that was mathematicians like
Alan Turing, for example, some of you may have heard of Godfather of
AI, godfather of computer science.
The Turing test, right?
Yeah, exactly.
The guy who came up with the Turing test.
They just were theorizing about what AI could look like, what neural
networks could look like, what was the mathematical foundations of these.
That went being theoretical until about the nineties, roughly when
people started actually being able to implement neural networks.
And those are nothing like the AI we see these days.
They were five layers as opposed to a thousand layers and a thousand parameters
instead of a trillion parameters, which is the size we're looking at now.
So much smaller, really task specific but they worked and it was cool.
And then around 2000, late 2010's, 2016, 2017, that kind of era, is when we
started being able to run these on GPUs.
And that is when AI kind of took off because now it stopped being theoretical
or super simple and we started being able to build really serious models.
And that kind of culminated 2023 GPT 3.5 came out ChatGPT as everybody knows it.
And AI moved from this niche research space, where there were already a lot
of us implementing it for many years and getting really cool results, but it became
just like generally good at everything as opposed to having to be really
trained hard on task specific things and entered the public consciousness and
became AI as everybody sees it today.
Which is pretty cool.
Yeah, so again, you think in 2023 we finally hit that culmination where like
our tech level and our software finally just coincided in this miraculous
point where it was good enough to offer it to the general public as a
tool instead of just something that was in a closed lab at a university?
Yeah, it was the perfect storm because people had been working on
the software side of things since the fifties, that's all the theory.
The hardware side of things had finally caught up, right?
We'd been able to run things on GPUs for five, six years, we were finally
learning how to chain multiple GPUs together and train on these huge clusters
of GPUs and also the internet, right?
Having data to train these models on was really important.
These initial models are trained on all of the data on the internet.
That's how they know things about the world and without that data there to
train the models, it wouldn't matter how good our hardware is or how good our
software is, we still wouldn't have AI.
So this kind of perfect storm conditions happened and we were able to build these
models that are generally intelligent.
They were called foundational models.
So they're no longer super task specific.
They're pretty good at everything.
So that is when people paid attention.
It was like, wow, I can use this in my job and what I do every day
Mm-hmm.
And it wasn't just somebody in a really niche field with a fine tuned
model saying, I can use this model to do one specific thing in my day.
It was everybody in every industry around the world saying, oh, I can use this.
Yeah.
And that's why it blew up, ChatGPT reached a million users, I think four
or five times faster than the last app which I believe is Instagram.
Now I think that's still fascinating because I feel like we've gone from hey,
here's a cool tool that you can apply in any field to do anything and the
pendulum is swinging now as we applied it in healthcare and we went, oh okay.
It can help, but man, it sure makes a lot of mistakes.
And now the pendulum is swinging and people are going what you really need
is this AI that's tailored for medicine.
It's now task specific.
The pendulum seems to be swinging the other direction.
Yeah.
That is happening everywhere.
Being generally good at everything does not make you a good doctor.
Usually Usually these doctors have to go to school.
They have extra years of schooling and residency and all these things which
hyper specialize them to be good at this one specific thing: medicine.
You can't expect just an average person who's good at everything
to come in and be able to do that.
So what we're starting to see is exactly what we've been seeing before.
Basically, the way that I've always been doing AI is task specific, which
is still the best way to do things.
Mm-hmm.
But it used to be we have to build a very specific model architecture, get a very
specific set of data, and train it to solve a really specific set of problems.
So we're starting from nothing and building a really narrow solution.
Now we're starting with a really general solution, this model that
is pretty good at everything.
And the trick is how do we get this general model to focus up, right?
Like, focus in on medicine.
Be better at that specific task.
So instead of starting from nothing and building something narrow, we're starting
from something very wide and focusing it in, into kind of a focused beam.
Which is how a lot of folks have been approaching this and been seeing a lot
of success along with LLMs have some very foundational failings like memory
and knowledge and that kind of stuff.
So building structures around these AI systems to shore up those areas
where it's a little weaker, has led to some really big gains as well.
One of the things that's fascinating is that the large language models actually
have a really good command of language.
If you can give them something that then just needs to be summarized or
searched or understood and comprehended, that seems like a no brainer.
Really haven't seen any issues there.
It's more pointing it in the right direction for the content , that
seems to be the specific target that we're hitting for healthcare
now in these newer products.
Does that sound right?
Yeah, totally.
It's hard.
It's a hard problem to solve.
Think about how much more medical knowledge there is now than there
was a decade ago or 20 years ago.
How much more training doctors have to do and papers people have
to read to stay on top of things.
It's a hard task and the expectations for these models are high, right?
This is not just a general assist tool.
This is supposed give me really specific answers for really specific
edge cases in my specialty, right?
Yeah.
We're expecting the single model to perform as well as top tier specialists
and sort through all of medical literature on the internet to find the answer
. Yeah.
Easy.
Come on.
Yeah, easy, right?
Why isn't it better?
So yeah of course it makes mistakes.
It does hallucinate.
That's a foundational problem here.
But the question isn't, how close is this to perfect in my mind.
The question is always, what's the alternative?
What are we comparing this against?
Holding it to the standard of this needs to be perfect or else.
It's tough, right?
It feels good.
It feels like that's what it should be.
It's a robot.
It's a machine.
It should be perfect.
But as long as it makes an improvement over the current system, it's really good.
Now the tricky part comes in.
So in AI, we really talk, especially in healthcare,
about human in the loop, right?
Mm-hmm.
These models are really good, but they do make mistakes and have zero
accountability for their mistakes, right?
No one's suing the AI model for getting something wrong, they're suing you.
You have to be in the loop in supervising and monitoring these outputs.
But, is right 85% of the time, 90% of the time maybe.
So it's really easy for a doctor who is stressed and overworked and having
to do documentation and see a thousand patients every day to take the time to
sit there and meticulously check an answer that is right 90% of the time.
Hmm.
It's really easy for that to just slip through the cracks.
So in my mind, a lot of the work that still needs to be done is building
systems and policies and things around these tools so that everybody is
using the same tools and using them in the same way and using them in a way
that doesn't easily lead to mistakes.
Right now, I wonder if you have the same experience with this, but to me
it feels like the wild West out here.
Everybody is trying their own tools and coming up with their
own workflows, and some people do
Yeah.
a lot better than others, and some people do a lot worse, and
everybody's just trying to figure this out on their own, and that's
not really sustainable, you know?
Yeah and no one's really sharing their benchmarks publicly.
I couldn't tell you today that chat GPTs error rate is 15%.
Geminis is 8%.
Open evidence has a 3.5% We don't know, but it sure would be nice because we do
have some model for that in healthcare.
We never really quantify on an individual physician basis what the error rates are.
Like, I couldn't tell you what my error rate is compared to my partner.
But we can tell you that if you follow, say, an institutional guideline or
one of our organizational guidelines, on say, acute coronary syndrome.
You know what's your risk of a heart attack?
What's your risk of missing a heart attack if you have two normal blood
tests and a normal EKG and you've been observed in the emergency department?
Well, it's less than 1%.
So at least I can have that conversation with a patient and say,
"Hey.
We did these two blood tests.
They're very high sensitivity.
We did this EKG.
You have chest pain, but your chances are less than 1%.
You don't have very many risks.
You're looking at about this.
So you know, happy to put you in observation and do a stress and do
all these things, but it's not gonna really reduce that risk very much.
It's not a very valuable.
Use of your time and of our healthcare resources, but I'm happy to do
it for you if you're still super concerned." And most of those patients
appreciate that kind of quantification.
I don't really get that when I'm using a model and saying, "Hey, what's the
answer to this question?" It seems like it's very confident in itself, but unable
to tell me its own level of inaccuracy.
Totally.
Okay, there's so much to unpack there.
In terms of confidence, very interesting phenomena with AI.
It's not an accident.
So there's a couple things at play here.
AI AI mostly is trained on the internet.
I don't know if you've been on the internet, but people tend
to speak very confidently.
That's how the AI learned to speak, right?
It's a predictive model.
It samples from its distribution of what it knows, and what
it knows is the internet.
And the internet is very confident.
You don't see a lot of people just questioning life and being uncertain or
talking in like very statistical terms.
It is out there, but it's not the main thing out there.
That's right, my blog of uncertainty is not very popular.
Exactly.
So overcoming that is tough, just inherently.
And the second thing, which you may have noticed, it's not quite as much
confidence, but more reinforcement.
AI likes to tell you that you're right.
It does it a lot.
I'm sure you've seen, especially ChatGPT is the biggest offender of this.
Claude is a little bit better.
But ChatGPT you say something and it'll be, oh, what a great idea.
You are a genius.
This is why you're right.
And then you go, wait a minute, I'm not right.
And it's like, oh, wow.
You are so insightful.
Absolutely, I was wrong before.
Now you're right.
There's been studies that if you just prompt an AI to ask it,
should you flip your answer or not?
About 75% of the time it will, whether the original answer was right or not.
But the reason
Wow.
it does this is because they train it to.
People, it turns out, like hearing that they're right.
These AI systems are built to make money, right?
They want people to use them.
And so it's tuned for that engagement level.
They actually have a specific phase of training towards the end called
reinforcement based on human feedback, where they tune the model to give
answers that people prefer and people prefer to hear that they're right.
Some models, like I said, Claude is doing a lot better about mitigating that, but
it's just baked into human psychology.
That we like to hear that they're right and these models are built to cater to
us humans And so that's what they do.
Okay, back to the benchmark stuff, 'cause this is actually very interesting.
There are a lot of benchmarks out there.
You should always be very skeptical of benchmarks unless they are brand new.
The thing about AI systems, is they are very good at gaming things.
Very good at memorization.
Very good at fitting to a specific task.
Like a high school student?
Kind of, the AI systems will cheat wherever possible.
Cheat and memorize as much possible to get the best results on a specific task.
Which is called overfitting, that's the technical name for it, overfitting.
You're overfitting to the task.
And what happens when you do that is you become less good at everything else.
So you can think of it a s memorizing the answers to a test without
actually learning the content.
So when people release benchmarks publicly, there is incentive for
people to train the models to those benchmarks, and they might preform
Mm-hmm.
really well on those benchmarks and be kind of bad when you actually
go to use them in the real world.
This is something that Meta was a little bit notorious for, especially with
Llama 4 that they released last year.
Llama 4 came out, open source model.
Big splash in the AI community.
It was killing all the benchmarks, looking really good.
And then people went to use it and and they're like, wait a minute,
Yeah.
While it would be awesome to say like, lets get public benchmarks out
there to test these models and see how they stack up in the real world,
it's not really possible because the systems will become better at those
benchmarks and it's hard to know.
Are these systems better at those benchmarks because they've been
out for a year or two and all the AI developers know these are the
benchmarks that people are looking at?
Or because the models have gotten better.
This is like the AI can pass a board examination for a specialty
certification in some area of medicine, but it's still not a good doctor.
Yeah, exactly.
There's so much to being a doctor, but doing one test.
It can't encompass all there is.
So defining the intelligence of the system in a really huge area with
just a few questions isn't gonna work.
But more importantly, having one measure for that across everybody.
It's like if, everybody took the same test in the world and the test
questions stayed the same every time and then everybody knew what the test
questions were and knew if they did that, they would be hired into a job.
You're gonna study those questions.
You're not gonna how to do the job.
It's just kind of about this implicit reward and bias of these models and
these systems and how it's trained.
And there's not really much to do about it.
It's not like the developers are wrong for doing that, or we
should say, Hey, don't do that.
Again, it comes back to human nature and how we build out these systems.
So in my opinion, it's very important to have your own personal test.
What are the things that you do?
What are problems that you've tried to solve with AI where it didn't do good?
Or where it did do good?
If you're using these tools on a regular basis, it's not that hard to just take
one question you ask it a week and the answer that you liked or didn't like, put
that down in a spreadsheet or something, and then the next time a new system
comes out, run your 15, 20 questions through it, and then see how well does
this do versus the last thing I tried.
It a little while to build that up.
It's not something I would say take a week and do it all right now.
No, do it as you naturally work, but it'll really be helpful for just having
a little bit of empirical evidence.
Particularly related to you and the work that you do, that you can use
to judge these systems by and I think that again, comes back to these kind
of departmental policies, right?
Wouldn't it be great if your department had a set of benchmarks that they
don't release publicly that you use internally within your department?
You can't train on those, you can't tune on those.
You can't game that system and it tells you how well that model is gonna
work for your team in your area with the patient population that you have
and the problems that you deal with.
I I think having those kind of system level benchmarks, as opposed
to general global level benchmarks, are really the future to measuring
the success of these models.
But they take time and money to build and maintain and it's tough
to invest in that because you don't get an immediate reward, right?
It's this kind of slow rolling thing where like eventually you'll adapt better
models and improve better efficiencies, but it's tough to justify the budget for
those kind of things right now, I think.
Yeah, so tomorrow I'm gonna release the Sam Ashoo Benchmark System that's
completely black boxed and closed, and it's just gonna be a number
associated with the LLMs and no one's gonna know how I scored them or why.
They're just gonna have to take my word that, Hey, this one is performing
better than that one for today.
And I'll run it again in a month and I'll let you know if that's still a
live running, recurring benchmark score.
It's funny, you say it as a joke, but people are actually
starting to do things like that.
Yeah.
Some of these bigger companies there, there's whole companies now who
specialize in building benchmarks.
And they're starting to do more along those lines of we're not really
gonna tell you a lot of the details.
We're gonna tell you like the general concept that we're testing for and we're
gonna have a public testing set where you can use to measure yourself against.
And then we're gonna have a private testing set that is different and
we're gonna test it independently.
And that's kind of how it's gonna work.
But those are trust-based systems, which are tough in healthcare
Yeah, exactly.
That's right.
Trust me, it's okay.
Yeah, it's tough.
Doctors like to see evidence trails and papers and published results and
study criteria, and if you publish that stuff, then the AI is going
to be able to gain your system.
So it's a really a tough situation that we're in here.
And then some of that can be overcome in the system design, right?
So like we talked about how it might tailor a response because it knows
that people like to be reinforced or told that they're correct, but you
don't have to have that version, right?
It would be ideal to say, well, in healthcare I've got a bunch
of doctors using this system.
Can we just turn this feature off?
You know, can I just toggle that to off so that it just gives me an
honest answer and doesn't try to reinforce a yes or try and reinforce
my bias as I'm entering this question.
Is that not possible?
I love it.
The answer is no.
Not possible, but let's break it down as to why.
Okay.
There's two things at play in AI systems that are different and that
most people conflate as being the same.
AI models have a set of parameters you can think of if you think of an AI
model as a brain, which was the original inspiration for how these models work.
It's layers of artificial neurons that are all connected to each other and
then each connection has a weight.
You can think of it as a neuron firing.
It takes a certain amount of charge into a neuron and then it fires an electrical
signal, which passes to the next neuron, and then that continues along this whole
big network to take these raw inputs.
Which to us are our senses and to the LLM as text language and
translate them into outputs.
So those weights are what we train when we are building an LLM.
That is what is the difference between one model and the next.
That is those training phases I talked about where it's trained on all the data
on the internet, and then it's tuned on things like human reinforcement.
Those are baked into the weights.
You can't just turn those on or off.
It would be like deleting neurons in your brain to try and make your brain perform
better at one thing versus another.
It's just not really possible.
What you could do, is you could fine tune your own network, right?
That doesn't have those specific post-training steps or has
post-training steps more geared specifically to the things you want.
But turns out that is super, super expensive to do and to do well.
That's why there's only, maybe three or four frontier level AI models.
We have Claw, Gemini, Grock, ChatGPT that are all kind of state of the art and
you don't have Joe Schmo in his basement building his own state of the AI system.
It's pretty expensive
yeah.
to do.
Takes a lot of data.
Takes a lot of compute resources.
Takes a very long time to train these systems.
So the customization layer though is very interesting.
'Cause there's all sort of.
tools and harnesses and scaffolds and frameworks that you can put these models
into to make them more specialized and perform in a manner that you want.
So things like prompt optimization, right?
Giving the AI a persona, telling it explicitly things
that you want or don't want.
Giving the AI a memory.
So built into those weights, is a form of memory because it can
remember everything that it's seen.
But what if you want it to specifically focus on medical research as opposed
to weighting medical research as equally as Elon Musk Twitter posts
. You know, it happens in the AI system.
It considers all the information on the internet, not just a specific subset.
You can give it memory features.
These are systems that are built in between the user and the AI.
Where you ask a question and then you can imagine you have a whole database
of information and you can go and query that database of information
using your question and say, give me everything related to this question.
And then it can automatically pull all the stuff outta that
database related to the question.
Paste that right on top of the question, say, answer this question
according to this information.
And then you feed that to the LLM and now it has a really focused chunk of memory
that is directly associated to what you need from your set of resources that you
say, this is the stuff that I want you to pay attention to, and that we way we
can reinforce what we want it to know.
That's how systems like Open Evidence work is with those kind of intermediary layers.
They're generally referred to as RAG Systems: retrieval augmented generation.
And then there's all sorts of other things, like you can fine tune
models, you can do your own little training phase and tweak the actual
weights of the system to be what you want, although it gets really tough.
The more you tweak them, the further away they get from the general intelligence
and the more specialized they get.
So they might become better at one thing, but worse at everything else.
And then also these systems are moving so fast, right?
If you train your own system.
What happens next year when ChatGPT six comes out and it's
way better than your system is?
So you have to train that.
It becomes this kind of expensive game of maintenance and whatnot.
Dropping these large language models into these kind of scaffolds things
like RAG and things like, you know, enforcing citations on the system and
checking the output against the model.
There's a lot of just small steps before and after you get the output
from the LM system that you can use to verify, reduce hallucinations,
focus its intent on specific things.
That has been, in my opinion, the best way that we've adapted these models to
perform well in specialized domains.
And I think that's most of what you're seeing when you are seeing
Bob's AI for doctors or whatever new specialty tool comes out.
There's so many of them these days.
A lot of them are using those types of techniques as opposed to changing the
foundations of how the model is trained.
Yeah, I mean, so things like OpenEvidence, Doximity , DocsGPT, UpToDate's AI,
all of these guys have taken some form of a large language model and
just fed it or restricted it to their library, whatever their library is,
and said, you will search just this library and nothing else, and you're
not allowed to access the internet.
Just find the answer to the question from this library, and if it isn't
there, just say, Hey, it isn't here.
I don't have any evidence to answer your question.
Right?
It sounds simple, right?
It should be right.
It should be.
Trying to reign it in is proving to be more difficult.
Yeah, it's a lot of work.
You're fighting the nature of these systems a little
bit when you're doing that.
And there's a lot of techniques, and like I said, this is the wild West.
This technology basically hit the public three years ago.
These types of systems of controlling these and specializing
these models, two years ago.
This is all brand new stuff and AI moves fast and people are coming up
with new tools and techniques every day.
And the amount of people working on this is staggering.
And the knowledge sharing is excellent, but it's still a new tool.
And there are new systems and we're still figuring out
really what are the best ways.
I wouldn't say there's any established best practices for how to build these
systems or things like, this is the right to do it, this is the wrong way to do it.
We're still figuring it out as we go.
Like today in April, 2026, if a colleague came to me and said, Hey,
I'm going to use AI to answer this question or I want to do a study on
whether AI is helpful in the clinical setting with my residents or something.
I wouldn't recommend that they just run with out of the box GPT and have at it.
I say, use something that's actually tailored to medicine and
has some kind of medical library.
And not one of just the commercial, yeah, I use Gemini, you know, I'm
like, well, that's good for you, but I wouldn't trust anything that's
coming out of any of those models to treat a patient right now anyway.
I think there is, at least for the time being, an understanding that,
yeah, okay, there is like this general commercial product, which is great.
You wanna throw in some PDFs, crunch some numbers, have it spit out an
Excel sheet, whatever, that's great.
But if you're actually treating someone in front of you or putting
someone's health at risk, you need to be careful which one of these things
you're gonna put some trust in, right?
And even that still isn't gonna give you a confidence percentage
or anything of that sort.
Exactly.
I fully agree, but you know, which of the systems?
There's so many coming out.
Epic has some tools.
Claude Anthropic has tools specifically designed for clinicians.
ChatGPT is starting to get into that realm, although they're more on
the consumer health side of things.
There's all sorts of boutique tools that are coming out there.
Open evidence.
Which tool do you use and why, and how?
Right.
And how.
What are the cases where you should be using it?
What are the cases where you need to be checking it?
How do you check it?
These are systems, that at this point, people just learn by doing.
Yeah.
I bet that you didn't take a course on how to use these systems or have somebody
tell you like, oh, this is the best way.
You probably learned by doing it and talking to your colleagues
yeah, absolutely.
And different people figured out different things.
And that's not sustainable, right?
You can't establish real systems that work effectively just through
everybody figuring out on their own.
So to me that's where the real challenge lies is in adoption of AI systems.
There's a fun stat that came out from.
Ooh, I can't remember.
I think it was outta MIT, I'm not sure.
But they said 95% of AI pilots fail in deployment.
Most of the reason for that, it's not because the models aren't good.
These models work.
They are good, they give good results.
It's because people don't know what they're doing.
People the user?
Or people the programmers?
Or the people in between?
Which people are we talking about?
All of 'em.
All people.
Okay.
Not just the end users.
It's the folks who are developing the systems.
Don't think about the end users enough.
How many AI tools are just a new screen, a new tool, a new app that you have
to go out of your workflow, go load up another thing, copy and paste stuff
into, copy and paste the answer back.
Yeah.
There's a lot of friction in those systems, which just regardless of how
good they are, make them hard to use.
Doctors' times have already been eaten away by dealing with patients and
dealing with charting and all this stuff.
How much time do they have to open up a new screen outside their
system, learn to use a new app, copy and paste between systems?
It's a tough ask.
Yeah.
How do you validate these systems moving forward over time?
I have this great story, one of the very first AI tools I built, again, this
is, eight, nine years ago, whatever.
It's a great tool.
It was predicting surgical skilled nursing facility placement
after total joint replacement.
Who's gonna have to go to a nursing home for recovery
after hip or knee replacement.
And it worked great.
We were getting fantastic results for about a year until the system changed
and we went to bundle payments and the incentive structure for how people were
sent to these nursing facilities changed.
Almost overnight, our model went from very accurate, very predictive.
We were doing like intervention planning.
What happened if the patient stopped smoking or lost weight?
How would that change their outcome?
It It went from working really well to, it it didn't work at all.
But we had to completely retrain it to get it to work with a new system, and
that took time and it took monitoring.
If we weren't watching it, we never would have know that it didn't work.
A lot of people aren't thinking ahead enough as to model drift and
how priorities change over time.
What would happen if we had AI before COVID, right?
If AI had become popular five years before COVID and had been trained
to work in those types of hospital systems and then COVID hit and people
were relying on AI, the whole medical system was turned on its head.
The problems people were dealing with on a daily basis changed entirely and
it wasn't stuff that was out there on the internet for you to train on.
These AI systems would have gone from really useful to, you know
Garbage.
Yeah, exactly.
So friction in the system, how easy is it for people to actually adopt it?
How well is this system going to hold up over time and consistently
give you good results that you can have confidence in that those are
good results, and then training.
The number of people that I've seen open up a specialized AI tool, ask it
one question, not get the answer they like, and then never use it again.
It's staggering.
So many people try that, and usually it's not because the AI tool is bad.
It's because they didn't ask it the question in a good way.
Or they asked it something that it never stood chance of answering
or they didn't give it enough context around the situation.
So just knowing how to interact with these tools and utilize them
in the way that they were designed, to enable them to give you the good
results that they can, is skill and it's not one that's taught usually.
Usually when an AI system is deployed they show benchmarks that they've tested on.
They show, here's what we predict if everybody in the hospital uses
it 100% of the time, here's the amount of savings you're gonna get.
They deploy it and then they move on to the next system.
The
hospital, the next
AI system, the next thing.
Sticking around to do that maintenance, the training, the
monitoring, the dirty work, that is where all of the value lies, right?
You can build the best system in the world, but if nobody
knows how to use it or uses it regularly, doesn't really matter.
At least in our community hospital here, I'm starting to get involved in
like an AI curriculum for residents and physicians and teaching them more
about the differences, what models are all about, what prompts are all about,
and how the way that you ask a question can lead down a path that is gonna
give you an incorrect answer simply based on how you ask the question.
Which seems a little strange to say because you're dealing with a
model that has a command of human language that surpasses most humans,
and yet you still have to be careful how you phrase a question as though
you were speaking to a human.
Like I could go up to a resident and say, Hey, don't you think this person
is a little too high risk to be putting in observation for their chest pain?
And just by the nature of how I ask that question, that resident's gonna go,
oh clearly he thinks that he a person.
Let me reevaluate.
Lemme take a look.
Okay.
Yeah.
Okay.
I'm just gonna, I don't really know the answer, I'm just gonna
reinforce what he is saying.
So yeah , actually, you're right.
This person is too high risk and thank you for bringing that to my attention.
I'm gonna go change that right now.
When I don't expect that from an artificial intelligence LLM, but that is
the case How I ask a question matters.
Yeah.
I mean there's so much detail to communication.
First off, when you're communicating with a human being, it's not just text, right?
Unless you're literally texting them.
Which we all know, texting can lead to miscommunications plenty of times.
Yes.
You're dealing with voice inflection, cadence, facial expression,
body language, speed of speech.
These different factors beyond the words that you're saying are really important.
But also at the end of the day what's even more interesting, it's not just
if you ask a AI a question in different ways it'll give different answers.
You can ask it the same question, using the same language, and still
get different answers because these are probabilistic models at the end
of the day, they're not deterministic.
They don't always do the same thing, but what they basically do is you
feed it a list of words, which is your prompt, and then at the end, all it
does is try to predict the next word.
And then it takes that next word, adds it onto the list and
tries to predict the next word.
And it just does that over and over again until it gives you a response.
And the way it chooses that next word is it looks at all of the words it knows
and it says out of these words, what are the most likely to be the next word and
then it does a probabilistic sampling.
It doesn't always pick the top one.
It picks out of the top 10, 15, whatever, based on how likely each
one is to be the right answer.
So it can give you different responses.
Now, these days, that's not a big of an issue as it used to be a few years ago.
But still something to be aware of.
These are not humans.
These are not human brains.
These are probabilistic reasoning machines.
So the amount of information you give, it matters.
The more context and information you dump into that string, into that
prompt, the better it's gonna do because that's going to basically
clue in that probabilistic model.
Okay, what is the right thing to answer this?
If you just leave a vague question and you don't add enough
detail, it's not gonna know.
Said, the way you ask questions and the way you expect responses and
interact with these systems, again, very important just in terms of how
the system functions, how it's been trained, how it deals with information.
It's very tricky and there are some guidelines that you can build out, but it
is a little bit of a feel kind of thing.
Yeah and just to add another layer of complexity, there's the reference
or the knowledge base that it has.
Which if you're using one of these general models is updated,
you know, intermittently.
You don't know.
You could ask it.
You could ask Chad GPT, Hey, when was your knowledge base last updated?
And it'll tell you.
I think it was last month, I sat and I went through some of the commercial
ones, OpenEvidence and DocsGPT and a couple of others and some of the general
models and I just asked them about a medication I knew that the manufacturer
had withdrawn from the US market in the end of December of last year.
And I said, what is the adult dosing of this medicine?
That's all I asked and my assumptions in asking that question were, it's
knowledge base is probably not up to date enough to answer this
question and know it's withdrawn.
And second, it should at least know from my IP address or from my previous
conversations that I'm a US physician.
So I'm working in the United States.
And sure enough, out of like the five or six models I tested, only one of
them threw back an answer saying, here's the adult dosing but first you
should know this has been withdrawn from the US market as of December 24th.
And then it spit out the adult dosing because that's what I asked it.
All the others just gave me the answer and maybe suggested some follow up questions.
And I went, gosh okay.
And so then ,I went back and asked each one.
Okay, when was your knowledge base last updated?
And half of them told me, the other half said, yeah, that's not in my
programming to answer and I can't give you the answer to that question.
So again , it's one of those scenarios where you have to be like four steps
ahead to understand what the output is coming because ultimately it's
still a machine and if you don't know the parameters, how good it is, what
library it's accessing, when it's knowledge base was last updated, yada.
You take this answer and you kind of go, okay.
And on top of that, I don't know any physician who's verifying the output.
If it agrees with me, it must be right.
If it disagrees with me, then I'm gonna verify or start looking
at some of those citations.
And like we said, these models are tuned to agree with you.
That's their job.
Is to agree with you.
It's crazy.
And it's tough to get 'em to break outta that.
A lot of companies are trying to get 'em to break outta
that behavior and it's tough.
It's an interesting problem out there.
Hey, listen, we have taken up enough of your time.
I really appreciate you being on the podcast.
I would love to have you on here again to share your knowledge and to continue
to engage in this dialogue about artificial intelligence in medicine.
Tell me before we leave, if there's someone out there to looking for
a consultant and they wanna try and reach you, is there a webpage
or how did they get ahold of you?
Yeah, totally, linkedIn is always good.
I'm always active on there.
Just Jack Teitel, T-E-I-T-E-L on LinkedIn, or you could go to my
website, title T-I-T-L-E - ai.com, and there's a little contact form there.
Always happy to chat.
Fantastic.
Thanks Jack.
I really appreciate you being on the show, sharing your knowledge, and I
look forward to talking with you again.
All right, thanks Sam.
Thanks for having me on.
It's a pleasure chatting with you.
And that's a wrap for this month's episode.
I hope you found it educational and informative.
Don't forget to go to ebmedicine.net to read the article and claim your CME.
And of course, check out all three of the journals and the multitude of
resources avAIlable to you, both for emergency medicine, pediatric emergency
medicine, and evidence based urgent care.
Until next time, everyone be safe.