Hello, everyone. This is Jacob Emerson with the
Becker's Payer Issues podcast.
Thrilled today to be joined by doctor Nathan
Kronk, who is the manager of AI engineering
and operations
at Premera Blue Cross. Nathan, thanks so much
for taking the time to be with me
on the podcast today. My pleasure. Thanks for
having me. Yeah. And before we dive into
everything we wanna talk with you about in
a special new project that's been happening at
Premera, can you tell us a little bit
more about yourself and what it is that
you do at the company?
Sure. Yeah.
So I have a my background is in
academia. You know, I I I was researching
AI,
you know, back in 2016 for my PhD,
and,
it kinda sort of taken off, making some
big splash out in the world. And I
thought, you know, maybe I'd like to get
out of the, the the rubber room, the
ivory tower of the the the crazy space
of the academics and go make an impact
with some of this stuff. And health care
seemed like a really,
you know, it transcends race, gender, ethnicity, politics.
Everybody needs health, so it seemed like a
nice place to go try and make a
contribution. So
I joined, Premera, you know, about three years
ago,
and I've had the really enjoying opportunity like,
the enjoyable opportunity to help kind of encourage
and engender this this, adoption of AI technologies
by I go do, like, AI roadshows around
the company, presentations,
you know, on jailbreak attacks, you know, trying
to help people become familiar with, the capabilities
of this technology, building prototypes to show the
art of the possible.
It's been a really fun role that I've
had had here at the company.
But since then, we've actually moved from, you
know, roadshows in education into production. We've got
some pretty impressive,
governance standards in place. We've got, you know,
an AI security gateway that's,
pretty cutting edge and some real Gen AI
products in production making difference in the health
care space. So,
that's a bit about me.
Fantastic. Well, like I said, we really appreciate
you taking the time to be with us
today.
And let's talk about some of those generative
AI products that have been happening at the
company.
There's a new generative AI chatbot called Alice.
So can you walk us through what exactly
that is, how it was developed, and some
of the timeline from concept to the actual
rollouts? Ultimately, can you give us that sense
of of scale in terms of what what's
happening with this at the company?
Yeah. There are some really important details,
to make to make these timelines make sense.
Right? So
about two and a half years ago, we
saw
I mean, you know, my myself and my
team, we came from academia. So we kinda
knew what the cutting edge tech was and
what the art of the possible was. And
we saw this agentic
future coming, or at least we believed that
it was coming, and we started,
building an infrastructure very early on. So some
of the timelines and things I'll share with
you sound pretty quick, but I wanna, you
know, couch them in the the reality, which
is that we started building an AI infrastructure,
for the company that was reusable and reproducible
about two years ago.
So but the the reality is is the
CSR team, you know, they had, the Dynamics
three sixty five tool,
advertised to them as a nice tool to
help support the CSRs in finding information quicker.
And they they went and tried it out,
and it unfortunately,
right, very early on, these these generalist AI
solutions that try and, like, boil the ocean
for everyone,
they they can they can work, but not
fantastically. Right? I mean, when you try and
have a generalized,
you know, the vector database, right, which basically
takes any type of data of any form
and chunks it, partitions it, makes it searchable
for your AI agents,
you're losing a lot of domain expertise that
can be exploited,
by people who really know what the application
is is for. So that was where we
came in. They they came to us and
said, hey. We're trying this Dynamics thing. It
isn't quite working. Why why not? And we
said, well, it's not because there's a lot
of idiosyncratic domain expertise that needs to be
baked into the chunking parse processes and and
how the text is formatted and how the
agent reasons over it. Let's let's show you.
And they said, okay. So, honestly, it took
us about a week to make a prototype
given that we had the infrastructure and everything
in place. And we had a, a reasoning
agent that was built on a technology called
the LLM compiler,
and it searched through thousands of methods and
procedures and would give you answers, would reason
over them and give you answers. And we
said, this is the kind of thing, that
we can do. And they said, great. Let's
let's build this out and go into production
with it. So,
it took us about three months to get
into,
product like, the UAT phase. So that was
when we rolled it out to a couple
of different lines of business to start experimenting
in a small in some small ways. And
then over the past about two to three
months, we've been slowly rolling it out to
more and more, and, you know, we're up
to hundreds of CSRs that are now using
using this tool to help them find methods
and procedures information, more quickly to help members
and and people who call in.
Wow. No. I mean, that's amazing. So a
lot of time went into this, but in
terms of the infrastructure, but then it sounds
like it really a lot of it came
together quickly, once you started developing that prototype.
You mentioned, you know, this is for customer
service operations.
So can you give us a sense of
some of the impact you've seen so far,
what you've heard on the ground,
from from people actually using this tool?
Yeah. That's, that's some of the most rewarding
part of it here. I mean, you know,
we are we love the AI and the
building stuff, but that's you know, hearing the
impact on the front lines has been great.
Yeah. There's there's so many quotes that we've
heard. Firstly, you know, just allowing the CSRs
to hear that the members are happier, like,
on average because they can get information faster
so that they don't sit there and get,
you know, frustrated on the phone. Well, please
give me a minute to go look this
up. And, you know, they're they're calling in
on their lunch breaks or or maybe, you
know, in a break in work. They don't
have a lot of time. That that's a
bit of a luxury. Right? So,
for the CSRs to be able to get
them answers faster has been,
rewarding both for the members and then also
the CSRs to give them more job satisfaction,
which is which has been pretty great.
The the leaders are also quite happy because
we're seeing, trending down in call handle times,
which is,
you know, as as leaders are are well
aware, that's a pretty important metric for,
success in call centers. So average handle time
has been,
trending down as more and more adoption of
the the tool is used.
And in addition,
something else that's pretty neat is we have,
the c the TSSs themselves, which are the
technical support specialists that help CSRs when when
calls get, tricky.
Usually, they're inundated with questions.
But now that the CSRs
are able to go to Alice for some
first level triaging of of challenging questions, the
TSSs now have a bit more time to
go actually improve the documentation
that Alice is using to give the CSRs
the answers, which has been great because, I
mean, there's thousands of documents and they're they've
been maintained by humans. So as you know,
there's there's, of course, gonna be a few
errors and mistakes in them. So Alice has
been surfacing those where maybe Alice gives a
quote, unquote wrong answer. And they go, well,
Alice made a mistake, and they dig in.
And they're like, no. Actually, our our policies
or things have have evolved or changed over
time, so now we can update this. And
it's just making everything better where the documentation
is more aligned more quickly, CSRs get better
answers more quickly, and the leaders have been
pretty pleased with it too. So, it's been
a a pretty rewarding, turnaround to see how
people have been adopting it.
Absolutely. So, really, this sounds like a win
win dynamic both from the member and the
internal
perspective, especially with that average handle time trending
down, the technical support burden, being lifted so
they can focus on other things. For the
other health plan leaders that are listening in,
Nathan,
from that operational and that ROI perspective, I
know you touched on a few metrics here.
Are are there any lessons you would share
in terms of scaling an in house AI
solution, that could that they could apply to,
their companies all over the country?
Yeah. That's a that's a really important question,
and I think we're at a very critical
point where these decisions need to be made
now and they need to meet be made
correctly. Right?
So
everybody's got this build versus buy idea, and
it seems attractive. Right? Well, why would I
try and get an in house AI engineering
team to build these things out when
pretty much every company out there now is,
you know, advertising
AI AI everything. You can buy AI powered
pizza on the way home. Right? Well, of
course, you know, everybody's gonna advertise that because
that's the new attractive hot topic that people,
you know, wanna wanna market. But what I
would encourage leaders to acknowledge is the following.
Most of businesses
is institutional knowledge. Right? Your your real intellectual
property and value is in
oftentimes, your data, of course, but all the
knowledge and expertise
that your staff and your employees bring, all
the processes and and procedures that you've distilled
over decades
of of honing
all this this knowledge and expertise. So
if you can build
AI agents internally that crystallize
all of that expertise that you've owned over
time,
you're basically
locking down that intellectual property.
And that's now an asset for your your
enterprise,
And you can evolve it, grow it over
time, allow those agents to cooperate with other
agents to solve more complex problems as we
start emerging,
evolving towards these emerging, like, multi agent systems.
So
my advice is early on, if possible, if
it makes sense for you, really do try
to to to think through if you can
try to build your own custom,
agents or at a minimum ensure that you
have full IP over the agents that you
are trying to build internally. Yeah. That that's
my advice. Sure. It makes a lot of
sense, and I I think that's great advice
for our audience as well. The the last
thing I wanted to ask you, Nathan, is
how you see AI enabled customer service tools
like the one you've built. How is that
gonna continue to shape Premera if you look
forward the next three to five years? I
know this is very, quickly evolving technology, and
that is a long time frame. But,
what's your predictions here for for both your
company but also for the wider industry?
Yeah. I mean, I can I can tell
you? Right? So, I mean, we
have across all of our organizations, right, cybersecurity,
IT, corporate data and analytics,
you know, we've been coming together, and we've
got a core group of people
that have been making, you know, our enterprise
AI architecture and trying to ensure that our
we're future proofing our adoption
of AI and where we're scaling towards. So
the main thing to acknowledge is that, you
know, health health insurance
I mean, it's it's a math problem at
at the end of the day. Right? So
first and foremost is you gotta solve the
math problem. Computers are better at solving that
math problem, and we wanna acknowledge that that
is the core capability that we want, you
know, technology to support in. But then also
there's the human element. I mean, health is
really important, and we need to be able
to reach out to people, be sensitive,
and be able to connect with people
as well. So what we're thinking is that
this insurance is basically a process driven industry.
So the vast majority
of the the math problems that are being
solved and the input and processing of documents
and output of
documents. This these are things that AI agents
can help automate. And what we would then
be able to do is to leverage all
of the human expertise to to really touch
on the human side of things and make
sure that all the members,
are really feeling cared for or that they
have the support that they really need, so
we can automate the laborious slow sort of
procedural factory line type work, and allow humans
to connect more on the health element, which
is, you know, I think at the end
of the day what what we really want.
That that that's one of the things that
is gonna separate the health industry from others
in AI adoption is that it's very personal
and it's very, very important. And so how
AI is integrated there is gonna be, a
pretty important
thing that these companies need to think through
carefully.
Wonderful. Well, Nathan, I wanna thank you so
much for taking the time to chat with
us and with our audience and for sharing
our insights with all the leaders listening in.
We we really appreciate it. It's my pleasure.
Thank you for having me. Yeah. And and
to our listeners, if you'd like to listen
to more podcasts from Becker's Health Care, you
can visit beckershospitalreview.com.