Welcome to episode 336 of the Microsoft Cloud IT Pro podcast
recorded live on May 31st, 2023.
This is a show about Microsoft 365 and Azure from the perspective of it pros and
end users where we discuss the topic or recent news and how it relates to you.
Microsoft build has come and gone and we have some updates to discuss this week
regarding announcements made at the conference. As can be expected,
there was a bunch of AI related news around co-pilot chat G P T and OpenAI
and then finally a new SaaS-based data product, Microsoft Fabric.
Are you gonna catch me up because I was on vacation last week and you would be
proud of me Scott.
I turned on my computer I think once the entire week cuz
I actually had to do some podcast stuff.
I have tried to catch up on blog posts and news and all of that,
but I have not had the time I wanted to to dive in to build stuff and I saw
you put stuff and I've flagged some articles I saw but again,
I have not gone into depth on anything in particular.
So what shiny build stuff should we talk about? So let.
Me catch you up on some Microsoft build stuff.
So Microsoft Build 2023 happened last week, two days.
You can absolutely go out and watch the on-demand sessions at this point.
It's all out there ready to go. All the blog posts and things,
we'll have links in the show notes and all that good stuff.
So I highly recommend everyone go watch the keynotes
even though keynotes are largely useless.
I think it's very interesting to just see the kind of tone and approach
around things like ai. I was,
I was kind of joking around as the keynote was going on.
Like my big takeaway from the keynote was that you're going to be able to build
co-pilots for your co-pilots to pilot your co-pilots,
to get you another co-pilot to do something for you. And,
and I was very confused by this because I had been introduced and
I think most of us has been introduced to co-pilots as marketing terms
and not as a,
a feature like it's a product versus a feature like GitHub co-pilot
is, you know, GitHub's ability to do AI generated,
ML generated code for you M 365 co-pilot like copilot for
Word is the same thing,
it's a large language model that can respond to prompts in Microsoft Word
and Microsoft kind of came out and they said sure we have copilot
and it's a marketing term so it's,
it's a product and a feature and now we're gonna let you go build your own
co-pilots as well. So not only are there the existing co-pilots,
but theoretically you could build a co-pilot for another co-pilot.
Like say you wanted to, I don't know,
you wanted to build a co-pilot for Word that interacted with Microsoft's
co-pilot. Like that would theoretically be possible.
Like you could do some own pro,
some of your own prompt engineering and prompt hacking and inject on
top of their L L M with your L L M and and potential
responses or context of your business and and things like that on top of it.
So that was kind of interesting to go through and see.
I also recommend that everybody watch the keynote that happened right after
that, which I believe was with Kevin Scott,
he's the recently crowned VP of AI or something
like that at Microsoft. I forget his ex exact title,
but he kind of goes into more of that ecosystem and everything that's going on.
So a couple things, uh, co-pilots for everybody.
Like absolutely you can go out and build your own co-pilots.
There's a service inside of Azure called Azure AI
Studio where you can go and train these models and design your
co-pilots and release them out there.
Some of the demoware I saw was kind of,
it was a kin to the old like chatbot in five minute kind of demos that you
used to be able to do.
I think it's a little richer than that just based on kind of the language models
and and what they have access to today and just kind of how far prompt
hacking has has come along or I guess prompt engineering to get things to where
they need to be.
The other interesting thing that was announced was
that plugins are coming to chat G P T integration in
Bing. So today bing kind of runs on the,
I think the G P T three five turbo model and it is coming over
to G P T four and when it does that,
it will have access to the same exact plugins that you have access to if
you are a premium subscriber open AI today.
So if you're already paying open AI,
the monthly fee for access to the paid version of that product,
you can already flip over to the G P T four model and with
G P T four you get plugins.
So plugins will let you do things like have more context around
kind of these outside data ecosystems.
So some of the examples that they brought up cuz it's, it's bing,
it's kind of consumer grade and and consumer wear. Yep.
Where things like open table Zillow,
you know like hey I want you to go out and build me a table of
houses that are within five mile radius of this address that meet these
criteria and put them in a table and rank them by price and number of
bedrooms, number of bathrooms, things like that.
So plugins are coming and and rapidly coming and I'm kind of looking forward
to those. I think they'll be nifty especially once you know,
some more stuff comes out there like I would love to see a plugin for something
like Reddit like that's built around the Reddit API and not just bing
scraping Reddit and it has context of things like upvotes and kind
of hot articles and and things like that.
I think it's gonna make data mining with some of the LLMs just a
lot smoother at the end of the day.
How do these plug-ins work then? Where do they live? Is this again,
I haven't read up on this at all.
Is this plug-ins to connect to Bing two services or plug-ins that you can put
like in Bing for instance you said build me a table of houses
on Zillow. Does that return,
is that a plugin for Bing to go out to Zillow and then it just returns on like
the Bing chat UI or how,
I guess how do those plugins work and where do they sit? Does that make sense?
Yes, so the plugins, my understanding of them,
and I haven't written one of these so don't like hold my feet to the fire.
My understanding is plugins are executed within
the context of chat G P T but then they basically have
connectors to go out and be able to interact with those other
services.
So you can almost think about it as like a plugin is exposing another API
to chat G P T so that it can go out and interact with that data and then that
data's returned in a format and in a manner that is all up friendly
to chat G P T and and being able to have everything just kind of come back
and be able to be massaged into the right format. So you're likely,
it's not like,
I don't think it's like Bing has gone out and indexed all of Zillow,
it's more like it's calling Zillow with a pre-engineered prompt and then
Zillow has an a p I endpoint that's able to respond to that prompt based on
either a data set that they've built out or maybe they even call into another L
L M on their side to get some data back from it and then they return all that to
the client munt together and it's all kind of ready to go.
Okay. I think that's making sense and I'm still trying, it's.
Basically like APIs, like it is just API interaction.
I think the interesting thing is it's almost like APIs talking to APIs just
because of the human language nature of L L M prompts and kind of how,
how they're constructed, right? It's it's,
it's not like you are writing a database statement and you're saying you know,
set options strict on,
you're going in just human language and you're saying hey large language model,
I need you to imagine that you are a CEO
at blah blah blah blah company and you always act in this manner and you're
gonna be kind and you're gonna respond in this way and you're gonna be held to
this set of standards in your response. You know,
where I think we're used to more like just arbitrary like binary on off kinds of
things as we're, you know,
maybe scripting or or interacting with programs today or or writing our own.
So it's a little more free flowing than that. So I, I don't know,
we'll see what comes out of it. Like.
Developers are writing these that and essentially submitting these two yes G P
T or Bing and saying hey here's my plugin.
I want to bring in this additional data or add this additional functionality
to G P T. So is this adding it to chat G P T?
Is this adding it to open ai? Is it adding it like this is also,
I was starting to look through this,
this is getting all kind of confusing because everybody kind of uses chat G P T
as a default but it looks like open AI is now using Bing as the
search experience for chat G P T but I don't think Bing Chat is
necessarily chat G P T and trying to keep all the terminology straight in my
head too. So.
Open AI is the company, they're the big broad company. They build a bunch of.
Models and the learning models and all of that. Yep, yep.
They build a bunch of products.
One of those products is called Chat G P T and then within
chat G P T you have various flavors of language
models that are available to you basically like what's the switch?
I wanna flip and chat G P T for the types of responses that I wanna give back.
Those are all the G P T models you hear about.
So you hear about like G P T three five, G P T three five Turbo,
G P T four, those are kind of the features.
Right? Built by open ai right?
And then chat G P T is sitting on top of those different flavors of G P T
3 3 54 chat.
G P T is just branding like chat G P T is a website.
It's the thing you go to to interact with a set of features that you turn on or
off and the models are the features that you go back and forth and you turn on
and off that way.
So you could go in like if you're a paid subscriber to chat G P T today,
you can go into chat G P T and you can say okay I wanna work with the
G P T three five Turbo model and the G P T three five turbo model has
a certain set of constructs.
It was built on top of one Lang large language model.
Or you can say I wanna do G P T four And probably the biggest difference between
like some of the older models and G P T four is that G P
T four is the one that has the plugins and it can do like real live internet
searches.
So remember like when you chat G P T initially came out and they said okay we
have this model right that you can interact with and don't worry about it.
One of the big constraints of it was that model was only trained on a data set
that was current up to late 2021. Yeah.
I remember there was a date there, I couldn't remember if it was 2020 and 2021,
but I do remember that where you'd go ask it stuff and it'd be like my data's
limited to data up to this point in time because it,
it couldn't go grab recent news.
Recent articles didn't really touch the internet. Correct.
So G P T four opens that up,
lets it go and do more realtime contextual searches and
the Bing part of it is Bing is using ,
the G P T four model or transitioning to it cuz they will be as they bring like
plugin integrations and things like that. Yep.
So Bing is using a large language model in the background and then the
background of GP chat G P T, they're also using Bing as their search engine.
So what they've said is like G P T four when you're using that model then it
needs to reach out to the internet and it needs to do a search in real time.
It's gonna use Bing as the search engine of choice to do that.
Got it.
So that's how they're kind of connecting G P T four and to be able to search
some of the internet as leveraging Bing.
But then Bing is using G P T four in the background as well.
It's a.
Little weird. So Bing would be the backend search engine for chat G P T. Yep.
And chat G P T would be the backend large language model for Bing. Okay.
You could almost like draw like a weird circle and put some double arrows on it
and it, it would make for you know, a pretty funky physio diagram. That's.
What I was trying to kind of wrap my head around as we were talking through this
is how all of those were tied together but what you just explained makes sense
and then they're using all these same standards then for plug-ins that if you
write a plug-in for open AI because of the weird circle model,
all those plug-ins will work in chat G P T and they'll work in Bing and
they'll work in co-pilot and all of that because it's all just one big happy
circular family. Correct.
Like theoretically that's, yeah that's, that's the way it all goes.
So we'll see if it manifests that way but .
And then you can go build your own co-pilot on top of all that using those
add-ons and yeah.
You can.
So you can go out into Azure AI studio today and you can do that another place
that co-pilot is coming in the Microsoft ecosystem that I
thought was kind of interesting as I spend more and more time there is Power bi.
So there's gonna be a Power BI desktop developer mode
that's going to enable a workflow for interacting with,
well I guess a couple things.
So you're gonna get co-pilot integration on Power BI and then you're also going
to get integration and some other things coming to Power BI that I thought were
kind of exciting. Like I, I would totally use that all the time.
I'm tired of sending the same P B I X file to 30 people and then wondering where
they've uploaded it to.
Do you feel overwhelmed by trying to manage your Office 365 environment?
Are you facing unexpected issues that disrupt your company's productivity?
Intelligent is here to help much like you take your car to the mechanic that has
specialized knowledge on how to best keep your car running Intelligent helps you
with your Microsoft Cloud environment because that's their expertise.
Intelligent keeps up with the latest updates on the Microsoft Cloud to help keep
your business running smoothly and ahead of the curve.
Whether you are a small organization with just just a few users up to an
organization of several thousand employees they want to partner with you to
implement and administer your Microsoft Cloud technology,
visit them at intelligent.com/podcast.
That's I NT E L L I G I N
k.com/podcast For more information or to schedule a 30
minute call to get started with them today.
Remember Intelligent focuses on the Microsoft cloud so you can focus on your
business. Oh co-pilot everywhere,
you know where else co-pilot's coming everywhere to Edge .
Did.
You see co-pilot as coming to Edge too? Yes. So this one could be interesting.
This one of some of them intrigues me a little bit where it essentially uses the
website as context. So some of these news articles or if you pull up a website,
this one would be interesting to play with co-pilot to get like a summary of a
webpage or to find out more about the webpage.
This one is one of the co-pilots that intrigues me more.
I haven't played with like Power Apps or Power Automate or some of those
co-pilots.
I'm really looking forward to getting my hands on all the co-pilots and seeing
which ones I actually like and I find useful and which ones are just kind of eh.
So for Power Automate I haven't used it much but I'm not
a large power automate person.
Like I'm not using it in my daily life but I do use Power
BI almost daily and okay, I've got Power BI desktop open all the time.
So there's a quick little video preview.
It's actually on the news site from like Microsoft Global Comms.
I'll put a link in the show notes to that which is copilot in Power BI
and you can kind of see how it works.
So one of the interesting things to me about copilot and Power BI is you know
how Power BI has kind of the q and a widgets and things available
today? Yeah. Where you can go in and it understands your data set and so,
you know, show me sales by quarter for the past two years ranked by
industry of company, blah blah blah or industry of customer. Uh,
and and it can do all that.
So that already exists today but all that's doing is building out like singular
visualizations for you.
The co-pilot in Power BI lets you go in and it says hey here's a
data set, like here's a database and here's some tables in it.
Like maybe go figure that out for me and then tell me what I can do with it.
Oh and by the way go ahead and actually build out the P B I.
For me it's kind of like I,
I get it's all demoware and when it works it works but it's super
cool just to kind of like see it just, you know,
get released and go against things like that. Cause if you think about it,
like lots of companies are the same at the end of the day.
Like if you're a publicly traded company,
like the way you're keeping the books is gonna be very similar to all other
publicly traded companies usually in your country cuz that's some type of
regulated industry.
So there's gonna be certain requirements for you for reporting your data,
for storing your data.
Like it's always gonna have to come out in a common format anyway.
So it's one of those things that you can absolutely train a model on and then
have that model come back and kind of immediately have context to your
data but it already knows things like common schema, common terminology, all,
all that kind of stuff.
Interesting. This is, I'm Scott,
I'm committing to not adding everything to my list.
This one I don't use Power bi enough.
This is not gonna get added to my list but it would be interesting and maybe if
I started pulling more data into it, yeah, I'll have to play with it.
Maybe this is a maybe how about it? Maybe on my list, maybe not.
Uh, maybe on your list. Okay.
So maybe one last one for today before we run out of time,
cuz I think we both have meetings coming up.
And this was in the same article with introducing co-pilot and Power BI
was introducing Microsoft Fabric.
I have seen a boatload of headlines about Microsoft Fabric.
I am not a data guy and this looks
like it's tied in a lot more to Synapse data,
factory data warehousing data science, that type of stuff from the headlines.
And what I've gathered is that it's like this cross
cloud data ingestion thing where I can like go grab all my
data from all the things everywhere and pull it into Microsoft Fabric and
integrate it into some of these data analytics tools into Synapse Data
Factory. Is that right? Or like what,
what is fabric at a high level?
Because I also don't know that we have time to dive into it in detail.
I don't know , I don't know enough about Fabric. All.
Right. Fabric. Well first of all it's a horribly named product.
I think some marketers fell asleep at the Wheel.
again.
When they came up with the name for it. Y yeah.
Cause there there are already things called fabric in the Azure ecosystem
and and in the broader Microsoft ecosystem as well, there's.
Lots of fabric. Like my first thought was App Fabric.
I can't remember what App Fabric was from. Was that a SharePoint thing?
Well it was Azure. There's App fabrics. Service fabrics, yeah, yeah,
there there's fabrics for service fabrics.
All the things.
So Microsoft Fabric is a
SAS service.
So it's kind of interesting in that it is something
that wraps a bunch of Azure services while not
necessarily being Azure itself.
What I mean by that is if you think about some of these things, like we were,
we were talking about Power BI like a couple minutes ago.
Power BI is a SaaS service. It's licensed by the user.
Like you don't go out and buy a Power BI tenant and then
buy a Power BI database and then kind of put all these pieces together yourself.
You say like, oh no,
I'm just gonna buy a per user license to this thing called Power bi and it's
gonna let me do visualizations based on a bunch of data sources that exist out
there. So it's,
it's a little bit of a weird model for me to wrap my head around,
I guess is just like data as a service that's not
necessarily Azure focused.
It's like it's Microsoft focused. So it's not even M 365,
like it's a whole new weird thing. But at a high level it's,
it's a SaaS offering that brings together a bunch of building blocks for
you inside of a single service offering.
So while you have all these component pieces and parts like Power BI and
Synapse and Data Factory and Azure Data Explorer and all these things
that sit out there, you know, they are disjointed.
So you could go down the path of picking up a Power BI license for your users
and then saying, okay,
I'm gonna use my existing Azure data explorer thing over here.
But then what do you do when you want to go grab data from like a raw data set
from a data lake and transform it with Spark? Okay,
well now I gotta go create a,
a storage account and activate hierarchical namespace on it and then,
oh by the way, I might need like a, a synapse spark pool to spin that up and,
and you just end up with all these pieces that you need to manage and it might
be a little bit easier to have it in a managed offering,
which is kind of what Fabric does.
So it brings together Data factory synapse, data engineering,
synapse data warehouse, synapse data science, synapse real-time analytics,
Azure Data Explorer, power bi,
and it brings together cloud storage as well
and even multi-cloud storage into a single
service offering. So now rather than saying, Hey user,
go to Power BI and then go talk to the developer over here to get a connection
to that data lake and then go talk to the developer over there to get a
connection to their, their data set. And then, oh by the way,
go talk to this other developer in this other team who's been doing this Delta
Lake project and blah blah blah. You can just have it all sit in one place,
which is your one lake, which is kind of like your foundational data set.
And then all these services automatically have connectivity and they know how
to talk to that one lake and consume data out of it.
So it's a little weird. Like you get all these Azure services,
you get some M 365 or like data SaaS services like
Power bi, then you can also pull in data from outside sources.
Like we'll see how it goes. I, I would recommend there's a,
in the getting started docs,
there's a fabric terminology bit of documentation and it
takes you through like general terms of the service. Like hey,
what's a workspace? You know, how do they measure capacity? Things like that.
But then it breaks down into the individual pieces and components like inside
Data Factory inside of Microsoft Fabric.
What do you get inside of uh,
synapse data science inside of when it's wrapped in Microsoft Fabric,
what do you get?
So I can see it like making things a lot easier for people. You know,
if you think about like, you know,
just that headache of like getting access to a data set and getting like my VM
and my managed identity to be able to talk to this storage account or this thing
over here, this database over here. Like that's as painful, right?
Like why not just have it all managed by the service and wrapped around a common
data layer? Like that makes sense.
So I think that's kind of like how the service manifests and comes out.
I haven't seen,
and I I didn't do too deep a dive on it about things like pricing
and what that's gonna look like, like impacts to existing users.
It's kind of like if you're a Power BI , uh, the,
the way it is in the docs, if you go to the docs today and you say, Hey,
I wanna start a fabric preview trial,
it immediately opens up with what if you're an existing Power BI user.
And I think that kind of gives you a big hint as the target audience and,
and where they're going with it. Yeah.
They do have,
like I was looking through some of the documentation where they have by fabric
now or by Pro and I don't see though, to your point,
I don't see purchase fabric capacity. There's some pricing but man,
some of the naming in here too, Scott,
like I'm gonna go back to you with marketing.
I was flipping through this article and now you have
Data Lake but you also have one Lake data and then
you have uh,
there was something else in there that I saw that was named really similar to
that.
I think that fabric terminology site is gonna be a good site to keep handy as
you go sort through what all of this stuff is.
I think the big thing to keep in mind today is, like I said,
like it is a wrapped SaaS offering.
So you don't go and buy Data Factory by itself anymore.
Like you or you still can like I'm, I'm not saying you can't do that,
like absolutely like the component pieces and parts are there,
but if you walk into it from a service offering side of things, it's just,
I'm gonna go buy Fabric And Fabric has a data factory component within it. Yeah.
Well and it looks like because of that, as you need,
depending on the size of your data,
it looks like there is different capacities and different levels and there's,
this isn't just gonna be go buy fabric,
but you're gonna have to buy different levels or different capacity
of fabric based on the amount of data, how much is gonna be in there,
et cetera, et cetera.
Capacity is an interesting one. So it's a SaaS product,
so it it's a multi-tenant kind of product. So you,
you're basically buying a fabric tenant is is really what you're coming into
when you go ahead and purchase that.
So each fabric tenant has
capacity that's measured in things like compute, like hey,
I'm spinning up synapse in the background.
What are the different SKUs that are available to me?
How many cores is that gonna come with?
So immediately like if you start going down the fabric licensing hole and you
get into like capacity and SKUs,
it opens up with compute where you've gotta figure out like what's the skew and
fabric to the equivalent skew and Power BI and what does that translate
into V cores and how much you use
along the way Gets a little bit weird on that side.
The whole OneLink concept is interesting as well because it's
not a data lake that you manage. So I guess coming from Azure storage,
you know my take on, it's like just bring your data lake, right?
Like you've already got it out there, it's just a, a gen two storage account,
GV two with hierarchical namespace enabled, blah blah blah. Yep.
But now with one lake, because it's a multi-tenant model,
they've effectively like provisioned some storage on the background for you.
So I think that's gonna be interesting to see like how they just price capacity
and and kind of figure all that out.
And then they built connectors from things like one lake over to all
the other clouds as well to to Microsoft's cloud and then over to S3 and
GCP as well. So if you have a one lake,
there's a connector in there where you can go and say like,
Hey I have an S3 bucket that sits over here that maybe has a data set that you
want in it and you can actually connect your one lake over to that S3 dataset
and then the proxy layer of one lake just makes it super easy for Power BI to
talk to it. Like Power BI doesn't need to know that it's talking to s3,
it just needs to know that it's talking to one lake and then you're done.
Interesting. Lots of stuff to go figure out.
This is not what on my list either.
This is not a high priority for me given what I do,
but with all the news I've seen around it about it,
I figured I should go dig into it a little bit.
I think you're gonna see it more than you think in your world
because your customers are Microsoft 365 SaaS customers who
use things like Power bi. So if Power BI is an entrance point today,
like looking at the way that it's positioned,
it was announced and the way the documentation's oriented and things like that,
it's kind of like Microsoft is positioning it as every Power BI customer
should just become a fabric customer and then your life will be a lot easier,
right?
Because you get all the compute and it's managed and identities are managed and
you don't need to worry about security and things like that.
So if you in your ecosystem have customers who are doing Power bi,
it might be something you wanna spend some more time with cuz I bet that's where
like a bunch of the bleed outcomes from is existing Power BI customers going
like, huh, should I just be doing Power BI anymore or should I be doing uh,
fabric?
I feel like it'll depend a lot on the pricing just looking through,
I posted a link to it in Discord too,
looking through the whole Microsoft Fabric licensing.
It feels way more confusing than Power BI . But I guess to your point,
if you're starting to bundle in some of those locations where your
data might be housed,
maybe it's that the bundle does end up being cheaper,
the licensing does not look like it's gonna be straightforward based on that
fabric licenses. So I'll be curious and I'm gonna keep an eye on it.
It's gonna take time to get there because Fabric is kind of,
cuz you're buying Rebundled capacity.
So now you're back to like this weird model of saying like, oh, okay,
like what's the skew I get and what's the capacity that comes with it
along the way,
which can get confusing pretty quickly.
Well, with that, I should probably go jump to my next meeting. I can let you go.
We've been chatting for a while now. All right,
we can continue on Friday cuz our recording schedule is a bit
on with some vacations coming up,
but we'll talk to you again in like two days for more topics. All.
Right, sounds good. Thanks Ben. Thanks.
A lot Scott. We'll talk to you later. If you enjoyed the podcast,
go leave us a five star rating in iTunes.
It helps to get the word out so more IT pros can learn about Office 365 and
Azure.
If you have any questions you want us to address on the show or feedback about
the show, feel free to reach out via our website, Twitter, or Facebook.
Thanks again for listening and have a great day.