Join us as we gather around the hedge,
where we dig into technology,
business, and culture with the finest minds in
computer networking.
Well, hello, Tom.
See, I said Yvonne I I started with
Yvonne last time, so I'll start with you
this time. Yeah. I switched it up. Hello,
Russ.
And now we cannot say happy New Year.
Why is that? Because we talked earlier.
Oh, is it still is it not still
happy? I'm still happy.
It's really not Happy how about happy Friday?
Okay. Happy Friday. For a long weekend. Yeah.
We we already do that. That'll work.
And hello to Tom's plant,
who is I I it's still back there
typing.
And
Little Shop of Horrors.
It's not the Little Shop of Horrors.
It's a nice plant.
What's the name of the plant in that
movie?
Oh, I can't remember. We're gonna have to
go find that now.
So hello, Yvonne. I hear you back there
in the frog lamp.
I am here.
Happy Friday. And then we were joined by
Mike Bushong,
who I mean, Thanksgiving.
I thought if we're just allowed to say
picking a random
a random,
day or holiday that's, like, way in the
past. If we're gonna do happy new year
this late new year, happy thanksgiving to you.
See, on our last recording, Yvonne dared to
challenge Russ on this very point.
Oh my goodness.
So let's talk about constrained
supply or constrained
source environments
because that is where we live today.
The price of memory is going up. Crucial,
I think it was, said
no more PC market for us. No more
no more,
oh, there it is. Anyway,
so, Audrey, that's now the name of your
plant.
Audrey.
Okay. Good to know. So Perfect.
Anyway It's settled. It's settled.
So crucial, I think it is said that
they're no longer gonna do PCs. They're not
doing consumer market any longer.
It's all gonna be server side AI side
because the demand is too high,
which is driving up the price of phones.
It's driving up the price of PCs.
It's actually getting hard to find things. In
fact, there was a really kind of
a funny thing I saw out on x
where people were saying, are game developers now
suddenly gonna decide to optimize their games
for better memory utilization since Crucial is no
longer in the market?
Like, you can't buy machines with massive amounts
of memory.
So we live in this world in processors,
and, Mike, you'll tell us that this is
true of GPUs.
This is true of network processing units, forwarding
engines, right, FEs.
Of all the sidecars
and everything that go with FEs,
including the memory,
the trace motherboards, everything, it's all resource constrained
right now.
And so
there are a lot of side effects of
that,
and they make some differences in the way
you architect things, the way you design things,
the way you even do your career
to some degree or another.
So let's talk through some of those things
because they might be really
very important in the way people think about
what they're working on.
So I don't know. Mike, you wanna start
with whatever's top of top of mind?
Yeah. You mentioned I mean, you you used
the words right now. I think it's the
the the scary part is not that supply
is constrained.
It's what if it's constrained
forever?
Because so here's the thing. During COVID, we
knew that there was a drop in manufacturing
capacity. Right? Literally, people couldn't work, and so
you saw, like, a physical
reduction in
how many, you know, units of whatever could
be produced, you know, in in all these
different plants.
And then we knew that there were changes
in, like, transportation, and so there was a
lot of discussion about, you know, if there's
fewer ships or people can't offload the cargo
at these ports. If you remember all the
like, looking at Los Angeles, the Port Of
Los Angeles and looking at all the the
the ships that were essentially waiting offshore because
they didn't have enough people to to offload
stuff.
And so we we kinda thought that during
COVID, everything was constrained just because the the
conditions led it to that point. What's happened
since COVID is we've never really returned back
to normal.
Like, what if this is the the new
normal? I mean, like, lead times are measured,
you know, typically in in months and quarters.
We're getting close to the point where it's
measured in years,
which is, like, insane.
And then as a result, all the pricing
changes. And so and there's some actually there's
some I know maybe not that interesting to
most people, but, like, there's implications to that.
If I
if I'm spending
if I'm making a purchase decision today, but
it's being shipped next calendar year,
I'm spending next year's money,
next year's budget.
And then if the budget if the cost
of memory as you mentioned, like, it's, like,
doubling,
if the cost of key components is increasing
in that period because everything is kind of
constrained,
then
I can't even use,
like, master purchase agreements to lock in pricing
on that. And so not only am I
spending next year's budget,
I'm not even positive how much of next
year's budget I'm ultimately gonna have to spend.
Yeah. I'm actually giving control of my internal
budget over to a supplier
in a sense. Effectively. And and then and
then
you you can kinda move beyond that, and
you say, well, then, you know, that's that's
even in the best of times. And then
you say, like, during these AI boomed times,
we're gonna spike demand.
And so, like, it's gonna it's gonna make
everything, you know, like like, way, way harder.
Like, that's kind of the environment we're in.
And and so the you know, for me,
the like, I'm I live on the vendor
side. Right?
On the vendor side, you know, you you
talk to customers and they're no longer longer
trying to figure out how do I
connect my servers, how do I
connect my sites. They're figuring out, like, how
do I
financially engineer
the
purchase I have to go make. Or
they're no longer worried about which protocol do
I use. They're like, well,
what does our supply chain resiliency look like?
They're making
architectural decisions based on procurement inputs, which is
something I don't think most people have even
thought about.
Yeah. And so, if you want it like,
as an individual then, if you wanna be
relevant in your company, you gotta realize that
it's more than connecting devices. Now it's like,
how do I navigate all of these different
variables, all these things that are important to
my company? Like, I just think that that
that impact is huge.
And then now when you think about, like,
the architectural
implications
of supply chain based decisions,
like, okay. So now I'm gonna go multi
vendor. Maybe I didn't wanna go multi vendor
before. Now I'm gonna go multi vendor, or
maybe I can't get access to supply, so
I can't go with vendor vendor of choice.
So I'm gonna go, you know, with with
a second vendor, which maybe has like, what
if it's a different operating system?
What if it's like a you know, I
gotta do the there's integration testing, and there's
qualification testing. What if you've got orchestration?
Like, I just the topic is is so
big and hairy.
Like, even just making sense of all of
that. I just it I I don't know
how we we collectively as an industry Yeah.
Make progress. I mean, it it there are
many things you could say come see come
out of this. Like, for instance,
okay, we talked about white box and des
disaggregation
separating our hardware of going to cattle, not
pets.
Oh my goodness.
Cattle not pets.
I mean, making your hardware and your your
your stuff interchangeable
because you never know where you're gonna be
able to get your next
set of 500 routers you might need for
your data center fabric or whatever it is,
or even to do your edge connectivity.
Telemetry
so that that's even right there a starting
point to me
is that
you need to stop thinking as an individual
about,
you know, these are individual devices and I
can do counts or no. It's the hardware
you can get
to get the job done
tomorrow
that you need to do.
Another interesting problem to me there is, what
do I do about financial planning? I've gotta
be able to justify
the cost of buying something two years from
now or three years from now as lead
times might happen to be.
And I don't know
if I'm gonna have the business to do
it.
Well, and it's not it's not and in
in in this constrained environment,
it's not just do I have the money
and the business to do it? Do I
have the scale?
Because in a source constrained environment,
the sellers are going, the people who actually
have the stuff
are going to sell to the people with
the biggest pockets,
deepest pockets who who can who can and
and this is like the way the world
works. I'm not even making a moral judgment
of this. This is just it's what you
would do. Right? If you can only sell
a 100 donuts
and somebody comes along who can buy 99
of them and is willing to pay your
top price, you're going to go ahead and
sell the 99.
And if somebody else comes along and wants
to buy two donuts for them and their
spouse, partner, whatever,
like, you're gonna be like, sorry.
Like, I've promised 99 to this guy. And,
oh, by the way, he can pay 1.52
of what you can pay for 99
even if you're willing to pay five x
for two. It it just it You see
it in a house, right? Math. Like, in
in housing, like, people come in with an
all cash offer. I mean people, you know,
you're you get outbid.
Yeah. It's and it's Right. It's and it's
Right. How frustrating that is, and like and
and you can always go look for another
house. I think when you get into like
IT, it's like it's tougher. Like there's there's
not always another house. You have to, like,
hire a whole different set of people and
figure out how to Something
else I was thinking is you and, as
Mike and Russ were speaking is, you know,
many of us learned to design and architect
and think about infrastructure
inside of a certain set of constraints.
And your mental models may be tuned to
a certain
set of constraints
that you that are unconscious to you. Right?
And so you kind of have to also
peel that back,
reevaluate
what assumptions you may have been making that,
oh,
consistency across the entire environment is the most
important thing. And oh, saving engineer time
is, you know, these things save engineers time
and and you've got a set of architectures
built around that. That may no longer matter
anymore.
And so it's it's important to also acknowledge
like, hey, there's a whole set of assumptions
that may not apply anymore.
Yeah. One of the I feel like one
of the lessons I learned,
from a technical,
subject matter but applies here in other ways.
I remember doing it first learning about vendor
interop like, early in my career, I got
another vendor besides the one I first cut
my teeth on. And, like, not only was
the syntax different, the way that that operating
system thought about hardware was different. And I
remember being just very, like, woah.
Okay. And and maybe do it that way?
Yeah. Yeah. Yeah. Exactly. And it what it
forced me to do is look at the
primitives. Like, look at what's the what the
protocol doing or the more important than the
protocol, like, what's happening on the wire. And
then once I understood that, then adding the
third vendor was really easy, and I felt
really empowered. And I felt a lot more
confident to deal with uncertainty in the future.
And I feel like the what you're talking
about with mental models, Yvonne, that's is is
making me think about that that the you
know, we we have to be able to,
uncertainty cannot can no longer be sort of
this nightmare scenario. We have to it has
a cost attached to it, and it has
constraints attached to it, uncertainty itself. And if
you can face it, I think that that's
a I think it's a really good thing
for individuals and probably for companies as well.
What you just said there is, like, huge
though. So, like,
for you,
you took a thing, and then it you
kinda went deeper and deeper and deeper into
it, which then gave you kinda control. And
so what you ended up with was great.
If you look at that same dynamic happening
now, it's happening at a time where we're
abstracting more and more of the of the
networking detail, the implementation detail away.
We're putting in a fancy GUI, and when,
like, the AI ops sort of rolls through
and everything is a natural language front end,
like, you don't even have the ability to
peel it back anymore.
Like, I that's actually I mean, I didn't
when you said that, I'd I'm it's mildly
I mean, I'm glad I was close to
I'm glad I'm old because then I don't
have to do it.
It is mildly terrifying. But, you know, it's
even more terrifying if you think about it
in terms of what you were saying before
that or what Yvonne was saying, like, okay.
Now the hardware
is not
is is used to be whenever I teach
networking classes. I'm teaching for UC right now.
And the first thing I tell people is,
what is the most expensive thing in the
network? It's the people.
It's the people who sit there and build
the network, manage it, control it, troubleshoot it.
What if that's no longer true?
What if it's the hardware
that becomes the most expensive thing,
the CapEx? What if you simply cannot
replace it? Plant. Right? Like, not even the
hardware, but, like The power. The power and
water.
Yeah. Right.
So what does that do to your entire
concept of life cycle management?
What does that do to your entire concept
of cattle versus pets and your entire concept
of automation
and all of that stuff that goes into
it.
What does that do to vendor differentiation?
Does a vendor now just differentiate themselves because
they have the gear to sell?
I will tell you. Like, that is so,
there there to me, there's there's a couple
of requirements in in networking today. Right? It
just in data center, it just works as,
like, the number one requirement. People just want
stuff that works, and and and we've had
private discussions. Like, that's not always the state
of play today, and so I think that
becomes the thing people care about.
It's easy is is kind of the number
two. Like, I gotta be able to to
kind of work with it.
And so you would think that all the
differentiation would be really around, like, let's say,
hardware and software quality and some of the
operations bits.
The number of deals these days that transact
primarily on, I can ship it in twelve
weeks or in, you know, ten weeks,
like, it is nuts. And so people are
and, like, think about the implication of that.
Imagine you were gonna buy, you know, name
any, you know, vendor you love, and imagine
now that the vendor you don't love has
equipment. And imagine that procurement team then goes
and says they're gonna buy that, but then
they hand that to the team. I mean,
that's, like, that's nuts.
And and and if you push back on
that and be like, well, I don't work
on that. I don't know how to do
that. The answer is,
okay. We're gonna go get somebody who does.
Like, you don't have
leverage there
And you need to, like, understand the world
that you're in
and be, you know, before you
you
you have an overinflated view of your contribution
to the process. We all contribute. It's important.
But you also kind of have to know
what's going on in the world around you.
And and and that's why these kind of
conversations are so important.
Well, that that would I mean, if I
were mentoring somebody and you're in an environment
that's purchasing at enough scale that they're likely
to to go down this path either intentionally
or because they're forced to. Like, now my
mentoring
changes. Right? You know, you should be
or maybe chasing the next cert isn't the
big thing to go do. Like, make sure
you're solid on the underlying bits, and then
start getting experience with virtual labs that give
you access to the different CLIs so that
regardless of which vendor comes down the pike
next,
you know, you're you're not you're not frozen,
you know, during the headlights. But, like, I
think people have to become maybe not multidisciplined,
but but, you know, certainly
broader than, you know, I'm a
a vendor bigots with my you know, this
is, like, the CLI that I know and
love, and I can only work in this
environment.
Yeah. Maybe maybe. And, and Tom and Russell
understand that we we, the last podcast we
recorded was on generalization
versus specialization.
And so
in a time like this, generalization
becomes more important,
across different technologies, different capabilities, different now that
you don't need to be specialized, but,
that breadth can be incredibly important.
Yeah. And if you think about this in
terms of automation
and
that first thing you said, Mike, when you
think about okay. So number one problem, it's
gotta be easy to deploy. Easy it's gotta
work. It's gotta work. It's gotta work in
a way that's easy. I don't wanna think
too hard about doing it. I have other
things I need to think about. But then
I'm resource constrained
or I'm source I'm I'm constrained in what
I can buy. So what that does is
it forces
you to actually build
a a layer of abstraction
between what you're trying to get done
and what you have to get well, the
tools you have to get it done
to get that done. So it almost enforces,
much more of a an intent based
or an automation
first view
that I want to stop caring
about
that CLI.
I want to stop because I don't have
the time, because now I've gotta deal with
five of them. I've gotta be able to
have the flexibility of sourcing
from multiple sources
to get the job done I need to
get done.
So for me to do that,
I need to extract myself from those
hardware specific bits
as it were? What do you think about
that? Is that like
an area?
And and how does that play into optics,
by the way, and and other physical stuff?
You basically just said all SDN really needed
all this time was a pandemic and global
conflict.
I
had a leader once who told me,
we were we were talking about some challenges
and constraints in our business, and one of
the things he said
is constraints breed innovation.
And I think ultimately,
like, this
particular time of constraint
will result in innovation that at some point
will likely
move the point of constraint to another place
in the stack.
But I think that there's a long tail
on this one. Right? And we're not getting
out of this one quickly, I don't believe,
unless there's something on the horizon that I
don't yet have visibility into. But I think
it's gonna it's gonna take a bit for
us to figure out this one. I think
there's
No. It's all good. I I I think
there's an an architectural skill set that can
be helpful if we're if we kind of
narrow it down to talking about network infrastructure
specifically.
Because we talk about if we have to
have multiple suppliers and maybe sometimes we choose
them not even for cost but for availability,
and we have to glue these all together
into something that serves the business. So there's
multiple places to place that abstraction.
So you could but and the architectural decision
to be made is, do we do we
that abstraction above the operating system layer? Like,
do we create orchestration to deal with multiple
quote vendors?
Or do we do we move that that
that point of orchestration down? Do we build
the operating system so that it runs on
all available hardware? And there's trade offs for
both approaches, of course. But but I think
the skill to know where to put the
abstraction and to not just assume that it
always has to live up here or that
it always lives down here, but to be
able to push the abstraction closer and closer
to the metal. And and that I think
that's a I think it's an important thing
for engineers to be able to at least
think about.
You got you got to where I was
going in a much
more intelligent way.
The the whole sonic thing, I mean, like,
it's an interesting space. Right? If if sonic
can unlock so if you're constrained resources
getting, you know, a certain number of chips
from a named supplier,
then having multiple,
you know, conduits into that supply is important.
And if you wanted to if the cost
of that is whatever the operational complexity is,
the question is how do you deal with
that? If you could solve the OS side,
you know, and and Sonic is interesting there.
Because what if Sonic meant that you could
dip into, you know, Edgecore or into,
you know, Celestica
or into, you know, whoever?
That gives people control.
You know, what they pick up in return
is that they have to then manage, they
have to understand how to work with a
sonic distribution and figure out the implications of
life cycle management.
But that's
pushing the
I I like the idea of pushing abstraction,
you know, closer to the metal. That's a
a good way of thinking about it.
And if that if that ultimately checked a
bunch of boxes, then the question is, you
know, what are the downstream architectural implications of
doing that? And I think people have to
start thinking about it. Because right now, like,
if you're into Sonic, right, then it's like
Sonic, you know, represents a certain number of
things. If you're not into Sonic,
it's, you know, it's that OS that cloud
companies use.
Right.
Yep.
Yeah. Exactly.
If you're not big enough to do your
own OS like some companies do. I mean,
there are some companies that don't have open
source. They just run their own OS, and
they run basically bare metal, and that's how
they manage their problem. They just buy white
box, so to speak,
and they just run everything on top of
it. Yeah.
So what are some of the innovations that
could cause a disruption here?
There are a few. First, we could say
I mean, you know, something that can't go
on forever will not go on forever.
Right? That's I mean, that sounds really stupid,
but that's the way And very few things
go on forever.
Yes. Even if they could go on forever,
they generally
don't.
So
let's start here. So we're in a re
we're in a source constrained environment,
particularly FEs, memory, GPUs,
traces on boards, all that kind of stuff
is all very constrained.
So
there's coming a point now where it's gonna
become unaffordable for people to buy
cell phones because the price of cell phones
is going through the roof because of memory.
The price of computers are going through the
roof because of memory. Everything is going through
the roof.
And so this can't go on forever. Right?
There's gotta be
some some way
that this gets resolved or something happens to
change.
So what are some of those things? We
could have a software breakthrough.
We could have, oh, all of the stuff
we need for AI today
can be solved in software
in a quarter of the time
because some new software
thing that comes out.
What was it I saw? There was a
Chinese
large language model that took up a tenth
of the I don't know where that went,
but it took up a tenth of the
processing power or something like that.
Yeah. Like an algorithmic innovation. Right? Like something
that that dramatically reduces the processing power required
so people just need less. Yep. So that's
that's one possibility.
And that seems like it's not remote. It
seems like it's quite possible
in the history of software.
Another one would be
something like CPO, co packaged optics,
causing a shift
in the way we process packets
such that processing packets becomes
much less expensive or much more efficient.
Any thoughts on that, Mike? Am I in
the wrong area, wrong ocean there? Is that
something? I mean, see the the tests out
of meta on CPO on on both reliability.
The the fear with CPO is that you
if you then have an optic failure, which
is, you know, historically the thing that went
down the most, You have to replace the
entire box. You know, Meta's reports out of
OCP last year saying that they were getting,
you know, fewer failures
overall and and and lower downtime than they
were seeing with, like, their their mainline network.
Like, that's yeah. I mean, CPO is gonna
gonna going to have some effect
in the scale up arena, like, on the
AI side. You're gonna see CPO and scale
up come together. And so what if that
comes together and then you see, you know,
dramatic improvements in in power?
You know, maybe that increases the the, you
know, the density
and that
which, you know, and and and drives cost
down, and and so it it it changes,
you know, where you can build. There's a
lot of implications of that, and
and I think if you if you do
that, then the economics change and and all
of a sudden the,
the number of units that you need to
to purchase to satisfy a workload goes down.
And if that's the case, then, you know,
maybe some of the the constraints go away
and then it opens things up. Like, that's
that's a possibility.
Yeah. There's a bunch of investments and that's
in in similar sounding things. I mentioned we
were chatting earlier, but there's a whole movement
around COOS, you know, chip on wafer on
substrate.
You know, imagine if the packaging wasn't like
on, you know, the the chip or the
dye. What if the packaging was bigger? You
put entire systems together.
You can condense things into very, very small
form factors, and then the the cooling properties
change dramatically.
You know, I mentioned I was talking to
a company that effectively says they think they
can reduce a a whole row of equipment
to something the size of a coffee table.
Like, what if if that happened?
Like, what would the implications be? I mean,
there's, like, it's it's hard to imagine. Like,
that kind of density, you know, it it
it changes everything.
Something like that would would change the way
we think about infrastructure entirely.
And then your constrained resources become, you know,
the it it it
I think, you know, Yvonne, you said that,
you know, the bottleneck moved to somewhere else.
I mean, that would that would move the
bottleneck.
Yeah. Yeah. I know that Jeff Tensorro said
the other day on a podcast I was
on with him
that
NVIDIA, I think it is,
has
CPO co packaged optics that are field replaceable
now, and that may be involved with the
meta
work that you're talking about.
Now on the other side, there are naysayers
for that.
When I was at AutoCon, I ran into
the most interesting
person,
weird that this happened. I ran into Lynn
Bozak.
Let's go. Yeah. Who was there,
and he was there representing,
what's the name of his company? Let me
see. It's,
XKL,
which is a an optics company, and they're
mostly focused
on long haul. And he was telling me
he's pretty convinced that you can't get the
power out of c p CPO
co pass co packaged optics that you need
for much more than, like, short runs between
racks.
So I don't know. You know, there's naysayers
in that space. Well, but even your naysayer
opens up an option. Right? So let's say
that that he's right. Let's say that all
the power goes the other way. Let's say
that instead of seeing vast improvements,
we see that the power you know, right
now we're going a 120 kilowatts per rack
to 600 kilowatts per rack with the Rubin
stuff.
What if what if we the the what
if the infrastructure outpaces
the the power generation? Then
maybe the innovation so if innovation if constraints
create innovation, then maybe we start doing more
work over wide area.
And so instead of having big centralized data
centers, you have federated, like, very distributed data
centers. Yeah. In Europe, I I don't know
if I if they're I I've I've read
it, so I don't know how to pronounce
it. Is it SYNNQ?
S y n n q? Like, they're they're
they've set up a federation between Finland. They're
doing AI clusters
between Finland and and and Italy.
And so they're putting geographical distance. And if
that happened, like, what if the future of
it wasn't
super density? It was like super distribution. Like,
that would also change everything.
And and that would have interesting implications on
things like data sovereignty
and those kinds of topics. Right? Yeah. So
and and so that is a new,
I you know, they there could be advantages.
There could be new constraints there. Right?
So,
yes. I I I'm I'm confident that
the the the market will find a solution,
and eventually, we'll see a new area of
constraint.
Right? The the the challenge is
either either predicting or then navigating in these
in these times
of severe severe constraint, frankly. Because if you're
if if you're a smaller purchaser
right now and you desperately need certain gear,
it's really, really challenging,
which which is gonna force you to innovate.
Yeah.
Well, so what do you do? I mean,
like so if you're let's say you're, like,
a medium sized purchaser. If you're a small
purchaser, you might lack the
the human power and the,
you know, and then the purchasing power to
to do much about it. But if you're,
let's say, you're in the middle of it.
You're not at the at the hyper scale
level of consumption, but you're not at, you
know, Joe's Tire Shop level
of of of consumption.
How do you even, like, how do you
prepare architecturally,
operationally,
even, like, from a workforce perspective? How do
you prepare?
Because if you don't my my, I guess,
my my hesitance is that
if you go
in,
and and you don't know
you haven't thought it through and you put
nothing in place, then you're gonna find out
that, like, the people are gonna you're gonna
rebel, and it's like you can't do anything.
Or
your tooling isn't going to extend to the
new thing. And so
you're architecturally kinda locked in, and you don't
have the ability to even give yourself choice
and control. I mean, how do you how
do you even prepare to get control back?
I think you have to think about
I think you have to think about portability.
Like, as a Yeah. A lot of us
have we have these mantras when we're building
something. Survivability
is really important or,
you know, or resiliency is really resilience is
really the biggest thing we've gotta deal with
and everything else could takes a second chair
to that.
I think one of the having as a
design mantra that portability
is is the most important building this thing
to be moved to another thing, and we
will sacrifice resilience and we will sacrifice
survivability
for this.
And optimization.
Yeah. And optimization.
Yes.
Exactly.
Should be one of the first things to
go in my opinion and one of the
last things that engineers ever wanna let go
of. Right? Yeah. Exactly.
Yeah.
So I would start I would start with
the tooling. Right? That's where I would start.
I mean, start with something that's an open
source,
widely available
tooling system that orchestrates
many different platforms,
extend it to the platforms that you need
to extend it to,
find people to extend it to those platforms
as the case might be.
Pay people to do it if you need
to. They go back to Yvonne's. There are
contractors out there. You can go hire somebody.
Oh, this orchestration system I want to use
doesn't orchestrate,
I don't know,
vendor x. Fine. Go find somebody who can
write that code and get it done so
you can orchestrate to that
and start building an abstraction there between
you and the hardware,
between your engineering
and the hardware.
Perhaps to me is like
a key way to build that portability
Tom's
thinking about or talking about and,
you know, and and be willing to give
up
the optimization, some of the optimization
to get there.
But I think
yeah.
Where I was gonna go in in a
much more generic kind of a way is
you got to figure out what you've really
got to have.
What do we really have to have and
what are the things that we can sacrifice?
And that's, that's kind of what Tom was
talking about by saying maybe,
maybe resilience isn't our number one
design goal now,
right? These are the things that we have
to have to deliver and to keep the
business running and everything else becomes secondary to
that. And that can be really challenging
when you've got,
when you've been operating under a in a
different environment for a long, long time. But
I mean, ultimately
and then I think you have to talk
about
what you're doing and why. Like these are
this is what we have to do. This
is what we'd like to do. These are
the constraints that we're facing. There's this great
moment in the movie Apollo 13, right, where
they realize that there's been an explosion on
the lunar module and they've got to get
people home and they can't filter the air
and and they're the interfaces don't match and
somebody walks in the room with a bag
and dumps it out on the table and
says,
this is what we have to keep them
alive and get them home. Now go figure
it out. I mean, you've got to kind
of have that moment, right? This is what
we have to keep our business alive and
these are what things we need to work.
Now. We've got to figure it out with
these things. Yeah. Well, you have to move
your optimization other places sometimes. Right? And your
resilience other places. You can't count on it
all the time to be there. Like when
I teach data center fabrics, I always talk
about
the importance
of minimizing
jitter. Well, you know what? You may have
to rejigger your application
to support a data center fabric with higher
jitter
because you just can't buy the hardware
that you need to get to
to build the application the way you want
to. That's a good example.
It just may be
a a thing.
Oh, well, that might mean I need to
distribute my front end, as Mike says distributed
computing, more distributed stuff,
out to a CDN instead of always storing
my front end server
in my own data center hosting in my
own data center.
Okay?
Well Maybe. Yeah.
You know?
That's that's the decision you've gotta make.
And so
yeah. And that goes back to Tom's point
on portability
right there.
Build things so they can be
moved other places.
I think some of the text coming along
to make it to make some of the
abstraction possible.
I'm not
I tend to be more of an AI
pragmatist. I don't know that I'd like to
breathe a lot of additional hype into that
bubble, but,
but I I do like natural language as
a front end as a means of providing
abstraction.
Because I think if you can get through
a lot of the day to day using
natural language, then
some of the reliance on on, you know,
rote memorization of complex syntax. If if you
can get rid of that a little bit,
then I think your workforce can kinda come
along. I do think it comes at the
expense of knowing the details, which I think,
you know, that part is
that's a that's a difficult trade off to
make in in the absolution.
I think in in different parts of your
workforce, it's a little bit easier.
But I think that gives you at least
buy you a little bit of breathing room.
I I'd be shocked if the if the
interface of the future is not closer to
natural language.
You know, and then and then use the
complex queries for troubleshooting and and monitoring that
kind of stuff. Like, make that stuff, you
know, quite a bit more,
specific and and prescriptive.
Yeah.
Yeah. And and and the thing with natural
language and AI front ends is what always
scares me a bit is that
you have a transition period
where there are people who lived
growing up and they and they knew the
technology.
Right?
And so the AI LLM or the natural
language interface
actually helps them to create better and faster.
And then you have a a generation that
comes after them
that doesn't know anything but
that
natural language interface.
And now the productivity
goes through the floor
and everything becomes stale. Okay. And I'm gonna
give a really weird example of this in
my head. But one of the ways I
think about it is in music.
So I know on x and various other
places, I hear people say all the time,
even my kids
who are not that young any longer,
but they still listen to the same music
I listened to as a teenager.
Right? And you think,
that's a twenty something year difference.
Why are they listening to that music?
And to me, if I look at the
music industry,
we started out with marambas and
acoustic guitars, and then they added electronics to
it. And there was a point
around the time of Kansas Styx, that time
frame, eighties, '70 late seventies, early eighties, there
was a lot of innovation
in music
around electronics,
and people could do a lot of really
cool stuff.
But the people
doing the cool stuff
grew up busting their fingers
on steel, you know, on steel guitar strings,
and they still had the feeling of how
to create music.
And so they used the tools to create
music.
When that generation went away,
modern music to me is all just bland.
Every song on the radio sounds the same
to me. It's all developed with the same
stupid
whatever whatever.
And so
there comes a point where I wonder about
that network engineering.
Like, at some point, are we gonna lose
the people who develop the protocols,
who fought
with the constraints,
and then develop tools to get around the
constraints.
And now the tools run the network and
not the person.
And and that that's a weird I don't
know. One step from being the angry guy
shaking a cloud shaking a cloud.
Get off my lawn.
Well and I think, like, I'm I'm kind
of a pragmatist when it comes to those
challenges. I mean, so yes, that may be
true. That may be happening.
Yes. I mean, you look at architecture.
Right? I mean, there was there was so
much richness in architecture.
Oh, yeah. Probably
the same way. 500
ago. Right? Even a hundred years ago.
Yeah. Like, it's not the same now.
Yep.
Like,
some of these things I think are just
the march of time and innovation and, like,
there's just nothing we can do about it.
Right?
I don't know. I don't I don't like,
is that a solvable problem? I don't know.
Because I I feel like we've got plenty
of examples that say
there the the creativity will move to a
different place of of humanity.
And, you know, I don't know.
Maybe it's just the market. I hope you're
I hope you're right, Yvonne. I'm kinda sanguine
about it because I don't that's like out
of my control. So
I think the instrumentation in networks today is
like so much better than it was before.
I think there's advanced I mean, the the
open source tooling that's available is
is way different. The the
the understanding of how to build an open
source practice inside of companies is
dramatically different than, you know, than even ten
years ago, fifteen years ago.
So I think a lot of stuff kinda
comes along, you know, the
the
I I I so I kind of agree
and kinda and kinda I'm not sure, but
maybe I'm more hopeful.
And the thing that gives me fear is
that people if if you relegate the network
to electricity where you don't, you know, even
like, no one turns lights on anymore. You
just walk into a room and lights happen.
If you relegate it to that kind of
utility, then I think
the number of people who pay attention to
it is is fewer, and and that's a
harder
like, that's the part I worry about is
you could drain
the workforce.
I but I do think that the the
workforce that's there with the the tooling that
they have,
it does give them more
capability than they otherwise. They're relying more on
the car, maybe on the tooling. Maybe it's
more like a, you know, stick shift in
an automatic,
like people can still drive cars. They just,
maybe they couldn't drive a stick, but the,
but the, they're
still able to make a car work. Maybe
that's where we end up.
It's an interesting question, how many people you
need to actually, know the underlying bits.
And then as you plan your workforce, then
the question would be, what's the ratio?
Like how many experts should you have on
staff? And how many, you know, generalists, you
know, this is a specialist in general generalist
conversation again. You know, what's the right ratio?
And so if you're planning,
maybe people should look at that and say,
let me get my hands on
the the three specialists I need now.
Let me make sure they're comped and and
good to go. And then let me set
up let me hire the rest, and then
let me put together a mentoring plan so
my three specialists are always mentoring the next
three that follow.
Yeah. Yeah. And this is all go this
still goes back to
be living in a constrained environment. Right? This
all still goes back to how do you
manage
things when you're in a
constraints in hardware, we are in in just
but we also face constraints
in in expertise,
in
in just knowledge.
And this, like, this intersection
of constraints that are causing
a lot of heartache right now.
The workforce constraints are are just as pronounced
as the the hardware constraints. Everyone right now
trying to build out GPU clusters.
The number of people, you know, it's like,
oh, I want someone who's got, you know,
five years experience with GPU clusters.
Like, go ahead. There are 10 of them
in the world. Right? Like, you know, it's
something like that. Yeah.
Yeah. It's it's it's insane.
And again,
this is where the generals comes back in
and says,
but I know about networks. I know how
to build data center fabrics.
I can pick up how to build a
GPU cluster
by just adding some bits.
I I can figure that out.
You don't need somebody who's done it for
five years or ten years exclusively
for probably 80%
of the GP clusters that need to be
built.
Maybe. I don't know. What do you think
of that, Mike? Is that a crazy?
I think I could probably get to the
same conclusion, but a different way. I just
I don't think that
and this is, I don't I don't mean
to wax philosophical on this, but,
like, I I think companies
are with with the whole AI thing in
particular, I think there's a lot of companies,
a lot of or maybe a lot of,
you know, execs that are looking to say,
how do I
effectively reduce a bunch of cost, and how
do I, you know, usher people out the
door? I think there needs to be, like,
a some loyalty there, and people have to
say, I'm committed to bringing people along. And
I think if you take someone who's motivated
and you put them in an environment where
the conditions are are healthy,
they can make the transitions.
I think companies should invest in that. I
think, one, it's a practical reality. There's not
enough people, not enough resources, generally speaking, that
have the kind of experience.
Two, I think that technology
in general is interesting. Technology contextualized
is difference making, and I think that's the
contextualization
that matters. And so there's it's in your
best interest anyway to do that.
And then I think if you do that,
then you can you can you can still
do staff augmentation, but I don't think you
need quite as many people. And then just
practically speaking, if you look at the economics,
if there's very few people who understand a
particular
skill or who have a particular skill,
then they're gonna be in crazy demand. And
so if you don't have the budget to
compete with a a Microsoft or an Oracle
or a or a Meta,
then
maybe what you should be doing is is
building out your own team and and, you
know, bringing people along. That's what I would
do. And so as you as you think
through, like, the the strategies, then it's really,
you know, again, going back to some of
the abstraction comments, it's like, you know, what's
the level of expertise I need?
You know, how many specialists do I need?
You know, what what am I optimizing for?
What are the architectural
concessions I'm willing to make? And then what
are the and then how much optionality do
I wanna maintain? How many sources of whatever
do I wanna have? And then use that
to dictate,
you know, where you where you go.
But that's very different than, you know, new
new project comes in, you know,
pull out the old RFP, you know, add
three lines to the Excel file and then,
you know, send it back out and and
and work on the next RFP.
And how do I keep people once I've
trained them?
And that's and that's and I'm just saying
that's something you gotta think about. Right? That's
always an issue.
Once I put people in the past That's
a whole podcast, man. Like, I know. I
know. But it is but it's just something
to consider.
I think you gotta pay people what they're
what they're worth before they ask for it.
You create an environment where they wanna be
and
keep other people from stealing them, which, like,
people are only gonna steal somebody away from
where they wanna be if you're you're underpaying
them. Like, if you pay them what they're
worth and you put them in an environment
where they're reasonably happy,
like, generally,
they're not gonna move. Like,
I know that sounds super basic. But Oh
my god. That that if you're willing to
pay somebody
after they've raised their, their their their counter
or their offer from someone else, If you're
willing to pay them then, then you should
be willing to pay pay them without the
offer.
Like, oh my god. Do right by your
people. I'll I won't go deeper on that,
but but Yeah. Well, maybe it's a good
show to have. Maybe we should talk about
it sometime. Even though I'll just say that
there is a con currently a constraint around
the kind of thinking that Mike Bouchon just
expressed on this podcast.
That's another constraint that we have. Yeah. Oh,
yes. Of course.
Alright. Cool. Well, I think we've hit everything
unless you have other things you wanna say
there, Mike. Are you kinda I'm good. How
about you, Tom?
No. I'm good. And you, Yvonne?
All good. Let's call it the weekend. Let's
call it the weekend. It is Friday afternoon.
Alright.
So, Tom, where can people get in touch
with you? If they want to, say your
word.
LinkedIn.
Did you time that one?
I did not time it. No?
I'll I'll bring the stopwatch out. We'll time
them in the future and figure out what's
going on with the with the variable length
pause. And Yvonne?
Yeah.
LinkedIn and Yvonne Sharp or on,
Twitter, x at Sharp Network. I'm I'm lurking
more than I'm talking these days, but I
am still around.
And Mike.
I'm also on LinkedIn these days. I gave
up the socials
for the tire fire that they are.
Okay. Man. Yep. Cool.
Alright. And I'm Russ White. You can find
me here
the hedge. You can find me at rule
eleven dot tech. You can find me on
LinkedIn.
I haven't logged into x in a while,
but, well, you know, if you PM me,
I'll probably eventually see it over there. Maybe.
Who knows?
And
we know that your attention is important. We
live in an attention driven economy, so we
thank you for listening all the way to
the bitter end of this episode of The
Hedge, and we will catch you next time.