Join us as we gather around the hedge,
where we dig into technology, business, and culture
with the finest minds in computer networking.
Well, hello, Tom, again, and hello, Audrey, again.
Hello, again. Hello, again. Third time's the charm.
Yep.
This is our third recording of the day.
I might three d print, like, a mouth
and put it back there like we were
talking about last time. But it has to
have teeth.
Yeah. Yeah.
Like Little Shop of Horrors. It has to
have, like, teeth and be like,
roar.
Russ is doing sound effects now. I don't
know what to think of that. Don't talk
to Russ about spirit animals. Things start to
happen.
So and
the third voice you hear there, of course,
is Yvonne. Hey. Hello.
I have to say I have a new
plant.
Okay. I don't know if you guys were
around for the great HVAC,
shed HVAC failure of twenty twenty six, but
my different Vachea that was back there
froze stiff. It was, like, 15 degrees out
here.
My
HVAC quit.
And,
I've got to tell you that a
a frozen stiff dip diphenbachia does not survive.
I'm gonna show this. I know the rest
of you can't see it, but I'm gonna
show this to Russ and Tom. This is
my other,
peace lily. Oh. It actually
survived the freeze. It was out here during
the freeze, and I took it in the
house and rehabbed it,
and it's in good shape. So this time,
we decided to get a peace lily back
here, and maybe it will
endure
the temperature
fluctuations
that that happen out here in the grand
old she shed. So I know that's more
than you wanted to know about my houseplants,
but
there you go.
No. It's all good. It's cool. Yeah.
So
we have a couple of topics to talk
about today.
We don't we're guestless. This is our roundtable
episode.
Guestless.
I I don't know. That could be I
don't know. Whatever.
This is our roundtable episode. So,
we'll start with
there's an article in
APNIC, and I just noticed this by George
Michaelson,
who's on the podcast on a regular basis.
And he's talking about,
why are we building all these data centers?
And I think it's interesting, because I think
that in IT, we kind of think we
know the answer to that.
Right?
We think we think we know the answer
because we need lots of processors for AI.
We need lots of processors for all the
stuff that we need processors for.
But I think there's a lot of
risk
in this development as well and a lot
of problems.
For instance,
what if in the end we actually don't
need all these data centers to do this?
What if someone really does create an LLM
that doesn't
need
all as much processing power
as we are trying to throw at these
things?
Or,
you know, what if we find a way
via power? Because everybody's very worried about power.
I don't know if you've been reading up
about ELS or whatever it is and co
package topics and even co package copper. I'm
doing a lot of research into those things
right now, so that's what comes to mind
about
power savings. And maybe we don't need quite
so much floor space
for the compute power we even need.
So I think there is some question
there.
I think I don't know. Tom, Yvonne, any
any comments thus far? Because
I remember I remember walking around when a
company I worked for was doing an acquisition
of a a data center shell in Salt
Lake City, Utah.
And the old century CenturyLink had, sold
sold them this this data center. And it
was it was hollowed out. It didn't have,
like, finished white space and raised floor and
stuff because they pulled all that stuff out.
And I remember walking around. It's huge, gigantic
building.
It was a Coke bottling plant back in
the day.
And, I just remember walking around in there
thinking, how are we ever gonna fill all
of this? And I don't know what the
state is now, but it's probably pretty close
to full. And then that makes me think
of other data centers I've been in where,
you know, as the life cycle goes,
lots of lots of racks of compute get
condensed into one rack of compute, and you've
got all this all this,
all this floor space. And I'm not sure.
I don't know. I'm sort of, waxing nostalgic
for no purpose here, but this seems to
have this is always happening. Like, this is
always like, oh, we need all the space.
Load it up. Oh, cleared out. It's all
smaller now, and it seems like it'll go
on probably forever.
Yeah. I think there I think there is
a cyclical nature to it. I think we're
in the middle of
an emerging technology,
and
we haven't gotten
exactly to the optimization
phase yet for that technology.
Right? Like, for for example, when,
servers and data centers were first a thing,
We were entering
this place where
we were running out of floor space because
every server was a physical box,
and then we had virtualization. And then all
of a sudden, like,
this,
you know, you could fit a 100 racks
of servers
in, like, a tenth of the space.
I think I think, ultimately, we will see
some more optimization around AI
when we get there,
but I don't think we're there yet.
And
depending on your workloads,
you know,
many, many, many
of your data centers, especially your hyperscalers, are
constantly battling capacity issues.
And I even heard just today
that,
a customer had a twelve to eighteen month
lead time for an interconnect
Yep. Because they were struggling
with and I and I don't know what
the particular holdup was in this case. I
don't know if it was an optic. I
don't know if if it was just physical
plant and construction.
I don't know what the holdup was. But
it it
it's it's not I don't believe that we
are building things to sit empty.
That's not been my observation
anyway.
I I do think that there will
be process improvements and consolidation that will you
know, either we'll get more out of it
or it'll render some of that space moot.
But I don't think we're there right now.
I think I think we need a lot
more compute to run the workloads that we're
running today.
So it's interesting because this brings up the
flip side of the problem. I'll give you
a couple of quotes. I'm I'm doing a
presentation in Anag, and these quotes come from
that presentation.
And
the this is Corning. Somebody at Corning, one
of the vice presidents at Corning, externally said,
just this year, The United States will need
to add 213,300,000
fiber miles
by 2029.
213,000,000
miles of new fiber
within the next three years.
Currently, The United States has about a 159,600,000
fiber miles,
and we need to increase by the 372,900,000.
That is
insane. And so if you're four x of
what we have today when you do the
when you do the math. Right? Yeah. You
take the 100 we got now and the
300 we need and yeah. That's
in in four year. And that's interesting because
that would have not been
the constraint I would have chosen.
Yeah. And there's another one. Yeah. Yeah. RVA
LLC, which is a research company, and they're
saying 92,000
new fiber miles
just
for data center
interconnect
and scale across
in the next year.
That's interesting because it feels like it doesn't
take into account
multiplexing. Like, DWDM,
you can get a lot out of a
fiber. I sort of I'm skeptical about these
numbers because, like, you can you can get
a lot of capacity out of one pair
of fiber tripling
the fiber footprint.
I don't know.
I it seems as little far fetched to
me.
Well, but I think I think what's happening
is is that
data center builders are running into construction
and power
constraints.
Right?
There's only so large of a field you
can put a data center in. And a
lot of communities don't want data centers over
a certain size. They're doing data center constraints.
Right? Right. On the size and the and
the power usage and stuff like that. And
then there's just certain numbers of places you
just can't get power. So some people are
solving this. We'll just put out nukes. Right?
We'll just put out little nuclear power plants,
and that's a thing. And that's going to
happen. People are doing it. Microsoft bought 3
Mile Island. People think 3 Mile Island went
away. It actually didn't. It's been operational all
these years, and Microsoft bought the entire nuclear
power plant so they could supply
their electrical needs for,
a massive build out of compute and stuff.
And,
I
the thing is is that when you do
that, when you start segregating things, you actually
need to split your
fabric across multiple buildings regionally.
And you like, if you're actually gonna split
a fabric, and I'm not not not not
like traditional DCI,
but, like, I'm gonna take two data centers
with five stage fabrics or four of them,
and I'm gonna make them into a nine
stage regional data center.
Like, the what the amount and you can
you can do DWDM, but there's only a
certain number of channels you're gonna get. And
you're looking at 64 cross connects
or, you know, a 120, whatever it is,
however many cross connects this is that you
start looking at the scale of these things.
And, like, yeah, that's this is a problem.
And so this is, like, the flip side.
Right? Where are we getting this stuff?
Like, seriously?
Fuck. Who's who's building
300,000,000
miles of fiber optic cables
or whatever it is
in the next three years?
Sure. Who who's creating that?
Where are those plants? And those are truck
rolls. Right? These are very
physical. I mean, you're you're
you're burying cable. You're you're rolling a trenching
truck. You're it it is a it is
a heavyweight
endeavor
Yeah. To to run fiber like that. You
know, we're not we're not even talking about
constructing
a data center, a building in a place.
We're talking about
significant
truck roles and infrastructure
and and all of that. Yeah. It's it
it I'm with Tom. It kinda boggles the
mind a bit. Like, that that's like,
you know, 1999
kind of filled out numbers. You know? It
it feels like
that. Yeah. Yeah.
Which which kinda drives more of the the
talk of, you know, can we actually do
this? What happens if we don't do this?
Right? Does this does this hurt the bubble?
Does this hurt the deployment
in a serious way? Because we just physically
can't build all this stuff. And and by
the way, a funny story from this week,
I think it's funny anyway,
is that there used to be a shoe
company called Allbirds.
Yeah.
They are now an AI company.
I thought we were gonna talk about AI
today. I know. We're trying not to.
But it is but it is rather humorous
to me. Right? Like,
I don't know where all this stuff is
coming from.
And I don't know how we're gonna build
all this data center space. I just don't
know how we're gonna power it. I don't
know how we're gonna water it. I don't
know how I just don't know how we're
gonna do it. And so that is that's
like an issue to me.
And communities are mad.
Like, I've seen videos of people who live
next to a data center that went up,
and people say, well, it's not that loud
over. And they just put their video camera
on their
porch. And you can't talk to anybody, each
other in the yard
because of the chilling towers
and because of the the power.
And it's like, I don't know. Like, I
if I live next door to that and
I couldn't go outside and play in the
yard, I think I'd be pretty upset.
So I don't know.
It will there be a reaction to this?
Will it slow down?
So I don't know. Anything else on that
one, Yvonne?
I do believe that in the long term,
we'll solve the problems. When
I I struggle sometimes to understand the pushback
of data centers because I live in a
world where,
in my state, we're known for auto manufacturing.
And I think, man,
a data center is nowhere near as disruptive
as a car plant. You know what I
mean? Like and so I have a I
I, you know, I kinda struggle with some
of the pushback,
to to data centers and, you know, I've
toured some big data centers that weren't loud
like that.
So I, you know, I I don't know
what to make of all that pushback. I
will say that.
But but,
yeah, I'm I'm like Tom. I'm flabbergasted by
that much fiber being needed Yeah. For the
next three years. It just seems crazy. I
mean, there's just I don't know. Alright. So
the second thing we're gonna talk about is
the foreign made router ban in The US,
which is fascinating
because I'm not entirely certain what to make
of it from a policy perspective.
Right? Is this really security or is this
really
rent seeking
in some which way or form? And I
don't know. I've not seen enough
about it to know
what I think. I know the FCC has
banned foreign made routers based on
security concerns.
And and in some ways, I actually honestly,
in some ways, I don't actually blame them.
Because many of these devices do have backdoors.
People on the Internet have done put
packet sniffers behind the home router,
and just there's the amount of data that
gets shipped to weird IP addresses,
and nobody really knows what it's doing,
is it's a pretty incredible amount of data.
And so I I kinda get,
and the possibility of their backdoors and stuff
like that. So I kinda get all of
that.
So I don't know. What do you think,
Tom?
Any I, I bought a I remember I
bought a super cheap
network,
connected
PDU
for my home lab, like, I don't know,
ten years ago or something. Had three outlets
on it, and you could and an Ethernet
port and you could remotely power cycle. And
then so I was just for fun. I,
I I captured the packets coming off of
that thing going outbound.
And it was pretty it was pretty hilarious.
It was a Chinese, device and and like
several IPs in China. And what was it
sending? I don't know. It was all HTTPS,
like, but they were pretty big packets. I'm
like, what are you saying about me
and my and my my network and my
power? And Yeah. So I I sort of,
you know, I
I sympathize. I think, like you, I think
I'm like, okay. There there probably are
legitimate security concerns.
It seems like an interesting way to solve
the problem, but are there other better ways?
I don't know. I I don't spend too
much time thinking about public policy and telecom,
but
Yeah.
It it may be the only hammer they
had to solve this problem. I have no
idea.
So, Yvonne, thoughts? You well, I mean, so
we we we all remember, like, the DDoS
attacks that were sparked by botnets,
botnet armies that attacked, you know, either IoT
devices or consumer grade routers that had,
and and some of some of that
is is
you know, it's it's hard to blame the
consumer
for not changing their
admin admin username password when they are, you
know, a nontechnical
consumer. Like so,
I, you know, I think it's clear
that there's absolutely has been some risk and,
some misuse
of those,
consumer grade
routers. Now,
you know, we could ask the question, does
does this actually solve the problem? I don't
know. Is some of it security theater? I
I feel like there's a lot of theater
going on these days
in the public policy space, so I I
I wouldn't put it past that. But, I
mean, I do think there was a
legit
problem.
I do think we've solved it with a
pretty blunt tool that
is is not gonna completely solve the problem.
But,
you know, it it is gonna create some
local markets, I think.
You know, and and and I think it's
also you know, we should be really clear
this is consumer grade gear. This is not
our enterprise corporate gear that we're talking about.
Yeah. But, yeah, I I, you know, I
think it's interesting.
I I see the article. I don't know
if it's been long enough for there to
be any documented,
you know, what are the impacts been if
there are downstream impacts. Are there, you know,
are there now
is there a supply chain problem with consumer
grade networking gear? You know? I I don't
know any of those things, but I think
it's,
I mean, I understand why we did it,
and I think, you know, the security implications
are real, and we've seen them. Many of
us have been on the, troubleshooting end of
them. Yeah. Whether or not there was another
solution, I have no idea. Whether or not,
you know, what other factors came into play,
but I agree with you. There there was
a there is a problem there. I think
one of the positive things I've seen out
of this interestingly
is I've seen a lot of videos of
people saying,
you know,
just stop buying consumer grade routers.
You can just go buy a Raspberry Pi
or a little used box
and make yourself make it into a router.
Now that doesn't solve the Wi Fi problem.
Right? That only solves the gateway problem,
but you actually get more control
over the device
because there are open source packages you can
download. You can use a Ubuntu or Debian
or whatever and put it on a a
lightweight version or or even, Apache or whatever.
Put it on a lightweight not Apache, whatever
it is. Anyway, and put it on a
on a Raspberry Pi and load up a
lightweight
software package that does routing.
And,
like, there's not many routes on these things.
There's no routing protocols.
Right? It's just it's just a few static
routes. It's a net box and a static
routes. It's pretty much it. Yeah. Pretty much
it. It's all it is. That is such
an engineer's
answer, though.
I'm trying to imagine
giving that answer to my parents who are
deeply nontechnical and in their seventies.
Right? And saying, hey.
Just go on Amazon,
order this Raspberry Pi,
and
do these 10 steps.
Right? It it it's it's just not Just
just configure IP tables, not a big deal,
and choose from 50,000 Linux distros. You can
do it. It's you know?
Yeah. Yeah. I don't know. I think I
think it's time for a trip to your
parents' house, Yvonne. I think that's what you're
saying.
I'm gonna give them your cell phone number.
I mean, I don't know. You know? But
that is but that is an issue. Yeah.
The the joke in my world is if
my phone rings at 07:30 or at 10:30PM
and it's my mother,
either somebody in the family is critically ill
or she's having a technology problem. It is
one of those two things.
And and I don't think I've ever been
proven wrong there. So yeah.
So the nice thing is my mother
is
not hardware savvy per se. Like, she can't
tell you what the connectors are and stuff
like that. But once you get the hardware
up and running, get it booted,
she's fine.
Because she used to code bash strips and
scrap bash scripts, and she knows Ubuntu and
Debian. And, you know, she actually used to
do lightweight shell scripting and stuff. And she
did a lot of accounting work on Linux
systems. And so she's you get her to
the boot prompt, and she's like, I'm happy.
You know? Yeah. She doesn't know
HDMI versus VGA,
right, any longer because all that has changed
since she was in the doing anything with
it.
But yeah. It it it took my parents
a while to come to the conclusion that
I wasn't throwing my career away with these
silly computers.
So yeah. Yeah. They they finally got there,
but it took a while.
Yeah. I don't know about computers. I don't
know if there's a future in any of
this network engineering,
computers.
Now
now robots and AI are gonna take the
whole thing over, and we're just not even
gonna know. And computers have nothing to do
with those things. So
I'm I'm I'm waiting to
buy my own robot army. I am completely
on board with that idea.
Oh my goodness. I did see by the
cloud.
Yeah. I did see somebody the other day
who was doing sheep herding with drones.
She's quite clever. I mean, I prefer the
dogs because the dogs are cuter, you know,
and they're fun and everything. Some companionship and
some emotional support and all of that. You
know? Exactly. But okay.
I guess what if it floats your boat?
My husband has said if you get a
robot, you cannot have it working while we'll
sleep while we sleep
because I will wake up in the middle
of the night and do it harm.
But if if I see a humanoid,
you know,
option moving around our house, I'm going I'm
going to strike first and ask questions later.
And I'm like, fair enough.
Okay. So our third article is a little
more comic relief.
So I remember
now this this goes back many years
when cloud first came out,
and I was working with a large enterprise,
quote, unquote,
large operator,
non provider operator.
And their vice president
had said, come back off of some trip
and said, we had all these presentations
on cloud.
What is our strategy for cloud? What are
we doing for cloud?
And so they went into all their network
diagrams
and renamed their data centers from data center
one five whatever
whatever
to cloud
with exactly the same numbers after them.
Okay. And they said,
we've already deployed it.
Like, we don't we don't know we don't
know what you're talking about. You know? You
went to some conference about this. We've had
this deployed for years. Like, I have no
idea what you're talking about. We're ahead of
the curve.
Nice.
Are you allowed to laugh at that joke,
Yvonne?
I can laugh at anything I want to.
Thank you very much.
So
so
yeah. So this article in the register, the
third one we picked picked up for today,
is
about
proving or showing your your managers or your
upper level management that you have deployed AI
in your network operations.
And it's and it's very it's a brilliant
I think it's absolutely brilliant.
It's just call it a zero token architecture.
My my favorite I just have to quote
something from this because it made me laugh
out loud. He's he's talking about continue using
automation tools like Puppet, Ansible, and Chef, and
and and put stuff in Cron.
But just rename that to etsyagent.d,
and you'll have all these agents doing all
these automatic things using the zero token architecture.
I love it.
It
it it would be less funny if it
weren't so true.
Right? But we have when you've been in
this industry for a while, you've seen the
the continual rename
and,
you know, buzzword chasing that goes on. And
so, like,
and for like, we we've been naming products
AI for a long time that are that
are just doing, like,
automation or data analytics or,
one of those things. But,
zero token architecture is,
I think, next level, and somebody
deserves
some clever points in my opinion.
This this ought to be a presentation at
every nug and every nog for the next
two years. There ought to be a standardized,
you know, like a standardized
presentation
on using zero token
architectures.
It's absolutely
brilliant. But, I mean, it makes a good
point, though. Right? Too. On the serious side,
it makes a good point. Sometimes,
you just need to, like, roll with the
punches.
Just like,
you know, oh, that's the buzz buzzwords you
wanna use this week? Okay. Well,
I can Right. I can use that buzzword
too. Like, whatever.
Okay. It's And, and and this is a
and this is a great like, the great
Kelsey Hightower
is is quoted here in coming up with
this quip, which is
even more fabulous.
Right?
But I I think it it
there there is an underlying
point here, and I think a lot of
times when organizations start talking about the things
that they could do with AI or exploring
that,
what they started to discover
is that, oh, wait. We could do a
lot of these things with traditional automation.
Right? A lot of it is very deterministic.
We just need to understand our workflows and
what we wanna accomplish,
and a lot of this is still
standard
automation. And a lot of those skills
that we've needed to automate our environments,
they're they're still very necessary. I mean, when
you're talking about
multistep agents,
it's
really just an automated workflow.
Right? Yeah. Yeah. Yeah. You might be using
natural language to to describe to the system
what you want it to do instead of
writing deterministic
code,
but it's still, like, step one, step two,
step three
that you are automating
in a system
that you're kicking off with a scheduler that
we used to call Kron,
and there's still some similarities
there.
Yeah. Yeah. And I think it's also interesting
because sometimes when you wanna get something done,
if you're a network operator and you've wanted
to automate your network for the last ten
years, and you could never get budget to
do it,
and you never get time to do it,
do it through a I mean, use
the
use the flow. Use the hype to your
advantage
to get done what you've always needed to
do or wanted to do
and get the budget and get things that
you need to do. And I think that's
a fantastic,
like, another fantastic way of of thinking about
this renaming of stuff.
Well,
another great quote here.
Kelsey says the agent will burn $2,000,000,000,000
with the tokens and call it an API,
he joked. He also shared a four letter
acronym he recommends as an alternative,
b
a s h for the bash shell, which
is, you know,
it's it's it's quite clever.
Yeah. But but his point is well taken
that
there are many solutions out there to our
problems, and some of them don't require tokens.
They require
a a deterministic,
bit of code that's gonna run on a
schedule
like a cron job
that is going to accomplish tasks.
Yeah. I love it.
Yeah. It's really it's really cool.
Alright. Well, I think that's all I have
for articles to discuss.
You have anything else, Tom?
Nope.
Anything else, Yvonne?
All good.
Alright. So, Tom, where can people find you
if they want to?
LinkedIn.
I I really I think we should just
do compression on that part
of the show.
What do you think, Yvonne? We miss we
miss things. Something's lost
when you do that. Yeah. Yeah. For us.
You mean?
So, Yvonne, where can people find you? Yeah.
You can find me on LinkedIn at Yvonne
Sharp.
You can also
find me at the Cloud Gambit, which is
part of the Packet Pusher's Network, my
other podcast,
or you can find me. I'm more lurking
on Twitter and x these days than I
am posting,
but I am over there at Sharp Network
if you wanna reach out.
Cool. Awesome. Alright. I'm Russ White. You can
always find me here at The Hedge.
My only podcast. I'm not Yvonne.
I have, you know,
Your one trick there. Says statement.
Yeah.
You have a day job too, Russ.
And a double header soccer game to take
a kid to tonight, just so you know.
Just watch
for
the ball. Don't get hit, Yvonne.
Yep.
I'm gonna put my AirPods in and listen
to my audiobook
and
clap at the designated times.
Okay.
Alright. So I'm Russ White. If I'm here
at the hedge, you can find me at
rool eleven dot tech. Find me on LinkedIn
and on x from time to time. I
don't know. I'm pretty easy to find. We
know we live in an attention driven economy,
and we thank you for spending the time
listening all the way to the bitter end
of this episode of The Hedge, and we
will catch you next