Geoff Huston 0:00 Wasn't it? Robert Frost, the American poet who said, the past is a different world. They did things differently there. It's true. They did it is a different world. But anyway, there it was. And this model was then instantiated by you've kind of built that cable as a common, shared resource. And then the shareholders would come to a shareholder meeting, and they would pair up, pair up, and go, Well, I want to get some telephone capacity between my company and the telephone system operator by Company B. So I would like to take out a certain amount of capacity on this cable. It doesn't mean anyone else couldn't do it, but, you know, I want to get some capacity on this cable. So they had this thing called an indefeasible right of use, which gave you guaranteed access to a certain amount of cable capacity. And normally it wasn't in perpetuity. The cable wasn't there that long, but there were time for about 15 or 30 year leases of this capacity for these so called IRUs, and that was a committed access to a cable George Michaelson 1:14 you're listening to ping, a podcast by APNIC, discussing all things related to measuring the Internet. I'm your host. George Michaelson, this time, I'm talking to Geoff Huston from APNIC labs again in his regular monthly spot on ping Geoff and I discussed the international submarine cable industry from its Victorian roots with gutta percha and tarry paper wrapping up the copper wire into the modern age of fiber optics and the way data bandwidth has scaled along that timeline. A modern concern with our dependency on these optical fiber links is a reprise of the historical cutting of telephone cables conducted in wartime to force strategic communications into radio where an enemy could listen in nowadays, the mechanism to listen in isn't exactly the same, but the risk issues around who deploys a submarine fiber link and who manufactures the digital signaling equipment, and even the fiber itself has become intensely politicized, and this is affecting How Internet links are being planned and deployed worldwide. Geoff, welcome back to ping. What should we talk about this time? Geoff Huston 2:26 You know, somewhere inside stuff that's on my screen, and there's a lot that passes my eyes, I've just come across a quote from John Perry Barlow of 1996 wasn't he with a Grateful Dead or something George Michaelson 2:42 he should have been. Is this the information quote that has the word free in it or a different quote? Geoff Huston 2:48 No, a Declaration of the Independence of Cyberspace. Cyberspace does not lie within your borders. Is kind of a very good summary of what he said. But he was trying to sort of put forward the view that this digital world valued at what 100 trillion dollars, most of the world's GDP seems to be tied up in. It doesn't have geopolitical boundary. George Michaelson 3:12 Great, problem solved. Amazing. How did you do it? Geoff Huston 3:16 Only an aging hippie could come up with such wonderful ideas. But you know, the tawdry world lives differently. And what I'd like to do is actually look at these geopolitical issues and apply that lens of realism, you know, money and politics into the fascinating world of subsea cables, the things that actually bind the continents together. And you go, well, that's not the only thing that doesn't satellites are up there too. The answer is, Well, yes and no, satellites have their own problems, [George: yeah] And be perfectly frank, there's just not enough of them. I can put terabits per second through fiber, [George: yeah] Try doing the same with unguided signals through air. You know, it's doesn't work the same way. So we're going to talk about sub sea cables and communications. George Michaelson 4:07 There are an awful lot of bits of wire down there, under the sea Geoff. I've been looking at the tele geography maps lately, looking at them by year, looking at them by location, and God, it must be impossible not to trip over the things if you're walking on the bottom of the sea. In some places, there's a lot of tech in the ground, a lot Geoff Huston 4:30 Well, don't walk on the bottom of the sea, George, it's actually not that far back. I make. The original Trans Atlantic was late 19th century, or sort of three quarters through, or something, single piece of conductor, I think it was iron with a whole heap of gutta percha as an insulator. The sea was the other half of the circuit. And, you know, they laid it down. I think Isembard kingdom, Brunel was actually engaged in that somewhere, and managed to actually get a cache. Table that telegraphed or sent Morse code from England through to, I think the first station was in Newfoundland. It wasn't all the way to America, but, you know, let's lie a bit and say it was worked for a while, but then again, [George: stopped] Well, you know, Gutta Percha isn't exactly waterproof. It started to leak through. They kept on upping the current, [George: yeah] and burned it out. Whoops. But that didn't stop us. We built and we built and we built, [George: yeah] and when kind of we got good at this? And we got good, I suppose, in the early part of the 20th century, the telephone companies had come come across, and we actually built an International Telephone System. And what made this work? Well, submarine cables were the backbone of this. [George: Yeah] as the years and the decades progress, we got progressively better and better. Originally, they were analog. They use frequency division multiplexing. We then got into digital signals and started to actually build digital cables, still based around conductors, and I think bizarrely, and you think about the electronics involved, they still had electrical amplifiers [George: down] there, 1000s of meters under the sea down there, George Michaelson 6:14 yeah, down at the bottom, think about designing systems that you're going To shroud in plastic and put two three miles down in a trench that you just need to work because the last thing in the world you want to do is pull them back up to fix something. Wow, Geoff Huston 6:30 Wow. So the engineering, and it was transistor based, you couldn't have done it with valves. Let's face it, was certainly amazing. But there's another thing about this, and as you say it was quite a big effort, a big sneeze. You didn't want to build for today's capacity, because you'd have to be building another one tomorrow. [George: Yeah] what you actually wanted to do was to actually over build. George Michaelson 6:52 Even back in those analog days, people could see capacity, potential demand to use this absolutely exceeded supply. So if you were planning on doing this, and you'd move beyond the point of putting down one piece of wire with one signal you already knew you were building for a future state that was getting bigger, Geoff Huston 7:11 yes, and even that one signal, that one carrier, you didn't sell it all at once, because if you did, the price would slump. You actually wanted to keep the price high, and you want it to keep capacity in reserve, so that when I need more capacity, oh yes, I have some in my books here. What's the price? Well, much the same as it was last week, or maybe a bit more. And so the cable system was actually a scarcity rationing system. George Michaelson 7:39 It was designed from the beginning around the economics of how you would price it. And it was not designed to price cheap. It was designed to price expensive. Geoff Huston 7:50 And these things were expensive. They were kind of like modern Air aircraft, or maybe even more expensive. They were big projects only telcos could afford it. [George; Yeah] and it had a second aspect to this, and particularly in the world of subsea systems, that it joined equipment material and services on Country A to equipment material and services on country B's territory, and often the client, or one of the clients was country C, that wasn't really, you know, at either end of the cable, land block, countries, etc, still needed to call countries far away in the globe, and somehow they needed to express an interest in cable where they weren't directly, you know, one party to it, George Michaelson 8:40 if we're talking about cables that are interacting with North America, it also brings to the table state actors, because most of the telcos were monopoly state functions in Europe, Africa, Asia and commercial operators, because America just ran to a different economic model. Geoff Huston 9:00 Oh, right up until the deregulation of the American telephone business, you know, the pulling apart of AT&T under judge green, right up until then, AT&T, if you squinted a bit, was kind of indistinguishable from all the other national government telcos. They had a monopoly. They spoke for the country. And so in some ways, it wasn't that diverse at that point in time. And so the way they devised things for submarine cables, which is where I want to head to, was actually quite an ingenious model. You all get together at a conference, preferably at a nice location with many bars around, and all the telco buying folk come to this conference, and then a couple of you wander off into the golf course or wherever you want to wander off to. And you form, well, let's call it what it is, a cartel. George Michaelson 9:52 You come to an understanding Geoff Huston 9:54 A cartel, and you agree that a cable between, you know, can't. Country A across some vast sea, the Pacific, the Indian Ocean, the Atlantic, whatever to Country B would serve the interests of all of these countries, national telephone systems. It would be a good thing George Michaelson 10:13 you might actually have said, and because we're doing that one, we're not going to do this one over here. There's kind of a bit of plus and minus. Geoff Huston 10:22 The dance card was always fascinating at these conferences. Now, what do you do then? Well, you form a company, and what does the company do? It goes to a few merchant bankers, because you do what you don't have, oddly enough, is huge amounts of capital. So you go and raise it on capital markets. Hi, we're proposing to build a cable from A to B. And here are the folk who are expressing interests, and they're all shareholders in our company. We want to actually get a line of credit to start building this enormous, you know, enormously expensive undertaking. George Michaelson 10:53 This is something that demands years of capital investment. I mean, you've got to construct the cable, you've got to hire the ships, you've got to do the geo tech survey to find the path. It's not like there's no money to be spent. There is a lot of money to be spent a lot. Geoff Huston 11:09 And at the time, you know, a Trans Pacific cable would be in the order of a billion dollars, US. And I say that in terms of, well, it's still probably a billion dollars, but the dollars worth less with Euro inflation. But it was that kind of price, and it was debt fund. It was funded by basically the banks. What was the service life? Well, it wasn't a bad idea, because that billion dollars would give you around a 25 year service life. And if you'd managed to get scarcity rationing going, you can make your money back in three years and enjoy 22 years of unbridled profit, George Michaelson 11:43 20 years of life of an asset, and we have to sit here in the modern world and think about our tax depreciation schedule for our laptops and our telephones. Are they on a 20 year life? No Geoff. They are not. So we are now living in a world where the behavior is. You get it in a 18 months to three year window, and you're talking about a world that used to exist where you put an asset in the ground with 20 years of revenue, Geoff Huston 12:11 Wasn't it? Robert Frost, the American poet who said the past is a different world. They did things differently there. It's true. They did it is a different world. But anyway, there it was, and this model was then instantiated by you've kind of built that cable as a common, shared resource. And then the shareholders would come to a shareholder meeting, and they would pair up, pair up, and go, Well, I want to get some telephone capacity between my company and the telephone system operated by Company B. So I would like to take out a certain amount of capacity on this cable. It doesn't mean anyone else couldn't do it, but, you know, I want to get some capacity on this cable. So they had this thing called an indefeasible right of use, which gave you guaranteed access to a certain amount of cable capacity. And normally it wasn't in perpetuity. The cable wasn't there that long, but there were time for about 15 or 30 year leases of this capacity for these so called IRUs, and that was a committed access to a cable George Michaelson 13:17 everyone's come together to form a construct to say, we want the cable. And they go to the finance system and they get debt funding to build the cable for a rate of money from that funding source. And then, separately, having done that, they form pairs, because there are the two economies either end. But there are lots of partners, lots of possible pairs, third parties, and they agree to divvy up the cable as these things you're calling IRUs, which I imagine Geoff I could then ultimately flog on to someone else. I mean, it would become a little value bundle all of itself. But if we put that to one side, the intention is I've got my ability to earn my revenue to cover my share of the costs of this cable and a healthy profit on top. Geoff Huston 14:05 Well, the other check and balance here is each IRU is an end to end. It's a half circuit. George Michaelson 14:10 It's a half Geoff Huston 14:11 each IRU is only half the way there. It's bit like we all meet at the satellite Well, we all meet midway in the middle. We actually match IRUs. George Michaelson 14:19 You mean literally. The idea is you fund the effort to get it to the fictional midpoint of this system, bloody hell. Geoff Huston 14:28 Well, notionally, but an IRU, you would be purchased by Telstra, another one, exactly the same IRU, you would be purchased by New Zealand telecom. I think they no longer exist, or new name, and they'd be matched. And so the full circuit, the bi directional circuit, would actually be this pair of half circuits, which is, in effect, jointly and equally owned by the two George Michaelson 14:50 man. That's complicated. That is complicated, Geoff Huston 14:54 ah, but they were trying to solve an even more complicated problem, and that complicated. Problem, which quite frankly, never got solved, was that you operate your telephone company in your country. Good on you. I operate mine in my country. You can't make money out of me, and I can't make money out of you. We have to share. [George: Ah] So the way it worked was you purchased your half of the facility with your money. I purchased my half with my money. Neither of us was funding the other. We could look each other in the eyeballs and say we're peers, George Michaelson 15:32 this peering term in cables in IRU, in telephony, sounds so close to what we carried into the modern world as BGP peering agreements between companies. It sounds like the desperate attempt. Geoff Huston 15:50 Oh George, you speak heresy. You speak heresy from the telephone viewpoint. The Internet was a shambles. It was never going to make money? It was always going to be a mess. No, the telephone company was extremely clear about money. [George: Yeah] the way it kind of worked was down at the bottom, moving calls around. It was meant to be neutral that folk operated the telephone company in their country. They got paid to do it by customers, and that paid the bills and made profit. But international calls kind of disturbed things just a tiny little bit. Now, the way we worked this out was that the person who dialed paid for not only the phone call in their country, but paid for terminating the call in the other country. Now that termination call, the costs were born by the other side, telephone company, and they claimed the money back from the telephone company that owned you, that billed you an international call accounting settlement fee to balance the books. George Michaelson 16:54 Balance. Geoff Huston 16:56 Yeah, right. It got widely abused because, you know, money always gets widely abused. But that was the kind of model, and it was this delicate balance, where the theory was, you made money in your country, I made money in my country. And even if folk in my country rang folk in your country or vice versa, it was all neutralized to some extent that you weren't just shoveling money from one country to another. George Michaelson 17:21 You and I lived through the reality of this. And the reality of this is migration from poor countries to rich countries. Happens. And what are those people in the rich countries want to do? They want to make phone calls home to the poor country, and they've got the money to do it as customers. Mother doesn't call son. Son calls mother, more calls from the rich country to the poor country. This has kind of got the problem baked in, hasn't it? Geoff Huston 17:51 Well, it's a feature of the international world that UN forces notwithstanding, the international environment has no cops, no police. So an international body, even a worthy one with the name of the ITU, the International Telecommunications Union, can't make rules. They can make guidelines. They can suggest. They can say, this would be a good thing if everyone did it. But that's all they are. They're just more in the way of guidelines than rules. And so if some country, like, I don't know France, decided that the costs of terminating an international telephone call that originated in the US, terminating a telephone system in France, was, I don't know, 500 francs a second. All that was France telecoms business. Guess who'd have to pay that rate. Well, ultimately AT & T because that was America's phone company. So Americans were calling France, it would cost a fortune. So, yes, the stuff got widely abused, very, very heavily abused. This whole idea of we were rationing scarcity to keep the price abundant, keep the price very high, and at the same time also doing a certain eclectic form of international bribery and corruption. When I said these guys were cartels, I wasn't lying. They really worried. [George: Yeah] it really was a pretty noxious system. But then, just to make life worse, along came the Internet. George Michaelson 19:15 Before we get full Internet, what you've described here is large capital outlay international finance, kind of unregulated international cartels that just have to exist, people gaming the system, governments, corporates, weird rules and laws and money, Geoff Huston 19:36 As long as the developing world was the beneficiary of this imbalance in international court accounting, you know? And it certainly was the highest court accounting settlement fees were in lesser developed countries that in some ways, because they were all government agencies, this was just another form of international aid. [George: Yeah] you just haven't been paid for by the telephone subscribers in your country. But it was. All international aid. So the answer was, well, aid is aid, no matter how it's packaged, that's a good thing. [George: Yeah], you know, yeah, we can live with this most of the time. Most folk thought, yeah, okay, except the Americans. George Michaelson 20:12 This is a pretty long way from John Perry Barlow statement. Geoff, I mean, Geoff Huston 20:16 oh no, we're moving there at the speed of light at this point, because AT & T decided that they were being done over it was costing the billions of dollars a month in international call accounting, settlement fees. And quite frankly, they weren't a government department. This was all shareholder profit being funneled off offshore. And when they went to the ITU-T, they were out voted, [George: yeah], you know, 140 to one or something, you know. And so they decided to start politicking at Congress. And while other institutions in Geneva were fine and benign and slightly curious, bodies like the International Postal Union, a happy little group of Swiss gnomes put put labels on postal bags. Cute the ITU-T was the devil incarnate. They were evil. They were swindling AT & T, and that damned them. This was un American, and I am exaggerating to some extent, but yes, it was very, very real. Along came the Internet, and there was a deliberate move all the way up to Clinton's presidency to withhold the Internet from the ITU-T and its regulations with hold international call, accounting, settlements, half circuits, everything was going to be renegotiated. And so when we started building the Internet business, and we went off to, say, an American Internet service provider, InternetMCI, I think, or something, what it was called, the answer was, what? You're not from here? Are you? We're Australian. Well, we're not going to buy a half circuit for you at our cost. You're going to pay us to buy the half circuit. You're going to pay to get your packets landed in the US all the way, all the way, [George: wow] And so if you're going to get a bill for our half circuit, and you've got to pay for your own anyway. And so all of a sudden, the money was sort of pushed against. Instead of other countries being a beneficiary of America's largesse they found that the tables have been turned completely, and everybody was paying to get to the US using a half circuit model that was well beyond its use by use by day. It just didn't work anymore. George Michaelson 22:32 The actual cables in the sea still respected the old funding model, the old debt funding model, the old construction model, and still were assets that, purely as things in service, were owned under that other framework, [Geoff: yeah], but the new rules of packets, on top of this was a totally different cost model, Geoff Huston 22:52 totally different world. George Michaelson 22:53 Wow. Geoff Huston 22:54 Now, not only that, I make it was an efficient use of infrastructure, but quite frankly, there were so many more packets than people talking. You know, for years we had, in Australia a submarine cable system capable of carrying 512,000 bits per second, which was the equivalent of about, you know, 100 voice calls or something, which was okay for years, but along came the Internet. It's kind of 512 K That's nothing. I need Meg's, man, I need much more than you've got. Let's build more cables. [George: Yeah] and there was a bit of a boom going on, coupled with the fact that the fiber guys, even in submarine cables, started to exceed their KPAs, their goals, because they had discovered, bless their little pointy heads, that instead of using electrical amplification of a digital signal under the ocean, which, if you think about it, is really, really, really hard decode those weak, little depleted photons as they come to your sensors, somehow figure out the ones and the zeros and then re amplify them for their next leg as ones and zeros for the next repeater, that's expensive, that's difficult, that's not very reliable. What we discovered was erbium doped fiber, where you actually take a length of fiber and dope it with the rare earth erbium. George Michaelson 24:18 How topical? Geoff Huston 24:19 Yeah. No, buy your erbium stocks here now. No, sorry, thimble size is all this. You put a laser and inject it into the cable, this erbium doped piece of cable, and bizarrely, what happens is these poor, depleted, tired photons, as they hit this energized section of fiber, get re energized, they get amplified. George Michaelson 24:43 So it's like pumping with light, but you don't have to have electronics down in the sea to the same extent. Geoff Huston 24:49 It's like a shot of Red Bull, you know? It's, oh God, I feel better. Let's go the next thing, no electronics, purely just lasers, and that's it. It's not completely passive, but it's close. To it. So all of a sudden we started to build bigger cables. Just at the time the Internet was wanting bigger cables, George Michaelson 25:07 significantly bigger cables, really quite significantly bigger. Geoff Huston 25:11 Oh, we were up from kilobits to megabits almost immediately. And then we started to do hundreds of megabits. And then, you know, life was moving very quickly, wavelength division multiplexing, bigger, bigger, bigger. But at the same time, the telcos still basically owned that space. It was all half circuits. It was a system that, you know, it's a bit like you're building 20 Lane freeways, but the underlying infrastructure is all about horses and carts, singularly inappropriate, inflexible, difficult to change, [George: yeah] and just not the right way of doing it. And so it wasn't long before the emerging giants, the Googles, the Microsofts, the Amazons, at the time, they were called Facebook, the metas of this world, start to figure out, you know, why don't we just cut through all this crap and build the cable completely. [George: Yeah]. And the answer was, well, they were any recommendations. There weren't rules. The only thing you can say as a country is, when that cable hits my territorial waters, my rules apply. And if you want to land that cable, put some equipment there, power and so on. Here's 24 different agencies that you need to get approval from. But once you get it, it's okay. You're there. And so we started to see the rise of the next form of submarine cable financing an organization, and it really just cut the telcos right out of it. It was fully owned submarine fiber cable systems end to end. George Michaelson 26:42 So this is kind of a submarine cable story, but it's part of the biggest story that the surface of companies that are providing communication services that used to be comfortably homed in telcos pretty much shifted to being a much more qualified statement. What kind of a company actually does Internet it is not necessarily an old fashioned telco anymore. It could be a very different structure. Geoff Huston 27:10 What kind of company is ultimately footing the bill, you know, if you get rid of the debt funding, because, you know, these digital giants definitely debt funding their own banks, it kind of the money came from the same, you know, company that was going to operate the cable, but there was still a dedicated sort of crew of expertise and knowledge about how to build them. There was KDDI submarine in Japan. Alcatel, originally based in France, which was a major player in this space AT & T I think Tyco was big there. There were a number of these companies that specialize in actually making these fiber repeaters, which was the major thing. And of course, you know, our dear friends, Crown Corning and similar to actually do the fiber strands. So, you know, subtly different world. But they were pushed into overload, build, build, build, build. And because volume and price kind of go together. As the volume goes up, the unit price comes down. Cables were actually getting to be per unit cost cheaper. And because of fiber amplification and, you know, frame wavelength division multiplexing, that was starting to get really thick, you know, a billion dollars wouldn't buy you a few 100 kilobits. A billion dollars was going to buy you a few gigabits per second and pushing upwards on that. And then there was, well, who's going to do the electronics? KDDI, they were good. Alcatel they were good. And the way it kind of worked was each cable proposal sort of got together a provider for the submarine repeaters, a vendor for the actual fiber cable, a factory to go and build the cable, because you had to build it in one length to spool onto a ship. You needed a ship to do the run from A to B, and then you needed a service contract that, God forbid, if it ever broke. And they do break, needed someone to, you know, whiz out, [George: yeah] whizz out to the break point and repair it. And so it was this sort of specialized business where the businesses themselves weren't changing, but who was funding them, [George yeah] and who was using them started to change a lot. And at the same time, the regulations didn't change. As soon as you came up out of the deep ocean, out of international waters into country space. I remember seeing a presentation by the folk who did Hibernia. They claim to land a cable in the continental US, Connecticut or something. They needed 26 different agencies, 26 of them, [George: yeah] the Port Authority, the Marine authority, national security, everything, everyone and their dog was involved in granting this approval, because these cables were still seen as special. [George: Yeah] they weren't just, you know, a piece of string, you know, sang out on the back, you know, hi, you know, catch my fiber for me. Bert, fine. We've got it. Let's go and run it. Nothing like that. It was a much more involved process, and it. Was regulatory heavy. The other part of this was that it was completely obvious that these cables were the super highways of the digital world. If you were a backwater, then your costs were going to get higher. If you were right in the thick of the action, like, you know, Singapore, all the cables came to you, even in the West Coast of the US, there was only really two massive, massive concentration points, southern part of Oregon, near a town called Banton, as I recall, and down near Los Angeles, near an area called Man Ventura County. Why? Well, it was kind of an the sea came up quickly up to land. It was a very convenient sort of point. There wasn't that much fishing around. It was safe, [George: yeah] Similarly, United Kingdom Cornwall was a great place to bring up a cable George Michaelson 30:52 excellent if we explore the backwaters a little bit we've had on ping, a French researcher Nomay, who's been looking at the cable issues surrounding economies like Pakistan. The Gulf waters that come up to Pakistan aren't a deep shelf that then suddenly rise and there you are on the land. It's quite a shallow sea. They have the massive structural problem that it's the shallow sea that have boats anchoring and people fishing in they lose cables because their geography, of their economy is just not a great place to put cables. Or you get things like the Straits of Taiwan, which is passing through an earthquake zone before we even begin to talk about the politics. Geoff Huston 31:37 Well, let's talk about that, not the politics, but the issue of risk, because you see, there are a number of ways that a submarine cable will stop working. One of them is earthquakes, submarine you know landslides. Landslides under water are vicious. You know you look at an avalanche down a ski slope or something. You think, 50 kilometers an hour or so on underwater, a good submarine land landslide, a big one is 300 kilometers an hour. It's huge. And when it strikes a few tiny snakes of fiber optic, they're cactus, you know. And so when you lay fiber across a sort of an active submarine area south of Taiwan, Indonesia, across the so called Wallace Line, across any of these plate tectonic boundaries, there is a risk that the cable will get snapped, and that's just, that's the environment we work in, [George: yeah] But at the same time, there are other risks, and increasingly, in this quite hostile world we live in, we're all mindful of what the British did to the Germans in 1939 where, as soon as war was declared, a quiet little expedition of British Navy ships headed up into the North Sea and dragged their anchors across the sea floor and deliberately cut the German submarine cables, forcing German communications with the rest of the world, moving out From the continent, up into radio quite deliberate, strategic even. But we don't forget, submarine cables are an issue strategically, [George: yeah], because while they have an enormous capacity and it's highly secure, it's difficult to tap them. They can get broken, and sometimes it's not force measure or, you know, the Earth's geology. It's someone on a deliberate course of action, and it's not a nice course of action, and they're deliberately looking for this cable and snapping it. [George: Yeah] there have certainly been accusations about this happening in the Baltic up near Estonia. There have certainly been accusations of it happening on a number of islands that Taiwan own near the Chinese mainland. [George: Yeah] there are accusations around the seas in the sort of southeast area of Asia, and generally where the ocean is shallow, it's there's nothing protecting them other than everyone's forbearance, then they can be disrupted. George Michaelson 33:58 Yeah, as long as we have a mutuality and there's enough benefit for everyone to leave things as they are. It can be an argument in the courts or an argument about money, but the minute, it's a strategic issue between national players, irrespective of the capital interests in this. If it's strategically useful to be disruptive, somebody's going to do it. Geoff Huston 34:19 Oh, so let's talk about these erbium doped fiber repeaters, [George: right] Because if I made them, and my country says Geoff relations with this other country, I'm sorry, they've gone to the point where, you know hostilities are underway. Make that cable not work. We know you can the answer would be, is this an instruction general? The answer is, well, yes, [George: yeah] And so kind of, who built this cable? What equipment are they using? Where is it landing? Equally, part of the kind of overall balance, because these are the superhighways of. Communication, and when things get a bit fraught and a bit tense, then certainly national security interests tend to come up. Now we've seen this expressed in a number of areas and a number of countries, Canada, the United States, Australia. I even think New Zealand has got to that point one way or another. Have said, Look, we're not going to have equipment made by other manufacturers from certain proscribed countries in our national infrastructure network. We're just not now. We know it's a deregulated market. We know you can, in theory, buy anything, but if you want to connect it to this network in our country, it can't have been made from these countries. [George: Yup] when in 2012 the Asian Development Bank said it would actually construct a spur on an existing cable from Australia to Papua New Guinea to link in the Solomon Islands, which would have been great for the Solomon Islands, [George: yes] it was announced that they do something in 2012 by about 2017 and nothing had happened. The Solomon Islands government moved in and said, Look, you know, well, okay, we'll do something here. We will actually make this happen. Oh, look, we've signed an arrangement with this vendor here to use their submarine equipment to attach to this cable. The Australians went, No, can't do that. We're not going to have equipment made by that country on facilities that we're using and reliant on. Tell you what. We'll build a complete system ourselves. We will not give you the permits to land your cable in our country. Do what you want in your country, but this is our country. It's the other end of the cable, and that was the first. Wasn't necessarily the first, but interestingly, this one, with equipment supplied by Alcatel submarine, went into operation in December 2019 and presumably everyone was happy, because the Australians went and paid for the lot anyway. But you know, that was the first of these. Well, you know, it's more than just a commercial deal. There are national interests involved. There was one in the East Micronesia cable system, a cable system that actually goes between Guam and the various islands in Micronesia, the Marshall Islands, Pohnpei, Kosrae. George Michaelson 37:14 You know, we should say at this point that Guam is possibly the most significant cable termination point in our part of the Pacific, because from Guam to Japan and other places in Southeast Asia is really quite a doable deal, and Guam is under US legal and commercial jurisdiction. So it's a boundary point that pushes from the West Coast of America all the way into Asia to say, I'm ready to do business Geoff Huston 37:42 more to the point I think it's conveniently located rock in the middle of the Pacific that has stable power and really long submarine cable runs are a bit of a pain. And quite frankly, if you want to junction off and put some part of the capacity to hear, some to there, some to there, and so on, Guam is kind of the center of a star where you can link to Guam and then link to a whole bunch of other countries with separate cabling systems. Everyone is happy. You know, this is all good stuff. So when they decided to do a Solomon Islands cable, linking all these countries together in 2017 then again, the Australians came in and said, You're basically going to the wrong area of vendor here. We're going to build the Coral Sea cable Island system, and we're going to get out one of our carriers to do all this for you. And again, surprise, surprise, Alcatel submarine off there. So again, same kind of problem. Now the next one's kind of touchier and more interesting, because you see, over in Asia, there are very few ways to get close to Asia. On the east side of the Philippines is the that magnificently deep trench, the deepest trench on the surface of the planet, right? So you've got to come through a number of very small, constrained gateways to the south of Taiwan, to the north of Taiwan, or all the way around through the archipelago of Indonesia to get into Singapore, George Michaelson 39:10 Which might interest different economies differently, because if a cable is passing you by, it's very tempting to suggest that You'd supply energy, termination and connection services, and also get an opportunity to get a lot of data to and from places in the region. So it's not like there are no upsides in this, but it is nonetheless a bit of a geography meets politics moment, isn't it? Geoff Huston 39:34 It is. And so one of these kind of it wasn't quite a golf course cartel moment, but I'm pretty sure it has its origins on a golf course where there was this agreement between a number of folk, Google, Facebook, and this odd group called the DR Peng group, from China, and they were going to build a direct link from Los Angeles to Hong Kong. And then sort of tie in and perhaps go down to Singapore. George Michaelson 40:05 So this is a convergence of business interests in mainland China and the international groups that are interested in having service delivery everywhere. This is a massive market opportunity. So you can see why people like Google and meta wanted to get into this combine. It made sense for everyone. Geoff Huston 40:24 Oh, and everyone, you know, the Chinese are a massive population. There's money to be made. So they're there, you know. So 12,800 kilometers of fiber, basically went from Los Angeles, had a touching point in the Philippines, and then off to Hong Kong. But of course, you've started Los Angeles, and in early 2022 the FCC approved a license for a cable system that certainly touched the earth in Los Angeles and surfaced again in the Philippines and also surfaced in Ching in Taiwan. But that was it. The other four cable pairs are unlit. They were the ones that were going to go to Hong Kong if anyone tried to shine light down them, the FCC approval for the Los Angeles end was going to be withdrawn. George Michaelson 41:08 So they've made the investment in the ground, but there's no light in the fiber. Geoff Huston 41:13 There's no light under the water on that one, for that particular part, another one the Hong Kong American cable. Well, it's exactly the same story, 12.8 terabits per fiber pair. Oh, my God, six fiber pairs. That's 76 terabits, or using 100 gig of coherent wavelength technology, Facebook, China Telecom, China, Unicom, Tata communications, India and indeed, Australia's Telstra, was all part of this. Whoops, the application to land the cable in the US was withdrawn in March 2021, [George: right], dead in the water. Not going to happen. Hong Kong Guam withdrawn in November 2020, same story. What's going on is that we're increasingly seeing a bifurcation, particularly in the Asia Pacific area, where, if you want to terminate one end of your cable, or even a junction spur into US territory, including Guam into US territory, then you're only going to get approval if all of the other termination points are in sympathetically minded countries. Is that's the best way of putting it and that does not include China, Macau or Hong Kong, absolutely, it probably also doesn't include North Korea, but I'm not sure everyone's ever thought about that anyway. George Michaelson 42:30 So again, looking at the potential for there to be two sides to a story. What you or I think about this doesn't really matter, because the governments are going to do what the governments do. But if I were Singapore or Taiwan or Japan or Korea or the Philippines or Malaysia, I'd be looking at this saying, I smell opportunity to become an entrepot, to become a point that is not physically on the path, but perhaps logistically, is a useful place for things to terminate, surface into systems, pass through Internet and then re inject into the framework we need it to be in. I could see this working for some players in the region. Geoff Huston 43:10 Well, the player of record is Singapore, and they are the absolute winner, because the later systems, whether it's apricot, 190 terabits echo, 144 terabits, Bifrost, that's, again, a massive 12 pair system, probably well into the 200 terabits are all having their one end in Singapore and then feeding into other cable systems, you know, as appropriate, and heading off into the US at the other end, perhaps with junction points in Japan and possibly the Philippines, possibly not, but it's more Singapore, Japan, the US and those the West Coast of the US, Los Angeles, Japan and Singapore are the focal points now for submarine infrastructure and then smaller cabling systems feed off into that that aren't subject to the same constraints that the US is applying to cables that actually surface in US territory. So it's not necessarily the shortest paths we do anymore, not necessarily, and in fact, the deeper, the better, as long as you avoid the really geologically actively bits and length is not the major issue. Length is kind of secondary to trying to sort of pick up the major markets without picking up too much geopolitical hot pressure. [George: Yeah]. And of course, you've got to beware of your equipment supplier. You've got to pick someone who, what they call the five eyes. Countries, US, Canada, New Zealand, Australia, UK. Approve of if you actually want again, your necessary approvals. George Michaelson 44:49 And this is not just the head end equipment. This actually is almost any aspect of the supply chain is now going to have to be considered for its risk, isn't it? Geoff Huston 44:58 Well, yes. And. And of course, it'll be unusual to expect the Chinese to go, that's good and fine. Do what you want. Huawei submarine and other entities are still going, you know, going at it and doing a fine job. Linking, other parts of the world that don't necessarily have the same sensitivity as the United States and their FCC, but it is an odd overlay where these systems are now being routed, not because they're the best possible path, not even because they go past the most lucrative markets to provide service to, but are actually willing, or necessarily meeting a bunch of geopolitical constraints and tactically, tactically coming up to the surface and redistributing the packets in places that are more politically accepting, I think, or ambivalent. That was the case in the past. George Michaelson 45:47 It's an interesting political overlay that, again, to come back to John Perry Barlow's observation really doesn't align with the idea of a fully independent space, that it's not bound by laws and economics of nation states, this is very clearly intensely politicized and bound into the realities of the modern world. Geoff Huston 46:08 Yeah, I think we can all weep at an opportunity lost in some ways. I think the hippies did have a nice idea of discarding some of these old fraught tensions that produce so many problems in the 20th century, but we don't seem to be able to discard those problems easily, and so we're stuck in the same kind of world. And yes, these kinds of projects are actually ridden with geopolitical aspects, whether the satellite folk and I see Elon Musk has now launched his 10,000th satellite. Oh, my God, George Michaelson 46:42 he has 60% of the orbiting material is now in Musk's system, 60% Geoff Huston 46:49 I'll bring my head out of my hands and stop weeping for a second and go, God, it's good technology, but it's a big but with the satellite systems, we'll get past this, which I doubt. I think it's the same set of geopolitical concerns everywhere that we in this sort of almost two color or three color world, and we just can't seem to get out of it. And that's certainly shaping the way we build our infrastructure these days, and even shapes the way who and how we talk to at least cost effectively over the coming decades, which is perhaps a less than encouraging sign, the whole old hippie idealism of Mr. Perry Barlow seems to somehow being conveniently set aside, which is a shame. George Michaelson 47:29 It is a shame. Geoff, you've written this one up. Can people find a blog about this online? Geoff Huston 47:34 I did write a blog on cable politics that actually described this situation and the rerouting and the positioning of various cables on my blog, in potaroo.net, and a number of other topics around the same sort of area of interest about what's going on in submarine cables over the years, it is the high technology of fiber cable systems. You know, where do you cut the real the real cutting technology, hollow core fiber, 1000s of kilometers. It's in submarine systems. George Michaelson 48:05 Yeah, cool tech. You know, as a technologist, I'm just sitting there going, so cool, so cool. hat's been fascinating. Geoff, thank you. Geoff Huston 48:13 Well thank you, George and thank you, dear listener. George Michaelson 48:17 If you've got a story or research to share here on ping, why not get in contact by email to ping@apnic.net or via the APNIC social media channels. Also remember the measurement@apnic.net mailing list on orbit is there to discuss and share relevant collaborative opportunities, grants and funding opportunities, jobs and graduate placings, or to seek feedback from the community on your own measurement projects, be sure to check out the APNIC website for All your resource and community needs until next time.