1 00:00:01,760 --> 00:00:03,759 Join us as we gather around the hedge, 2 00:00:03,759 --> 00:00:06,740 where we dig into technology, business, and culture 3 00:00:06,799 --> 00:00:09,139 with the finest minds in computer networking. 4 00:00:21,054 --> 00:00:22,355 Well, hello, Audrey 5 00:00:23,054 --> 00:00:23,875 and Tom. 6 00:00:24,414 --> 00:00:26,335 Audrey says hello. Audrey is the plant, you 7 00:00:26,335 --> 00:00:26,835 guys. 8 00:00:29,079 --> 00:00:31,400 Yvonne just named just named her last time. 9 00:00:31,400 --> 00:00:33,979 So Yes. Now we have to use Audrey. 10 00:00:34,359 --> 00:00:36,440 So, Tom, how are you this morning? I'm 11 00:00:36,440 --> 00:00:39,240 great. Getting, getting ready for the freezing cold 12 00:00:39,240 --> 00:00:41,100 to hit Texas. We're not ready. 13 00:00:41,560 --> 00:00:43,984 Oh, your rocket is upside down by the 14 00:00:43,984 --> 00:00:45,824 way. If you continue to allow it to 15 00:00:45,824 --> 00:00:47,204 go down Oh. 16 00:00:47,585 --> 00:00:49,824 Yes. It doesn't it doesn't it doesn't work 17 00:00:49,824 --> 00:00:52,625 that way, Tom. Okay. Okay. It over. I'm 18 00:00:52,625 --> 00:00:53,684 glad you told me. 19 00:00:54,704 --> 00:00:56,545 I wouldn't I wouldn't want bad things to 20 00:00:56,545 --> 00:00:57,684 happen to your house. 21 00:00:57,984 --> 00:01:00,869 And today, we are joined by Doug Madery 22 00:01:00,929 --> 00:01:01,670 from Kintek 23 00:01:01,969 --> 00:01:03,189 who is in the woods. 24 00:01:04,290 --> 00:01:05,349 Hey. How's it going? 25 00:01:06,609 --> 00:01:08,930 I mean, I just guessing from your background, 26 00:01:08,930 --> 00:01:10,290 you're out in the woods in a tent. 27 00:01:10,290 --> 00:01:12,129 I'm in my orange tent. Yes. You're in 28 00:01:12,129 --> 00:01:13,189 your orange tent. 29 00:01:13,810 --> 00:01:16,094 So there you go. And then we have 30 00:01:16,094 --> 00:01:17,715 Job. Job, where are you physically? 31 00:01:18,814 --> 00:01:21,295 I'm in Amsterdam at the moment. Thank you 32 00:01:21,295 --> 00:01:22,274 for having me. 33 00:01:22,734 --> 00:01:25,295 Cool. I actually love Amsterdam. Oh, and I 34 00:01:25,295 --> 00:01:26,594 should clarify. Amsterdam 35 00:01:27,215 --> 00:01:30,380 is a relatively large city in the beautiful 36 00:01:30,380 --> 00:01:31,520 country of The Netherlands. 37 00:01:33,260 --> 00:01:35,340 I, I have to explain because once I 38 00:01:35,340 --> 00:01:37,340 was in a a jeans store and I 39 00:01:37,340 --> 00:01:39,420 told the this little person I was from 40 00:01:39,420 --> 00:01:42,045 Amsterdam and they responded that they they loved 41 00:01:42,045 --> 00:01:44,405 Germany. And I you know, I never seen 42 00:01:44,405 --> 00:01:46,444 it in that moment. Wow. I feel like, 43 00:01:46,444 --> 00:01:48,444 you know, not everybody knows Hope you just 44 00:01:48,444 --> 00:01:50,704 walked right out. Yeah. Yeah. 45 00:01:51,965 --> 00:01:53,024 Wow. That's wild. 46 00:01:53,420 --> 00:01:55,019 The last time I was in Amsterdam, I 47 00:01:55,019 --> 00:01:56,060 don't know if they still have it, but 48 00:01:56,060 --> 00:01:57,739 they had a Van Gogh exhibit at the 49 00:01:57,739 --> 00:02:00,379 main museum there in town. And that was 50 00:02:00,379 --> 00:02:01,679 really that was awesome. 51 00:02:02,140 --> 00:02:04,539 Going to go look at the Van Gogh's. 52 00:02:04,539 --> 00:02:06,140 Yeah. Yeah. Yeah. We have a Van Gogh 53 00:02:06,140 --> 00:02:08,219 Museum, a permanent one, and it it is 54 00:02:08,219 --> 00:02:10,194 gorgeous. So if you come visit Amsterdam, 55 00:02:10,495 --> 00:02:12,974 come to museums. Yes. Yeah. Yeah. The museums 56 00:02:12,974 --> 00:02:13,634 are great. 57 00:02:14,094 --> 00:02:17,555 So alright. So let's talk about AS sets. 58 00:02:17,694 --> 00:02:19,055 For those of you who don't know what 59 00:02:19,055 --> 00:02:21,215 it is, an AS set is basically when 60 00:02:21,215 --> 00:02:22,995 you aggregate routes in BGP, 61 00:02:24,240 --> 00:02:26,319 you take all of the b g of 62 00:02:26,319 --> 00:02:27,699 the AS path elements 63 00:02:28,159 --> 00:02:30,819 and you combine them into an AS set, 64 00:02:30,879 --> 00:02:33,039 and it becomes a single entry in your 65 00:02:33,039 --> 00:02:33,939 AS path. 66 00:02:35,455 --> 00:02:37,694 This may sound strange. Wait a second. That 67 00:02:37,694 --> 00:02:38,514 is a difference 68 00:02:38,895 --> 00:02:40,814 AS set that I thought we were gonna 69 00:02:40,814 --> 00:02:42,814 talk about. Oh, okay. Well, then you then 70 00:02:42,814 --> 00:02:43,955 you define it. 71 00:02:45,614 --> 00:02:47,215 Alright. That was that was a plan. That 72 00:02:47,215 --> 00:02:49,395 was that was planned. That was planned. Yeah. 73 00:02:49,694 --> 00:02:51,210 We walked right into that, Doug. 74 00:02:53,129 --> 00:02:55,370 So so, Ross, I think you are talking 75 00:02:55,370 --> 00:02:55,870 about 76 00:02:56,810 --> 00:02:57,950 a s underscore 77 00:02:58,330 --> 00:02:59,950 set. Yes. That's 78 00:03:00,490 --> 00:03:01,550 correct. Mentally 79 00:03:02,010 --> 00:03:04,510 prepared to talk about a s 80 00:03:05,055 --> 00:03:07,935 dash set. Well, you talk about a s 81 00:03:07,935 --> 00:03:10,094 dash sets. That's fine. You go right ahead. 82 00:03:10,094 --> 00:03:12,834 Another another instance in the Internet routing terminology 83 00:03:12,894 --> 00:03:15,134 where you've got overloaded terms in a way 84 00:03:15,134 --> 00:03:18,030 that is problematic about Router switch 85 00:03:18,349 --> 00:03:20,030 aggregate Well, I've been talking about this in 86 00:03:20,030 --> 00:03:22,189 the last year. I have a slide right 87 00:03:22,189 --> 00:03:23,389 up front. I was like, this is a 88 00:03:23,389 --> 00:03:25,150 talk about AS sets. If you came for 89 00:03:25,150 --> 00:03:26,430 a talk about AS sets, you're in the 90 00:03:26,430 --> 00:03:28,430 wrong place. This is just AS sets. I'm 91 00:03:28,430 --> 00:03:30,270 not gonna be covering. So the the same 92 00:03:30,270 --> 00:03:32,449 term, you know, whether it's dash or underscore 93 00:03:32,995 --> 00:03:34,295 refers to two different things. 94 00:03:34,675 --> 00:03:36,435 I think the thing that, the the route 95 00:03:36,435 --> 00:03:36,935 consolidation 96 00:03:37,314 --> 00:03:39,014 piece is a is something that's, 97 00:03:39,875 --> 00:03:40,615 been deprecated. 98 00:03:41,235 --> 00:03:43,555 Although, it's still like, everything that's been deprecated, 99 00:03:43,555 --> 00:03:45,795 there's some traces of it out in the 100 00:03:45,795 --> 00:03:48,514 out in the wild. Yes. Correct. Is the 101 00:03:48,514 --> 00:03:50,830 ASF mechanism to try to, 102 00:03:52,250 --> 00:03:54,090 create kind of allow lists that you would 103 00:03:54,090 --> 00:03:56,090 allow what prefixes you would see over a 104 00:03:56,090 --> 00:03:56,989 BGP session, 105 00:03:57,530 --> 00:03:59,229 in the hopes of trying to prevent 106 00:03:59,689 --> 00:04:00,750 Okay. Well, 107 00:04:01,209 --> 00:04:03,610 that let's start there then. So explain to 108 00:04:03,610 --> 00:04:06,344 us where how people so this is an 109 00:04:06,344 --> 00:04:06,844 IRR 110 00:04:07,145 --> 00:04:07,645 application, 111 00:04:08,264 --> 00:04:09,965 right, or an I IRR 112 00:04:10,264 --> 00:04:10,764 application. 113 00:04:11,064 --> 00:04:11,564 Correct? 114 00:04:13,145 --> 00:04:13,645 Right. 115 00:04:13,944 --> 00:04:14,444 IRR. 116 00:04:14,905 --> 00:04:15,405 Incident 117 00:04:15,705 --> 00:04:16,845 routing registry. 118 00:04:17,225 --> 00:04:19,144 I always say them backwards, so don't worry 119 00:04:19,144 --> 00:04:20,250 about me. That's okay. 120 00:04:21,850 --> 00:04:22,350 And 121 00:04:22,970 --> 00:04:24,509 IR is is a mechanism. 122 00:04:25,129 --> 00:04:26,829 If you trace Beckett's history, 123 00:04:27,290 --> 00:04:29,069 you you end up at the 124 00:04:29,449 --> 00:04:31,149 late eighties, early nineties 125 00:04:32,089 --> 00:04:32,589 when, 126 00:04:33,865 --> 00:04:36,425 super clever people had the foresight that in 127 00:04:36,425 --> 00:04:38,664 order for the Internet to be able to 128 00:04:38,664 --> 00:04:39,164 grow, 129 00:04:39,544 --> 00:04:40,604 we would need 130 00:04:41,224 --> 00:04:42,044 some automation 131 00:04:42,584 --> 00:04:43,485 here and there. 132 00:04:44,185 --> 00:04:45,164 And this automation, 133 00:04:45,704 --> 00:04:48,610 these pipelines would need to extend across organizational 134 00:04:48,610 --> 00:04:49,110 boundaries. 135 00:04:50,209 --> 00:04:50,709 So, 136 00:04:52,529 --> 00:04:54,870 people realize, like, hey. I need to configure 137 00:04:55,089 --> 00:04:57,490 my equipment in a certain way, but I 138 00:04:57,490 --> 00:04:59,990 should also inform the world about how I 139 00:05:00,334 --> 00:05:02,735 am about to configure my equipment or how 140 00:05:02,735 --> 00:05:04,274 my equipment is configured. 141 00:05:04,654 --> 00:05:05,555 And then others 142 00:05:05,935 --> 00:05:07,795 based on that information can, 143 00:05:08,574 --> 00:05:09,634 generate configurations 144 00:05:10,095 --> 00:05:12,035 or or make routing decisions. 145 00:05:12,895 --> 00:05:13,395 So, 146 00:05:14,579 --> 00:05:16,439 IR is is one of the 147 00:05:17,540 --> 00:05:20,919 the earliest attempts at Internet skill coordination, 148 00:05:21,779 --> 00:05:24,040 in order to to to grow the network. 149 00:05:25,459 --> 00:05:27,220 So so when we when we talk about 150 00:05:27,629 --> 00:05:29,375 things. When we talk about configuration, 151 00:05:29,995 --> 00:05:32,095 what we're actually talking about is policy. 152 00:05:32,475 --> 00:05:35,035 We're not talking about, like, network statements or 153 00:05:35,035 --> 00:05:38,395 blah blah blah. We're talking about who transits 154 00:05:38,395 --> 00:05:40,814 where, who's allowed to transit, for what reason, 155 00:05:41,279 --> 00:05:43,279 who who's peered with who. That's the kind 156 00:05:43,279 --> 00:05:45,600 of policy we're talking about. Right? We talk 157 00:05:45,600 --> 00:05:48,399 about configuration for IRRs. Like, these are my 158 00:05:48,399 --> 00:05:48,899 upstreams. 159 00:05:49,680 --> 00:05:51,759 If it's not from my upstream, don't trust 160 00:05:51,759 --> 00:05:54,560 it kind of kind of policy. Is that 161 00:05:54,560 --> 00:05:55,300 a correct? 162 00:05:56,865 --> 00:05:58,064 Yeah. Yeah. It's it's, 163 00:05:58,464 --> 00:06:01,204 that's that's one example. And and now we 164 00:06:01,345 --> 00:06:03,604 almost immediately hit upon one of the challenges. 165 00:06:04,625 --> 00:06:07,504 IRR is super flexible. So not only can 166 00:06:07,504 --> 00:06:08,004 you 167 00:06:08,329 --> 00:06:10,810 try and express who your upstreams are, but 168 00:06:10,810 --> 00:06:13,370 maybe also who your downstreams are or your 169 00:06:13,370 --> 00:06:13,870 neighbors, 170 00:06:14,410 --> 00:06:17,370 of a particular type. Or and now things 171 00:06:17,370 --> 00:06:18,430 get kind of funky. 172 00:06:20,009 --> 00:06:22,810 You could make a list of odd numbered 173 00:06:22,810 --> 00:06:23,310 ASNs 174 00:06:23,615 --> 00:06:26,095 just because you feel like that is important 175 00:06:26,095 --> 00:06:27,634 for you to share with the world. 176 00:06:28,175 --> 00:06:28,675 And 177 00:06:29,214 --> 00:06:32,115 Doc may make a list of ASNs that, 178 00:06:32,894 --> 00:06:35,535 you know, he he just likes the shape 179 00:06:35,535 --> 00:06:36,435 of their numbers. 180 00:06:36,894 --> 00:06:38,514 So what an ASF 181 00:06:39,149 --> 00:06:39,649 is 182 00:06:40,189 --> 00:06:42,370 is is up to the creator, 183 00:06:44,269 --> 00:06:44,930 and everybody 184 00:06:45,310 --> 00:06:48,350 sees something different in them. So this is 185 00:06:48,350 --> 00:06:49,250 immediate like, 186 00:06:50,509 --> 00:06:52,430 it would be cool if the application is 187 00:06:52,430 --> 00:06:54,689 you can list your upstreams, but that's not 188 00:06:54,944 --> 00:06:55,444 technically 189 00:06:55,904 --> 00:06:56,404 what 190 00:06:57,185 --> 00:06:59,845 it always means. It it could be upstreams 191 00:06:59,904 --> 00:07:02,384 or downstreams or something else. It could it 192 00:07:02,384 --> 00:07:04,865 could be your summary peers. It could be 193 00:07:05,024 --> 00:07:07,904 Yeah. It could be the peers you pay. 194 00:07:07,904 --> 00:07:09,985 It could be private peering spy. It could 195 00:07:09,985 --> 00:07:12,199 be anything. It's just It could be anything. 196 00:07:12,199 --> 00:07:14,060 And and in computer technology, 197 00:07:14,439 --> 00:07:16,360 if the moment somebody's like, it could be 198 00:07:16,360 --> 00:07:19,500 anything, it's like, it will be everything and 199 00:07:19,959 --> 00:07:21,979 more. Yeah. It's where trouble starts. 200 00:07:22,519 --> 00:07:24,519 Maybe I'll maybe I'll mention, well, I think 201 00:07:24,519 --> 00:07:26,814 we'll circle back to these issues that, Joe 202 00:07:26,814 --> 00:07:29,615 was touching on. So what the way I 203 00:07:29,615 --> 00:07:30,915 got started on this, 204 00:07:31,454 --> 00:07:34,254 analysis was looking at routing leaks, something that 205 00:07:34,254 --> 00:07:37,214 I look at mishaps, routing mishaps, and, and 206 00:07:37,214 --> 00:07:38,975 noticed that I was aware of this issue 207 00:07:38,975 --> 00:07:40,894 of the AS sets, for a a number 208 00:07:40,894 --> 00:07:42,194 of years where there are some 209 00:07:42,879 --> 00:07:44,960 networks out there that that have the AS 210 00:07:44,960 --> 00:07:46,240 set or one of the AS sets that 211 00:07:46,240 --> 00:07:48,240 they or all the AS sets that they 212 00:07:48,240 --> 00:07:49,220 publish to the Internet 213 00:07:49,759 --> 00:07:50,500 are humongous, 214 00:07:50,960 --> 00:07:53,920 so big that they encompass basically the entire 215 00:07:53,920 --> 00:07:54,420 Internet. 216 00:07:55,254 --> 00:07:57,014 And what this is intended to be is 217 00:07:57,014 --> 00:07:59,415 a white list that you're or allow list. 218 00:07:59,415 --> 00:08:00,154 That's a, 219 00:08:01,014 --> 00:08:03,175 not a a technical term. It's a conceptual 220 00:08:03,175 --> 00:08:05,095 term of just what would you what prefixes 221 00:08:05,095 --> 00:08:06,775 should you be allowing across a speed to 222 00:08:06,775 --> 00:08:09,415 be session. But when these things expand to 223 00:08:09,415 --> 00:08:09,915 be, 224 00:08:11,330 --> 00:08:13,270 you know, expanding to be the entire Internet, 225 00:08:13,330 --> 00:08:16,129 well, now you've just it's permanent any any, 226 00:08:16,129 --> 00:08:16,449 and, 227 00:08:17,170 --> 00:08:19,089 and it's not doing anything. It actually and 228 00:08:19,089 --> 00:08:20,689 then it's not doing anything, but it it's 229 00:08:20,689 --> 00:08:22,550 even worse than that because it it's costly. 230 00:08:22,610 --> 00:08:24,314 It's done in a way that's that's very 231 00:08:24,314 --> 00:08:25,375 costly on, 232 00:08:25,995 --> 00:08:27,834 you know, the routers trying to expand these 233 00:08:27,834 --> 00:08:29,935 definitions to be millions of lines long, 234 00:08:30,314 --> 00:08:31,914 and white you know, white listing the entire 235 00:08:31,914 --> 00:08:33,754 Internet. Anyway so I guess I I was 236 00:08:33,754 --> 00:08:35,514 looking at routing leaks. It had happened a 237 00:08:35,514 --> 00:08:38,075 few times where, the the leak was going 238 00:08:38,075 --> 00:08:39,529 through, a network, 239 00:08:40,549 --> 00:08:42,629 that had a an asset that you could 240 00:08:42,709 --> 00:08:44,409 you can look it up and see it's 241 00:08:44,709 --> 00:08:48,169 it's just way too big. It's hard to 242 00:08:48,470 --> 00:08:50,230 with in a routing leak, it's hard to 243 00:08:50,230 --> 00:08:52,754 be certain because there's so many factors that 244 00:08:52,914 --> 00:08:53,414 influence 245 00:08:53,715 --> 00:08:55,235 the propagation of leak routes, 246 00:08:55,634 --> 00:08:56,995 and many of these I can't know. I 247 00:08:56,995 --> 00:08:58,434 can't be inside the router to know the 248 00:08:58,434 --> 00:08:58,934 policy. 249 00:08:59,715 --> 00:09:01,394 But I do know that this is not 250 00:09:01,394 --> 00:09:04,674 helping, at least, when the, the leaker has 251 00:09:04,674 --> 00:09:05,174 a, 252 00:09:06,274 --> 00:09:07,320 an ASF that's 253 00:09:07,879 --> 00:09:10,200 massive. And and so if any of its 254 00:09:10,200 --> 00:09:12,059 upstreams were using that ASF, 255 00:09:12,679 --> 00:09:15,000 they wouldn't be able to have stopped this 256 00:09:15,000 --> 00:09:16,139 adjacency leak. 257 00:09:16,600 --> 00:09:18,379 And so in the in the also just 258 00:09:18,440 --> 00:09:20,360 another clarification. On the in the topic of 259 00:09:20,360 --> 00:09:21,500 leaks, we've got 260 00:09:22,254 --> 00:09:25,315 origination leaks. So an AS, like, an announces 261 00:09:25,454 --> 00:09:27,294 appears as if it was itself in AS 262 00:09:27,294 --> 00:09:28,595 path. So right most AS. 263 00:09:29,695 --> 00:09:31,074 And so we have a mechanism, 264 00:09:31,695 --> 00:09:32,095 that, 265 00:09:32,894 --> 00:09:35,214 is in a is in relatively good place, 266 00:09:35,375 --> 00:09:35,875 ROV, 267 00:09:37,370 --> 00:09:39,850 route origin validation. So that just that just 268 00:09:39,850 --> 00:09:41,149 handles misoriginations. 269 00:09:43,769 --> 00:09:46,110 Mostly for this is really for accidental, 270 00:09:46,970 --> 00:09:49,449 incidents because, you know, one could add an 271 00:09:49,449 --> 00:09:51,129 AS to AS path. We wanna defeat this 272 00:09:51,129 --> 00:09:53,264 thing, but it's really for just trying to 273 00:09:53,745 --> 00:09:54,705 shore up all the, 274 00:09:55,184 --> 00:09:56,165 accidental misoriginations 275 00:09:56,545 --> 00:09:58,384 that that occur on the Internet. I think 276 00:09:58,384 --> 00:10:00,065 it's doing doing a good job. Now the 277 00:10:00,065 --> 00:10:02,465 other issue is these adjacency leaks. You have 278 00:10:02,545 --> 00:10:05,345 there's no new origination. It's just browser crossing 279 00:10:05,345 --> 00:10:07,730 edges that they're not supposed to. That's where 280 00:10:07,730 --> 00:10:08,929 AS sets, would, 281 00:10:11,250 --> 00:10:11,750 potentially, 282 00:10:12,209 --> 00:10:14,129 have a role to play. But like I 283 00:10:14,129 --> 00:10:16,769 said, so I started to see these AS, 284 00:10:17,570 --> 00:10:20,230 AS sets that were involved in, leaks, 285 00:10:21,235 --> 00:10:22,754 and that were massive. And I was like, 286 00:10:22,754 --> 00:10:24,595 just what how big is this problem? So 287 00:10:24,595 --> 00:10:26,194 I started to go down the road and 288 00:10:26,194 --> 00:10:27,634 just like, alright. What are all the ASFs, 289 00:10:27,634 --> 00:10:29,154 and can I expand them, and how big 290 00:10:29,154 --> 00:10:30,514 are they? I started to do that, and 291 00:10:30,514 --> 00:10:32,514 I noticed that bgb.tools, 292 00:10:32,514 --> 00:10:34,509 this is a neat free utility, kinda like 293 00:10:34,509 --> 00:10:35,809 bgb.h.net, 294 00:10:36,509 --> 00:10:38,509 has a a a mechanism that you can 295 00:10:38,509 --> 00:10:41,389 explore all these AS sets. And I and 296 00:10:41,389 --> 00:10:42,670 after clicking around for a little bit, I 297 00:10:42,670 --> 00:10:44,830 realized, like, this has already been done. I 298 00:10:44,830 --> 00:10:46,509 don't need to write this code. So I 299 00:10:46,509 --> 00:10:48,269 reached out to Ben Carre Cox, who's the 300 00:10:48,269 --> 00:10:50,785 founder of that tool of that, company. And, 301 00:10:51,024 --> 00:10:52,304 I was like, I think you've already done 302 00:10:52,304 --> 00:10:55,184 this, work. Would you mind just what what's 303 00:10:55,184 --> 00:10:56,785 the what's the big list or what's the 304 00:10:56,785 --> 00:10:58,865 worst? As we made this list of the 305 00:10:58,865 --> 00:10:59,825 worst s, 306 00:11:00,384 --> 00:11:00,884 assets 307 00:11:02,279 --> 00:11:02,779 and, 308 00:11:03,879 --> 00:11:06,299 and it's absurd. I mean, the the top 309 00:11:06,360 --> 00:11:08,860 20 all have over a 100,000 310 00:11:08,919 --> 00:11:11,399 ASs. So an AS set will expand to 311 00:11:11,399 --> 00:11:11,899 ASs, 312 00:11:12,600 --> 00:11:14,279 ultimately, and then those ASs get turned into 313 00:11:14,279 --> 00:11:15,980 prefixes based on what they're originating 314 00:11:16,475 --> 00:11:18,235 in as far as the IRR knows. 315 00:11:18,875 --> 00:11:19,274 And the, 316 00:11:21,035 --> 00:11:22,554 and so there are over a 100,000. There's 317 00:11:22,554 --> 00:11:24,095 only, like, 80,000 ish, 318 00:11:24,875 --> 00:11:26,715 ASs in the global routing table. So this 319 00:11:26,715 --> 00:11:28,575 is already, like, problematic. 320 00:11:29,434 --> 00:11:31,514 It's bigger it's bigger than everything that's in, 321 00:11:31,754 --> 00:11:32,309 in the routing 322 00:11:33,110 --> 00:11:35,429 table. And so, yeah, I started to compile 323 00:11:35,429 --> 00:11:36,169 that and 324 00:11:36,870 --> 00:11:38,710 So so let's so let's pick up a 325 00:11:38,710 --> 00:11:39,690 second and 326 00:11:40,070 --> 00:11:41,990 how would people implement this? If you were 327 00:11:41,990 --> 00:11:42,730 an operator 328 00:11:43,110 --> 00:11:45,190 and you wanted to use the IRR as 329 00:11:45,190 --> 00:11:46,570 a asset capability, 330 00:11:47,745 --> 00:11:50,304 Is there something built into routers? I mean, 331 00:11:50,304 --> 00:11:52,625 there is, but okay. So can we explain 332 00:11:52,625 --> 00:11:54,465 that a little bit about how you would 333 00:11:54,465 --> 00:11:56,945 implement this so that operators can kind of 334 00:11:57,024 --> 00:11:59,105 in their head, people who've never done this 335 00:11:59,105 --> 00:12:01,045 before can kind of understand 336 00:12:01,679 --> 00:12:03,059 how this would be used. 337 00:12:04,320 --> 00:12:06,159 Yeah. Let's let's talk a little bit about, 338 00:12:06,159 --> 00:12:09,539 like, the the happy flow, how AS sets 339 00:12:10,000 --> 00:12:12,240 normally are intended to be used and why 340 00:12:12,240 --> 00:12:14,559 why they existed. The happy flow. I like 341 00:12:14,559 --> 00:12:17,235 that. The happy flow. Integrating. So, 342 00:12:17,615 --> 00:12:20,274 Ross, you and me have a BGP session. 343 00:12:20,414 --> 00:12:23,214 I am your supplier, your vendor, your transit 344 00:12:23,214 --> 00:12:25,855 provider, and you are my customer. The goal 345 00:12:25,855 --> 00:12:28,529 is that you announce your routes to me, 346 00:12:28,610 --> 00:12:31,350 and I propagate them to all my neighbors. 347 00:12:32,610 --> 00:12:33,110 Now 348 00:12:34,290 --> 00:12:35,029 every time 349 00:12:35,570 --> 00:12:38,210 you sign up a new customer who has 350 00:12:38,210 --> 00:12:39,269 their own prefix, 351 00:12:39,730 --> 00:12:41,745 if you have to send me an email 352 00:12:41,745 --> 00:12:44,384 like, hey, Joe. There's a new downstream behind 353 00:12:44,384 --> 00:12:44,884 me. 354 00:12:46,304 --> 00:12:49,105 After some time, we both might find it 355 00:12:49,105 --> 00:12:52,725 tedious and even worse, error prone to error. 356 00:12:53,889 --> 00:12:56,610 For instance, you you forgot to email me 357 00:12:56,610 --> 00:12:58,769 or I I missed your email, and then 358 00:12:58,769 --> 00:13:01,330 then I didn't update the configuration on my 359 00:13:01,330 --> 00:13:03,970 side to add your your extra customer to 360 00:13:03,970 --> 00:13:04,950 the allow list. 361 00:13:05,649 --> 00:13:07,750 So instead of that manual workflow, 362 00:13:09,054 --> 00:13:10,034 the idea of 363 00:13:10,495 --> 00:13:13,714 IRR and AS sets is that at provisioning 364 00:13:13,855 --> 00:13:17,054 time, you tell me, Joe, this is my 365 00:13:17,054 --> 00:13:17,875 AS set. 366 00:13:18,575 --> 00:13:19,075 Periodically 367 00:13:19,455 --> 00:13:22,590 expand this AS set. Expand it by using 368 00:13:22,590 --> 00:13:25,149 tooling like BGP q four. Mhmm. 369 00:13:26,190 --> 00:13:28,429 And then whatever comes out of that extension 370 00:13:28,429 --> 00:13:30,370 is something you can load into the router. 371 00:13:30,830 --> 00:13:32,129 And the goal of loading 372 00:13:32,585 --> 00:13:35,225 some configuration into the router is to limit 373 00:13:35,225 --> 00:13:36,445 the blast radius 374 00:13:37,065 --> 00:13:37,565 of, 375 00:13:38,904 --> 00:13:40,524 route leaks if they occur. 376 00:13:41,144 --> 00:13:43,245 So the idea is that you know 377 00:13:43,945 --> 00:13:46,290 who your customers are, what you intend to 378 00:13:46,290 --> 00:13:48,470 announce in the normal course of operations. 379 00:13:49,410 --> 00:13:52,149 And if if there's some kind of mistake 380 00:13:52,290 --> 00:13:54,929 somewhere in the path, maybe a customer of 381 00:13:54,929 --> 00:13:56,870 yours or or maybe your own organization, 382 00:13:57,730 --> 00:13:59,110 has a typo, 383 00:13:59,649 --> 00:14:00,149 accident. 384 00:14:01,214 --> 00:14:03,934 If you inadvertently announce more routes than you 385 00:14:03,934 --> 00:14:07,054 should announce to me, then the filter derived 386 00:14:07,054 --> 00:14:08,355 from the AS set, 387 00:14:09,214 --> 00:14:12,595 should should hamper the propagation or maybe even, 388 00:14:13,375 --> 00:14:13,875 stop 389 00:14:15,350 --> 00:14:17,049 the leap right in its tracks. 390 00:14:17,909 --> 00:14:18,409 Now 391 00:14:19,029 --> 00:14:20,870 the idea of an AS set is that 392 00:14:20,870 --> 00:14:21,370 it's, 393 00:14:21,750 --> 00:14:24,970 it has a recursive property. So you create 394 00:14:25,669 --> 00:14:26,329 AS set 395 00:14:26,870 --> 00:14:27,370 ROS, 396 00:14:28,174 --> 00:14:30,514 and you reference in your ASR, 397 00:14:30,975 --> 00:14:32,674 maybe Doug's ASR. 398 00:14:33,534 --> 00:14:35,634 And then Doug, in turn, can, 399 00:14:36,414 --> 00:14:39,294 reference his customers. And the idea here here 400 00:14:39,294 --> 00:14:40,514 is that that you 401 00:14:42,049 --> 00:14:44,929 don't have manual work all up the chain 402 00:14:44,929 --> 00:14:45,429 where 403 00:14:46,370 --> 00:14:49,570 the the entire supply chain is telling each 404 00:14:49,570 --> 00:14:51,590 other, hey. I I got a new customer 405 00:14:51,649 --> 00:14:53,590 or customer was removed, 406 00:14:54,210 --> 00:14:56,389 but instead automate some of this. 407 00:14:57,154 --> 00:14:59,235 But this is also our challenge to start 408 00:14:59,235 --> 00:14:59,735 because 409 00:15:00,834 --> 00:15:03,394 once we automate the creation of, 410 00:15:03,875 --> 00:15:05,414 routing security configuration 411 00:15:05,954 --> 00:15:06,934 on my side 412 00:15:07,394 --> 00:15:10,514 and I blindly ingest whatever you put in 413 00:15:10,514 --> 00:15:11,654 your AS set, 414 00:15:13,259 --> 00:15:15,919 yeah, what's to stop you from misconfiguring 415 00:15:16,299 --> 00:15:17,759 your AS set object. 416 00:15:18,620 --> 00:15:19,840 And misconfigurations 417 00:15:21,019 --> 00:15:21,519 are 418 00:15:22,299 --> 00:15:24,000 way too easy in this context. 419 00:15:25,065 --> 00:15:26,825 This is because there there are, 420 00:15:27,465 --> 00:15:28,205 I'd argue, 421 00:15:28,985 --> 00:15:32,125 information architecture issues with with the IRR. 422 00:15:33,465 --> 00:15:34,925 What I mean with that is 423 00:15:35,625 --> 00:15:36,125 AS 424 00:15:37,225 --> 00:15:38,045 said ROS. 425 00:15:39,110 --> 00:15:40,490 If there are multiple databases, 426 00:15:40,870 --> 00:15:42,250 which of these databases 427 00:15:42,629 --> 00:15:45,529 contains the authoritative copy, the one that 428 00:15:45,830 --> 00:15:48,330 you, Ross White, famous from this podcast, 429 00:15:48,789 --> 00:15:49,289 created? 430 00:15:49,750 --> 00:15:51,990 Because there's multiple people on the planet named 431 00:15:51,990 --> 00:15:54,845 ROS. Right? So maybe I think I'm pulling 432 00:15:54,845 --> 00:15:57,085 an AS set Ross that it's by by 433 00:15:57,085 --> 00:15:59,825 an entirely different Ross. It's Ross Housley. 434 00:16:00,365 --> 00:16:03,725 That's fine. Right. Right. Yeah. So so pulling 435 00:16:03,725 --> 00:16:06,865 in the wrong AS set obviously leads to 436 00:16:06,924 --> 00:16:08,465 generating the wrong configuration. 437 00:16:09,740 --> 00:16:12,639 So then the the purpose of the exercise 438 00:16:12,700 --> 00:16:13,679 is somewhat defeated. 439 00:16:15,580 --> 00:16:17,259 And I could be pulling in the wrong 440 00:16:17,259 --> 00:16:19,899 AS set, but you might be referencing the 441 00:16:19,899 --> 00:16:23,279 wrong AS. Maybe you intended to reference 442 00:16:24,225 --> 00:16:24,964 doc Midori's 443 00:16:25,584 --> 00:16:27,924 AS set, but instead, you get doc 444 00:16:29,024 --> 00:16:32,164 Homer. Last name. Yeah. Homer Yeah. AS set. 445 00:16:32,865 --> 00:16:35,664 So do AS sets with the same name 446 00:16:35,664 --> 00:16:38,384 can exist in different databases and have different 447 00:16:38,384 --> 00:16:38,884 content, 448 00:16:39,860 --> 00:16:42,679 because it's it's not a globally unique namespace. 449 00:16:43,379 --> 00:16:46,919 So that is one major major quality issue, 450 00:16:47,620 --> 00:16:49,799 in trying to use AS sets. And 451 00:16:50,659 --> 00:16:52,964 you might ask, like, why why are we 452 00:16:52,964 --> 00:16:54,825 even using AS sets? Because, 453 00:16:56,085 --> 00:16:57,384 well, at the time, 454 00:16:57,764 --> 00:17:00,184 we didn't really have anything better. 455 00:17:00,644 --> 00:17:02,804 And you you, you know, you work with 456 00:17:02,804 --> 00:17:04,724 the tools you have. And if all you 457 00:17:04,724 --> 00:17:07,349 have is a a blunt hammer, then, you 458 00:17:07,349 --> 00:17:09,509 know, you you start hitting everything. And there 459 00:17:09,509 --> 00:17:11,750 were and there were tools built around this 460 00:17:11,750 --> 00:17:13,769 to expand these AS sets 461 00:17:14,549 --> 00:17:15,049 into 462 00:17:17,589 --> 00:17:20,410 act prefix list, not prefix, AS list 463 00:17:20,954 --> 00:17:23,775 for AS prefix list for Cisco and Juniper 464 00:17:23,835 --> 00:17:26,414 and Huawei and blah blah blah blah blah. 465 00:17:26,714 --> 00:17:28,315 And in fact, I think there are some 466 00:17:28,315 --> 00:17:30,794 tools built in to operate in network operating 467 00:17:30,794 --> 00:17:32,414 systems and the BGP implementations 468 00:17:33,130 --> 00:17:35,869 that will actually read data in particular formats 469 00:17:35,929 --> 00:17:36,829 to do this. 470 00:17:37,289 --> 00:17:39,470 And so it kinda became an ecosystem. 471 00:17:39,929 --> 00:17:42,009 It's not just like it's a single thing 472 00:17:42,009 --> 00:17:43,470 that somebody hacked out. 473 00:17:43,769 --> 00:17:45,869 It became an entire ecosystem 474 00:17:46,634 --> 00:17:48,575 that we began to trust 475 00:17:49,035 --> 00:17:50,654 to solve certain problems, 476 00:17:51,914 --> 00:17:53,134 in the global Internet 477 00:17:53,755 --> 00:17:55,295 as a as a way of automating. 478 00:17:55,914 --> 00:17:57,994 And so yeah. And so part of the 479 00:17:57,994 --> 00:18:00,279 problem is gonna be, of course, just that 480 00:18:00,279 --> 00:18:02,140 IRRs themselves are voluntary. 481 00:18:02,840 --> 00:18:04,779 It's whatever stuff you stuff in 482 00:18:05,480 --> 00:18:06,940 comes out the other side. 483 00:18:07,320 --> 00:18:09,880 If you stuff the wrong stuff in, guess 484 00:18:09,880 --> 00:18:11,880 what? The wrong stuff comes out the other 485 00:18:11,880 --> 00:18:12,380 side. 486 00:18:12,845 --> 00:18:14,605 And then, like you said, it's not globally 487 00:18:14,605 --> 00:18:16,464 unique. There's no global uniqueness 488 00:18:16,765 --> 00:18:17,265 to 489 00:18:18,125 --> 00:18:21,105 to the actual data that's being carried there. 490 00:18:21,644 --> 00:18:23,884 It's, One one one thing I might add 491 00:18:23,884 --> 00:18:26,445 on this is that, just on the topic 492 00:18:26,445 --> 00:18:26,845 of, 493 00:18:27,660 --> 00:18:30,059 every all the points Joe is raising are 494 00:18:30,059 --> 00:18:30,619 are great, 495 00:18:31,180 --> 00:18:31,680 just 496 00:18:32,059 --> 00:18:33,580 on on the idea of these things just 497 00:18:33,580 --> 00:18:34,960 expanding without end. 498 00:18:36,940 --> 00:18:38,619 As we as we kinda discussed, there's there's 499 00:18:38,619 --> 00:18:40,859 there's always been a lack of semantics around 500 00:18:40,859 --> 00:18:43,454 this of, like, you would use this type 501 00:18:43,454 --> 00:18:45,615 of ASR. Some some ASR sets, you can 502 00:18:45,615 --> 00:18:46,974 just read them by their name. This is 503 00:18:46,974 --> 00:18:48,335 a ASR set of all the ASR's at 504 00:18:48,335 --> 00:18:50,174 some IXP. Like, okay. Well, that might have 505 00:18:50,174 --> 00:18:52,815 some particular use. There's another ASR. These are 506 00:18:52,815 --> 00:18:54,335 all the peers of some other, you know, 507 00:18:54,335 --> 00:18:56,255 ASR. Okay. That's that's that's could be useful 508 00:18:56,255 --> 00:18:58,210 in some scenario. Here's the all the customer, 509 00:18:58,210 --> 00:18:59,970 the customer code of some AS. Okay. That's 510 00:18:59,970 --> 00:19:00,470 useful. 511 00:19:01,089 --> 00:19:03,089 What you find when you start digging into 512 00:19:03,089 --> 00:19:03,589 this 513 00:19:04,450 --> 00:19:05,009 is that, 514 00:19:05,569 --> 00:19:07,809 due to the recursion, you have an AS 515 00:19:07,809 --> 00:19:09,029 set that has, 516 00:19:09,490 --> 00:19:12,125 all the customers within within that 517 00:19:12,424 --> 00:19:14,205 is all the ASs of an IXP. 518 00:19:14,664 --> 00:19:16,744 And then and then again, all the I 519 00:19:16,744 --> 00:19:19,144 I the, ASs of a of a of 520 00:19:19,144 --> 00:19:20,105 a of a peer of one of the 521 00:19:20,105 --> 00:19:22,765 members of the ISP at the IXP. So, 522 00:19:23,384 --> 00:19:26,025 what ends up happening is, this system that 523 00:19:26,025 --> 00:19:26,640 was built 524 00:19:28,079 --> 00:19:31,140 to, police and stop routing leaks is itself 525 00:19:32,000 --> 00:19:32,500 experiencing 526 00:19:32,799 --> 00:19:33,299 leaks, 527 00:19:34,400 --> 00:19:36,320 that are very akin to like, if you 528 00:19:36,320 --> 00:19:38,180 were at a an AS at an exchange 529 00:19:38,400 --> 00:19:39,519 and you took all the routes that you 530 00:19:39,519 --> 00:19:40,305 learned at that exchange 531 00:19:41,184 --> 00:19:43,184 sent them through your transit provider, well, that 532 00:19:43,184 --> 00:19:45,025 would probably be a leak and you'd be 533 00:19:45,025 --> 00:19:47,525 inadvertently providing transit for everybody at the exchange. 534 00:19:48,785 --> 00:19:50,465 So that so we have the exact same 535 00:19:50,465 --> 00:19:52,725 thing as happening within the AS set world 536 00:19:52,865 --> 00:19:54,144 where these things get, 537 00:19:54,705 --> 00:19:58,059 you know, rolled into other, types and the 538 00:19:58,059 --> 00:20:00,319 semantics are not well established or understood. 539 00:20:00,619 --> 00:20:01,119 It's, 540 00:20:01,500 --> 00:20:03,259 I know Joe's got opinions on this of, 541 00:20:03,259 --> 00:20:05,680 like, it's almost a a state where, like, 542 00:20:05,900 --> 00:20:07,660 it's not clear. You could you could say, 543 00:20:07,660 --> 00:20:09,174 like, oh, you need the ASPA or some 544 00:20:09,174 --> 00:20:11,174 kind of another mechanism just for the AS 545 00:20:11,174 --> 00:20:12,855 sets. You know, that was that's never gonna 546 00:20:12,855 --> 00:20:15,095 happen. That's I'm not suggesting that. But, 547 00:20:18,535 --> 00:20:21,095 it it may be unfixable, and we have 548 00:20:21,095 --> 00:20:22,875 to have different mechanisms to, 549 00:20:23,734 --> 00:20:24,234 to 550 00:20:25,349 --> 00:20:27,589 police these kind of adjacency leaks. So is 551 00:20:27,589 --> 00:20:29,269 there any is there any kind of I'm 552 00:20:29,269 --> 00:20:30,710 assuming the answer is no. But is there 553 00:20:30,710 --> 00:20:32,890 any kind of semantic for do not recurse 554 00:20:32,950 --> 00:20:35,029 or, like, I don't wanna be involved in 555 00:20:35,029 --> 00:20:37,109 this ever expanding web of everything? Is there 556 00:20:37,109 --> 00:20:39,049 any way to express that? 557 00:20:39,454 --> 00:20:41,375 No. And actually that I I I when 558 00:20:41,375 --> 00:20:43,055 I last year, I was giving, you know, 559 00:20:43,055 --> 00:20:44,494 a talk along these lines a few times. 560 00:20:44,494 --> 00:20:46,575 I I'd get operators at the end, share 561 00:20:46,575 --> 00:20:48,815 their, you know, experiences with this. I had 562 00:20:48,815 --> 00:20:50,494 one that was a a guy who handles 563 00:20:50,494 --> 00:20:52,355 this for a major European carrier. 564 00:20:52,654 --> 00:20:54,095 And one of the things he he tried 565 00:20:54,095 --> 00:20:56,230 to do was go into every AS set 566 00:20:56,230 --> 00:20:58,470 where his AS appears and shouldn't and see 567 00:20:58,470 --> 00:20:59,589 if he can reach the person there. And 568 00:20:59,589 --> 00:21:01,349 just, like, hey. Would you mind removing it 569 00:21:01,349 --> 00:21:02,970 and maybe tell me why? 570 00:21:03,750 --> 00:21:05,529 Most times, he couldn't reach anybody. 571 00:21:05,910 --> 00:21:07,529 The other and the other popular, 572 00:21:07,910 --> 00:21:08,295 outcome 573 00:21:09,494 --> 00:21:11,494 was that he would get a response back 574 00:21:11,494 --> 00:21:13,894 saying, oh, that, the guy who made that 575 00:21:13,894 --> 00:21:15,054 left, like, ten years ago. 576 00:21:15,494 --> 00:21:15,994 And, 577 00:21:16,775 --> 00:21:18,855 and and you're like, well, can you remove 578 00:21:18,855 --> 00:21:20,855 it? This actually has operational impact on the 579 00:21:20,855 --> 00:21:22,500 Internet. And you're like like, we don't even 580 00:21:22,500 --> 00:21:24,340 know how how we would go about doing 581 00:21:24,340 --> 00:21:26,740 that. So, like, this is We don't know 582 00:21:26,740 --> 00:21:28,039 where the password is. 583 00:21:28,740 --> 00:21:29,240 Exactly. 584 00:21:29,700 --> 00:21:30,200 So, 585 00:21:31,859 --> 00:21:34,099 yeah. That I mean, I don't know. We've 586 00:21:34,099 --> 00:21:38,235 already named enumerated maybe a dozen issues here. 587 00:21:38,235 --> 00:21:38,555 But, 588 00:21:39,595 --> 00:21:40,815 so now another issue. 589 00:21:41,835 --> 00:21:44,315 The feedback loop in all of this is 590 00:21:44,315 --> 00:21:44,815 horrendous. 591 00:21:45,755 --> 00:21:48,335 So let's let's take a VGP routing. 592 00:21:48,839 --> 00:21:51,559 If that happens, then the circuits that is 593 00:21:51,559 --> 00:21:53,960 affected by the route leak will probably start 594 00:21:53,960 --> 00:21:55,740 running hot because your leak 595 00:21:56,200 --> 00:21:59,000 is pulling in traffic that that shouldn't be 596 00:21:59,000 --> 00:22:00,220 on that path. So, 597 00:22:01,000 --> 00:22:03,785 the signals you have are, hey. There's congestion, 598 00:22:04,325 --> 00:22:07,445 on the path. There the the monitoring shows 599 00:22:07,445 --> 00:22:10,345 that interface is, 100% utilized. 600 00:22:11,684 --> 00:22:14,085 Maybe there's a counter that shows that you're 601 00:22:14,085 --> 00:22:16,725 receiving way more routes than you usually receive 602 00:22:16,725 --> 00:22:19,019 from the peer. So these are all, I 603 00:22:19,099 --> 00:22:19,839 would say, 604 00:22:20,380 --> 00:22:21,359 fairly direct 605 00:22:21,660 --> 00:22:23,759 indicators that something is wrong. 606 00:22:24,380 --> 00:22:26,160 But in the IR ecosystem, 607 00:22:26,940 --> 00:22:29,440 if you add a reference to the wrong 608 00:22:29,980 --> 00:22:31,839 AS set or the or there's, 609 00:22:32,299 --> 00:22:34,080 duplicate names for different 610 00:22:34,654 --> 00:22:35,875 AS set objects, 611 00:22:36,414 --> 00:22:38,355 there's not really a warning system 612 00:22:39,054 --> 00:22:40,115 that materially 613 00:22:40,575 --> 00:22:43,714 affects your operation like a congested circuit would. 614 00:22:44,095 --> 00:22:46,680 So for most people, they will just 615 00:22:47,240 --> 00:22:50,140 add to the AS set until whatever 616 00:22:51,000 --> 00:22:53,500 task they had in mind is done. 617 00:22:54,119 --> 00:22:55,660 But removing information 618 00:22:56,440 --> 00:22:59,240 is a source of great uncertainty because who 619 00:22:59,240 --> 00:22:59,740 knows 620 00:23:00,365 --> 00:23:02,924 what the purpose of that information was and 621 00:23:02,924 --> 00:23:04,445 why But, Joe, it's even it's even worse. 622 00:23:04,445 --> 00:23:05,725 It's even worse than that. Let's say you're 623 00:23:05,725 --> 00:23:07,484 making your ASF and it is airtight. You 624 00:23:07,565 --> 00:23:09,404 this is the best ASF in the world 625 00:23:09,404 --> 00:23:10,144 on the Internet. 626 00:23:10,924 --> 00:23:11,664 And tomorrow, 627 00:23:12,045 --> 00:23:14,125 one of the members of that ASF adds 628 00:23:14,125 --> 00:23:16,259 something ridiculous and you just Yeah. Your thing 629 00:23:16,259 --> 00:23:17,460 just went to crap, and you have no 630 00:23:17,460 --> 00:23:19,539 idea because you what you're looking at is 631 00:23:19,539 --> 00:23:21,140 the same as what it was yesterday when 632 00:23:21,140 --> 00:23:22,900 it was great. And and and that only 633 00:23:22,900 --> 00:23:26,259 is seven layers away or something. Yeah. But 634 00:23:26,259 --> 00:23:28,359 the recursion can also be circular, 635 00:23:29,115 --> 00:23:30,974 and the recursion algorithms 636 00:23:31,434 --> 00:23:32,894 are not really standardized, 637 00:23:33,994 --> 00:23:35,615 or deterministic because 638 00:23:36,394 --> 00:23:39,755 one operator might use IRR databases a, b, 639 00:23:39,755 --> 00:23:42,714 and c, and another operator might use IRR 640 00:23:42,714 --> 00:23:43,214 databases 641 00:23:43,869 --> 00:23:45,710 b, c, and d so that, you know, 642 00:23:45,710 --> 00:23:48,190 there's if you imagine a Venn diagram that 643 00:23:48,190 --> 00:23:49,649 there's partial overlap. 644 00:23:50,109 --> 00:23:53,089 And if there's duplicity of name naming collisions, 645 00:23:53,789 --> 00:23:57,009 different contents with different databases, and the databases 646 00:23:57,390 --> 00:23:57,890 are 647 00:24:00,044 --> 00:24:02,524 looped through in a different order, you know, 648 00:24:02,524 --> 00:24:05,565 one an American operator might prefer to first 649 00:24:05,565 --> 00:24:06,065 lookups 650 00:24:06,444 --> 00:24:07,585 in Aaron's database, 651 00:24:08,125 --> 00:24:11,004 whereas European oriented operators might, 652 00:24:11,484 --> 00:24:14,784 configure their system to first do lookups in 653 00:24:15,490 --> 00:24:16,230 Ripe's database, 654 00:24:17,089 --> 00:24:18,630 you get very different results. 655 00:24:19,009 --> 00:24:21,250 So debugging this like, hey, Doug. Are you 656 00:24:21,250 --> 00:24:22,309 seeing this issue? 657 00:24:23,089 --> 00:24:25,490 It's really challenging because we might be looking 658 00:24:25,490 --> 00:24:26,390 at entirely 659 00:24:26,769 --> 00:24:27,269 different 660 00:24:27,795 --> 00:24:28,615 data structures, 661 00:24:29,315 --> 00:24:30,134 and confusingly, 662 00:24:31,555 --> 00:24:34,775 there might be duplicity of names. So So 663 00:24:34,914 --> 00:24:37,174 is there is there anybody who's just 664 00:24:37,555 --> 00:24:39,154 opted out? Are there any operators that are 665 00:24:39,154 --> 00:24:40,275 just like, it's not worth the mess? We're 666 00:24:40,275 --> 00:24:42,240 just not even gonna use this information anymore. 667 00:24:42,240 --> 00:24:44,000 I got so what what before I was 668 00:24:44,000 --> 00:24:46,400 getting to I was starting to do a 669 00:24:46,400 --> 00:24:48,019 little research to to, 670 00:24:48,960 --> 00:24:51,600 write this, analysis last year, I talked to 671 00:24:51,600 --> 00:24:52,320 a bunch of, 672 00:24:52,720 --> 00:24:54,434 kinda like what what we consider tier one 673 00:24:54,434 --> 00:24:55,414 operators, cloud, 674 00:24:55,795 --> 00:24:57,634 operators. Hey. What how are you guys is 675 00:24:57,634 --> 00:24:58,755 this a problem for you? I guess it 676 00:24:58,755 --> 00:25:00,755 was my first question. They're like, oh, yeah. 677 00:25:00,755 --> 00:25:02,755 Like, we have to we basically had to 678 00:25:02,755 --> 00:25:03,894 make another system, 679 00:25:04,595 --> 00:25:05,075 that, 680 00:25:05,554 --> 00:25:07,336 that runs over this. So people have had 681 00:25:07,336 --> 00:25:07,890 to independently 682 00:25:08,429 --> 00:25:08,929 solve 683 00:25:09,309 --> 00:25:11,869 around this. It can't be used as as 684 00:25:11,869 --> 00:25:12,369 is, 685 00:25:12,910 --> 00:25:13,650 and so, 686 00:25:14,670 --> 00:25:17,569 Google's got one solution. AWS got somebody else, 687 00:25:17,950 --> 00:25:19,644 and everybody's kind of written a bunch of 688 00:25:19,644 --> 00:25:22,444 software to try to take the sorry state 689 00:25:22,444 --> 00:25:25,025 and then try to use some other heuristics 690 00:25:25,085 --> 00:25:25,984 to pair it down. 691 00:25:27,005 --> 00:25:28,684 Some things that are that are difficult are 692 00:25:28,765 --> 00:25:30,605 there's like like, everything in the Internet, there's 693 00:25:30,605 --> 00:25:32,630 all these corner cases. So you can have, 694 00:25:32,630 --> 00:25:33,929 like, let's say, 695 00:25:34,470 --> 00:25:36,390 if your approach bay is based on, hey. 696 00:25:36,390 --> 00:25:37,609 What was routed yesterday? 697 00:25:38,390 --> 00:25:40,470 Let's just assume that's probably what we're gonna 698 00:25:40,470 --> 00:25:41,130 see today, 699 00:25:41,990 --> 00:25:44,069 or maybe an hour ago. Let's assume it's, 700 00:25:44,069 --> 00:25:45,164 you know, let's say it's even 701 00:25:45,725 --> 00:25:49,085 timely. You've got, like, a few different weird 702 00:25:49,085 --> 00:25:49,725 things like, 703 00:25:50,285 --> 00:25:52,205 b g b b g b based DDoS 704 00:25:52,205 --> 00:25:54,945 mitigation that could conceivably be announcing, 705 00:25:55,884 --> 00:25:57,484 a lot of a lot of different things 706 00:25:57,484 --> 00:25:58,205 on the Internet, 707 00:25:58,605 --> 00:26:00,630 that won't wouldn't be there an hour ago. 708 00:26:00,630 --> 00:26:02,630 But when it when it kicks on, it 709 00:26:02,630 --> 00:26:04,710 it really needs to have those routes propagated 710 00:26:04,710 --> 00:26:05,210 quickly. 711 00:26:05,509 --> 00:26:06,009 So, 712 00:26:07,589 --> 00:26:09,829 it's it's hard to make, I guess that's 713 00:26:09,829 --> 00:26:11,750 true for any security mechanism to try to 714 00:26:11,750 --> 00:26:13,450 deal with that scenario. But, 715 00:26:15,134 --> 00:26:16,974 yeah. Every I think everyone that I I've 716 00:26:16,974 --> 00:26:19,795 I've spoken to has had to write software 717 00:26:20,414 --> 00:26:21,955 to So nobody nobody's 718 00:26:22,414 --> 00:26:25,775 nobody's abandoned it per se. They've just tried 719 00:26:25,775 --> 00:26:27,154 to fix it internally. 720 00:26:27,615 --> 00:26:29,349 And and by the way, the one of 721 00:26:29,349 --> 00:26:31,190 the problems you have here as an operator 722 00:26:31,190 --> 00:26:34,069 is the bigger your footprint is in terms 723 00:26:34,069 --> 00:26:36,710 of peering, the better your own internal solutions 724 00:26:36,710 --> 00:26:37,609 for this are. 725 00:26:38,309 --> 00:26:40,809 So smaller operators like Google's 726 00:26:41,109 --> 00:26:41,609 are, 727 00:26:42,950 --> 00:26:44,730 give off a false sense of security, 728 00:26:45,085 --> 00:26:47,244 unfortunately. Yeah. Well, that's gonna be true too. 729 00:26:47,244 --> 00:26:47,744 Yeah. 730 00:26:48,125 --> 00:26:50,465 So you you have this system. It 731 00:26:50,845 --> 00:26:51,904 takes this input, 732 00:26:52,205 --> 00:26:53,025 as parameter, 733 00:26:53,485 --> 00:26:55,805 the name of an ASR. It expands it 734 00:26:55,805 --> 00:26:57,184 through some algorithm 735 00:26:57,670 --> 00:26:59,049 into a list of prefixes. 736 00:26:59,670 --> 00:27:02,150 Let's say a list of, 50,000 737 00:27:02,150 --> 00:27:04,410 prefixes. Which by the way is not guaranteed 738 00:27:04,630 --> 00:27:06,890 either. The entire idea that 739 00:27:07,190 --> 00:27:10,009 anyway, continue. But that that's another entire problem 740 00:27:10,234 --> 00:27:13,355 It's like this correlation between prefixes and AS 741 00:27:13,355 --> 00:27:15,775 numbers is not as Right. 742 00:27:16,154 --> 00:27:17,615 So so through some 743 00:27:17,914 --> 00:27:18,734 non deterministic 744 00:27:19,115 --> 00:27:22,075 process, it generates a blob of config that 745 00:27:22,075 --> 00:27:25,455 is then uploaded into your routing equipment. 746 00:27:28,440 --> 00:27:30,119 But this may give a false sense of 747 00:27:30,119 --> 00:27:32,299 security because it might be the wrong 748 00:27:32,600 --> 00:27:33,100 blob. 749 00:27:33,720 --> 00:27:34,220 And 750 00:27:34,600 --> 00:27:37,660 as these blobs become larger and larger because 751 00:27:37,880 --> 00:27:38,380 of, 752 00:27:39,065 --> 00:27:39,565 circular, 753 00:27:40,424 --> 00:27:40,924 recursive, 754 00:27:41,384 --> 00:27:41,884 references 755 00:27:42,265 --> 00:27:43,164 or references 756 00:27:44,265 --> 00:27:45,964 to wrongly named objects, 757 00:27:47,144 --> 00:27:49,944 you may be like, wow. My configuration is 758 00:27:49,944 --> 00:27:50,765 10 megabytes. 759 00:27:51,109 --> 00:27:52,650 It must be super secure. 760 00:27:53,029 --> 00:27:55,349 But the opposite is happening. What what happens 761 00:27:55,349 --> 00:27:57,130 is you you made a, 762 00:27:58,309 --> 00:28:00,170 a door that is wide open, 763 00:28:00,549 --> 00:28:02,490 but it is obfuscated by, 764 00:28:03,910 --> 00:28:04,410 the 765 00:28:05,029 --> 00:28:07,875 the the sheer size of the the configuration 766 00:28:08,015 --> 00:28:08,994 that was generated. 767 00:28:09,454 --> 00:28:12,414 And it's not actually blocking anything anymore because 768 00:28:12,414 --> 00:28:15,615 it's it's it became a really expensive allow 769 00:28:15,615 --> 00:28:16,434 any any. 770 00:28:17,054 --> 00:28:17,554 And 771 00:28:17,855 --> 00:28:19,315 if you think of the firewall 772 00:28:20,570 --> 00:28:23,470 policy term, allow any any, it's one sentence, 773 00:28:23,529 --> 00:28:25,609 three words. You know? It's a very short 774 00:28:25,609 --> 00:28:26,109 way 775 00:28:26,490 --> 00:28:27,390 of describing, 776 00:28:28,890 --> 00:28:30,809 what you want to pass through. I mean, 777 00:28:30,809 --> 00:28:31,309 everything. 778 00:28:32,035 --> 00:28:34,835 But imagine you could also make a list 779 00:28:34,835 --> 00:28:35,815 of 4,000,000,000 780 00:28:35,875 --> 00:28:39,315 entries with the 4,000,000,000 IP four addresses and 781 00:28:39,315 --> 00:28:41,255 look try and load that into your router. 782 00:28:41,315 --> 00:28:43,555 You'd be like, look. It's super secure. My 783 00:28:43,555 --> 00:28:45,974 firewall filter is 4,000,000,000 entries. 784 00:28:46,835 --> 00:28:48,849 And then so at some point, somebody's gonna 785 00:28:48,849 --> 00:28:50,789 come along and be like, you know, Jill, 786 00:28:51,009 --> 00:28:53,349 this is actually it's, you know, it's very 787 00:28:53,410 --> 00:28:56,049 expensive to maintain, but it's not secure because 788 00:28:56,049 --> 00:28:56,549 you 789 00:28:56,929 --> 00:28:59,250 you you tricked yourself. It doesn't allow any 790 00:28:59,250 --> 00:29:00,529 any and not a 791 00:29:01,545 --> 00:29:04,345 allow a specific subset that complies with my 792 00:29:04,345 --> 00:29:04,845 business, 793 00:29:05,305 --> 00:29:05,805 requirements. 794 00:29:06,825 --> 00:29:09,225 So this is a challenge with IR because 795 00:29:09,225 --> 00:29:11,545 when we talk with operators, they're like, hey. 796 00:29:11,545 --> 00:29:13,305 But I'm I'm doing the right thing right. 797 00:29:13,305 --> 00:29:13,740 I, 798 00:29:14,140 --> 00:29:16,700 the MANRRS initiative is encouraging me to filter 799 00:29:16,700 --> 00:29:19,339 the routes, and I've generated giant filters and 800 00:29:19,339 --> 00:29:20,880 uploaded those to my routers. 801 00:29:21,420 --> 00:29:23,179 What am I doing wrong? Why are you 802 00:29:23,179 --> 00:29:23,679 unhappy? 803 00:29:24,299 --> 00:29:26,059 And then you have to go and explain, 804 00:29:26,059 --> 00:29:27,965 like, well, it's it's it's a garbage in 805 00:29:27,965 --> 00:29:30,945 garbage out system. So you've pulled garbage in 806 00:29:31,245 --> 00:29:32,065 that looks 807 00:29:32,605 --> 00:29:34,305 legit, but you have no 808 00:29:34,684 --> 00:29:37,325 way of knowing where the data originated. There's 809 00:29:37,325 --> 00:29:37,825 no 810 00:29:38,205 --> 00:29:39,424 cryptographic signatures 811 00:29:39,805 --> 00:29:41,025 on the data itself. 812 00:29:42,759 --> 00:29:45,259 And you may be downloading this via FTP. 813 00:29:45,320 --> 00:29:46,700 Okay. Then it's not secure. 814 00:29:47,160 --> 00:29:49,259 But and maybe you download it through HTTPS. 815 00:29:50,119 --> 00:29:52,759 Great. But that's transport security. That doesn't tell 816 00:29:52,759 --> 00:29:53,660 us anything 817 00:29:54,119 --> 00:29:55,019 about whether 818 00:29:55,505 --> 00:29:58,065 that AS set was really created by Ross 819 00:29:58,065 --> 00:29:58,725 or Doug, 820 00:29:59,585 --> 00:30:00,805 or someone else. 821 00:30:01,184 --> 00:30:03,985 So the data is extremely problematic to deal 822 00:30:03,985 --> 00:30:06,404 with, but it it gives off these vibes 823 00:30:06,545 --> 00:30:08,465 of, you know, if you work with this 824 00:30:08,465 --> 00:30:10,200 data, you're doing something right. 825 00:30:11,480 --> 00:30:14,279 And persuading people to take a step back 826 00:30:14,279 --> 00:30:16,599 and let go of, you know, practices that 827 00:30:16,599 --> 00:30:19,099 they've been following for twenty plus years 828 00:30:19,480 --> 00:30:20,859 is not always easy. 829 00:30:21,240 --> 00:30:23,899 Yeah. So so my baby ugly. 830 00:30:26,355 --> 00:30:28,994 Yes. Sorry. Yeah. If it's IRR based, it 831 00:30:28,994 --> 00:30:30,535 is not a pretty baby. 832 00:30:30,994 --> 00:30:31,974 So so, Doug, 833 00:30:32,755 --> 00:30:34,454 you would you say that 834 00:30:35,075 --> 00:30:35,734 the origin 835 00:30:36,194 --> 00:30:38,295 AS number and and and 836 00:30:38,700 --> 00:30:40,559 prefixes are fairly solid. 837 00:30:40,940 --> 00:30:43,259 That's not been my experience when looking at 838 00:30:43,259 --> 00:30:43,839 the table. 839 00:30:44,779 --> 00:30:46,859 It's solid enough that it gives you the 840 00:30:46,859 --> 00:30:49,039 same sense that, like, Joe was talking about. 841 00:30:49,259 --> 00:30:52,234 Like, oh, look. Yes. 99% 842 00:30:52,234 --> 00:30:54,154 of this time, this prefix comes from this 843 00:30:54,154 --> 00:30:55,454 particular AS number, 844 00:30:56,394 --> 00:30:58,095 but that doesn't necessarily 845 00:30:58,714 --> 00:31:01,434 Okay. You're saying, like like, what's reflected in 846 00:31:01,434 --> 00:31:02,095 the IRR, 847 00:31:02,634 --> 00:31:04,575 data? Or Yeah. Oh, okay. 848 00:31:05,279 --> 00:31:06,880 I make no claims as far as the 849 00:31:06,880 --> 00:31:08,320 accuracy. I'm not, 850 00:31:09,119 --> 00:31:09,619 I, 851 00:31:10,080 --> 00:31:11,759 I don't, you know, in the in the 852 00:31:11,759 --> 00:31:14,160 work I do, I I rarely use that 853 00:31:14,160 --> 00:31:15,940 other than, like, in this pursuit. 854 00:31:17,039 --> 00:31:17,539 So, 855 00:31:18,400 --> 00:31:18,980 I would 856 00:31:19,394 --> 00:31:21,315 I would not be surprised by any kind 857 00:31:21,315 --> 00:31:23,955 of data gaps that exist there. Interesting. So 858 00:31:23,955 --> 00:31:24,994 I I didn't I don't think it's it's 859 00:31:24,994 --> 00:31:26,355 meant for a, you know, like a, 860 00:31:27,634 --> 00:31:29,315 Well, it seems to me it seems to 861 00:31:29,315 --> 00:31:31,394 be responsive that way. Go ahead. Yeah. It 862 00:31:31,394 --> 00:31:32,994 seems to me that part of the problem 863 00:31:32,994 --> 00:31:34,755 we face here is that when the Internet 864 00:31:34,755 --> 00:31:35,559 first started 865 00:31:36,740 --> 00:31:37,640 oh my goodness. 866 00:31:38,019 --> 00:31:39,960 Telling my age. Whatever. Anyway, 867 00:31:41,779 --> 00:31:44,339 there was a high correlate. Things were fairly 868 00:31:44,339 --> 00:31:44,839 steady. 869 00:31:45,619 --> 00:31:46,759 Like, you could look 870 00:31:47,355 --> 00:31:49,755 at an AS number or an AS path, 871 00:31:49,755 --> 00:31:51,434 and you had a good guess as to 872 00:31:51,434 --> 00:31:54,234 geographic regions. You had a pretty good guess 873 00:31:54,234 --> 00:31:56,494 as to what prefixes should be there. 874 00:31:56,875 --> 00:31:58,394 If you'd been on the on the d 875 00:31:58,394 --> 00:32:00,154 f z long enough, you just kinda knew 876 00:32:00,154 --> 00:32:00,894 these things. 877 00:32:01,515 --> 00:32:04,330 Nowadays, like you said, Doug, with with DDoS 878 00:32:04,789 --> 00:32:05,450 in particular, 879 00:32:05,910 --> 00:32:08,490 and also even caching and Anycast, 880 00:32:09,350 --> 00:32:09,850 and 881 00:32:10,309 --> 00:32:13,049 all of the other insanity that we see 882 00:32:13,350 --> 00:32:14,650 in routing flexibility, 883 00:32:15,335 --> 00:32:17,674 the way people are using the routing system 884 00:32:17,894 --> 00:32:19,255 to do a lot of things it was 885 00:32:19,255 --> 00:32:20,795 never designed to do. 886 00:32:21,734 --> 00:32:24,134 My sense is that there is just not 887 00:32:24,855 --> 00:32:27,015 the correlations just don't exist the way they 888 00:32:27,015 --> 00:32:27,660 used to. 889 00:32:28,140 --> 00:32:29,519 There's there's way more 890 00:32:31,019 --> 00:32:33,900 exceptions. Or they never existed to begin with, 891 00:32:33,900 --> 00:32:36,220 and we are just fooling ourselves when we 892 00:32:36,220 --> 00:32:38,400 collect these memories from old Internet. 893 00:32:39,019 --> 00:32:40,859 Like, yeah. Yeah. We we used to know. 894 00:32:40,859 --> 00:32:42,884 And maybe we never knew, but we you 895 00:32:42,884 --> 00:32:45,045 know, the scale of our mistakes just did 896 00:32:45,045 --> 00:32:46,984 a bubble up to nationwide news 897 00:32:47,285 --> 00:32:48,105 all the time. 898 00:32:48,964 --> 00:32:49,464 So, 899 00:32:50,565 --> 00:32:51,785 you know, at a previous, 900 00:32:52,244 --> 00:32:54,644 employer, I I phased out a lot of 901 00:32:54,644 --> 00:32:55,464 IR based, 902 00:32:55,845 --> 00:32:56,345 filters 903 00:32:56,859 --> 00:32:59,359 because we concluded, like, hey. There is a 904 00:33:00,380 --> 00:33:01,440 business risk 905 00:33:01,819 --> 00:33:03,679 in loading these giant configurations 906 00:33:04,139 --> 00:33:06,079 into the equipment that is already 907 00:33:06,619 --> 00:33:07,119 overburdened 908 00:33:07,500 --> 00:33:10,205 by the task that that the the equipment 909 00:33:10,205 --> 00:33:11,964 has to fulfill in the normal line of 910 00:33:11,964 --> 00:33:15,005 duty. So, like, adding megabytes of conflict to 911 00:33:15,005 --> 00:33:15,505 that 912 00:33:15,805 --> 00:33:16,305 that 913 00:33:16,765 --> 00:33:17,505 in retrospect 914 00:33:17,805 --> 00:33:20,285 are not achieving what we hope they would 915 00:33:20,285 --> 00:33:22,924 achieve. They're not making things more secure. They're 916 00:33:22,924 --> 00:33:24,305 making things unpredictable, 917 00:33:25,029 --> 00:33:25,529 brittle, 918 00:33:26,230 --> 00:33:28,549 and using a lot of resources while doing 919 00:33:28,549 --> 00:33:29,849 that. And when you 920 00:33:30,390 --> 00:33:32,230 frame it like that, like, hey. We're making 921 00:33:32,230 --> 00:33:35,289 our network less secure with this routing security 922 00:33:35,430 --> 00:33:35,930 initiative. 923 00:33:37,430 --> 00:33:39,109 Yeah. It's it's a bit weird in the 924 00:33:39,109 --> 00:33:39,914 beginning, but 925 00:33:40,315 --> 00:33:42,255 eventually, people warmed up to the idea. 926 00:33:42,955 --> 00:33:43,695 And especially 927 00:33:44,394 --> 00:33:44,894 when, 928 00:33:45,595 --> 00:33:47,934 I contrast it like, okay. If we remove 929 00:33:48,075 --> 00:33:49,295 IRR, we can 930 00:33:50,234 --> 00:33:53,055 supersede it or replace it with or substitute 931 00:33:53,195 --> 00:33:55,799 it with some other technologies that have better 932 00:33:55,799 --> 00:33:56,700 scaling properties. 933 00:33:57,400 --> 00:33:58,140 Like RPKI, 934 00:33:58,519 --> 00:33:59,019 ROV, 935 00:33:59,480 --> 00:34:01,880 ASPIT. I I I wanna talk about something 936 00:34:01,880 --> 00:34:04,700 you may not expect. It's a non RPKI 937 00:34:05,000 --> 00:34:05,500 solution. 938 00:34:07,400 --> 00:34:09,579 Because routing security is 939 00:34:09,934 --> 00:34:11,474 more than just RPKI. 940 00:34:12,414 --> 00:34:14,675 The one I wanna talk about is RC 941 00:34:14,735 --> 00:34:18,034 nine two three four, BGP open policy. 942 00:34:19,135 --> 00:34:21,454 And it's a really, really neat trick. So 943 00:34:21,454 --> 00:34:23,635 it doesn't require a central database 944 00:34:24,069 --> 00:34:25,829 that can go out of date or be 945 00:34:25,829 --> 00:34:26,969 still or be insecure. 946 00:34:29,190 --> 00:34:30,089 All it requires 947 00:34:30,389 --> 00:34:32,730 is a per BGP announcement 948 00:34:33,429 --> 00:34:33,929 marker. 949 00:34:34,549 --> 00:34:36,329 It's called the only to customer 950 00:34:36,710 --> 00:34:37,530 path attributes. 951 00:34:38,644 --> 00:34:42,005 And the the routing equipment that recognizes this 952 00:34:42,005 --> 00:34:43,704 new BGP path attribute 953 00:34:44,324 --> 00:34:46,025 can use this to 954 00:34:46,484 --> 00:34:48,025 understand is a path 955 00:34:48,405 --> 00:34:49,545 possible or not. 956 00:34:50,085 --> 00:34:52,324 Is there a route leak going on? Yes 957 00:34:52,324 --> 00:34:52,920 or no? 958 00:34:54,039 --> 00:34:56,440 So how this works is you and I 959 00:34:56,440 --> 00:34:57,420 set up a session, 960 00:34:57,880 --> 00:35:00,460 and on this session, we configure, 961 00:35:02,039 --> 00:35:04,619 the relationship that we have to each other. 962 00:35:04,679 --> 00:35:07,099 So I'll configure that I am the 963 00:35:07,664 --> 00:35:08,965 provider, the vendor, 964 00:35:09,585 --> 00:35:11,985 the upstream, like, whatever word you wanna use 965 00:35:11,985 --> 00:35:12,644 for it. 966 00:35:12,945 --> 00:35:15,344 And on your side, you configure that you're 967 00:35:15,344 --> 00:35:16,085 the customer. 968 00:35:17,585 --> 00:35:19,684 And then if both are, 969 00:35:20,809 --> 00:35:24,269 routing devices support RFC nine two three four, 970 00:35:24,889 --> 00:35:26,510 what happens is that this 971 00:35:26,889 --> 00:35:29,789 bhp path attribute is added to the announcements, 972 00:35:30,730 --> 00:35:33,630 and that helps restrict the scope of propagation, 973 00:35:34,755 --> 00:35:36,934 in such a way that it's no longer 974 00:35:37,315 --> 00:35:39,894 that it would not violate that model of, 975 00:35:40,434 --> 00:35:41,255 valley free, 976 00:35:41,875 --> 00:35:42,775 routing leaks. 977 00:35:43,635 --> 00:35:44,135 So 978 00:35:44,755 --> 00:35:45,255 long 979 00:35:45,795 --> 00:35:48,114 story short, but if I configure you to 980 00:35:48,114 --> 00:35:50,570 be a customer and you configure me to 981 00:35:50,570 --> 00:35:51,070 be 982 00:35:51,449 --> 00:35:54,329 the upstream provider, the routes you receive from 983 00:35:54,329 --> 00:35:54,829 me 984 00:35:55,130 --> 00:35:55,630 automatically 985 00:35:56,329 --> 00:35:57,469 will not be propagated 986 00:35:57,849 --> 00:35:59,230 to your other upstreams. 987 00:36:00,969 --> 00:36:02,269 So no cryptography 988 00:36:02,650 --> 00:36:04,269 involved, no central database. 989 00:36:04,844 --> 00:36:05,984 It's per prefix. 990 00:36:07,164 --> 00:36:09,105 So the information is packed 991 00:36:09,804 --> 00:36:12,304 right where it's needed to assess, 992 00:36:13,484 --> 00:36:16,144 the probability and possibility of apps. 993 00:36:17,690 --> 00:36:20,329 And and it there's almost no memory consumption. 994 00:36:20,329 --> 00:36:22,590 So it's it's a really, really cool technology, 995 00:36:23,530 --> 00:36:26,190 that is almost so simple to use that 996 00:36:26,329 --> 00:36:28,410 people might be skeptical, like, how can this 997 00:36:28,410 --> 00:36:29,550 be useful? 998 00:36:30,425 --> 00:36:32,045 And and and most implementations 999 00:36:32,344 --> 00:36:34,184 support this, by the way. Just for people 1000 00:36:34,184 --> 00:36:35,085 who are wondering, 1001 00:36:35,704 --> 00:36:36,844 as far as I know, 1002 00:36:37,224 --> 00:36:39,545 most implementations support this. Is that correct, Dov? 1003 00:36:39,545 --> 00:36:40,525 Is that your experience? 1004 00:36:40,905 --> 00:36:43,405 It it is a relatively new technology. 1005 00:36:43,789 --> 00:36:46,510 So the RFC was published, whatever, two, three 1006 00:36:46,510 --> 00:36:48,989 years ago. Mhmm. And you'll see that open 1007 00:36:48,989 --> 00:36:52,050 source BHP stacks have support for this mechanism, 1008 00:36:52,109 --> 00:36:52,929 like OpenBHPD 1009 00:36:53,469 --> 00:36:54,210 or Berth. 1010 00:36:55,150 --> 00:36:57,284 And in the commercial off the shelf world, 1011 00:36:57,364 --> 00:36:58,984 the the hardware based vendors, 1012 00:36:59,684 --> 00:37:01,684 it is coming. So if you use, like, 1013 00:37:01,684 --> 00:37:04,905 this year's release or last year's software release, 1014 00:37:05,364 --> 00:37:06,424 chances are, 1015 00:37:07,125 --> 00:37:09,284 there is support for RC nine two three 1016 00:37:09,284 --> 00:37:09,784 four. 1017 00:37:10,340 --> 00:37:12,340 And if not, you gotta ask your vendor 1018 00:37:12,340 --> 00:37:13,079 like, hey. 1019 00:37:13,539 --> 00:37:16,179 I wanna simplify my operations and make them 1020 00:37:16,179 --> 00:37:18,360 more reliable and more trustworthy. 1021 00:37:18,900 --> 00:37:20,280 I want support for 1022 00:37:20,739 --> 00:37:21,880 nine two three four. 1023 00:37:22,260 --> 00:37:24,260 So so in the past, if you weren't 1024 00:37:24,260 --> 00:37:26,355 a transit provider and you're just a customer 1025 00:37:26,355 --> 00:37:27,994 multiple networks, you'd use you would do it 1026 00:37:28,034 --> 00:37:30,034 use an AS path list that's basically empty 1027 00:37:30,034 --> 00:37:31,474 to do this. Right? And you would use 1028 00:37:31,474 --> 00:37:33,554 that. Is this a replacement for that then? 1029 00:37:33,554 --> 00:37:34,775 Or is it more 1030 00:37:35,394 --> 00:37:37,635 It it's it's sort of a replacement, but 1031 00:37:37,635 --> 00:37:40,500 it's also something new. So instead of trying 1032 00:37:40,500 --> 00:37:41,559 to port 1033 00:37:42,099 --> 00:37:43,320 the existing functionality 1034 00:37:43,619 --> 00:37:45,720 of ASets in the IR and 1035 00:37:46,340 --> 00:37:49,400 blindly copy over that concept into the RPKI, 1036 00:37:50,755 --> 00:37:53,795 something entirely different happened. Like, a fresh look 1037 00:37:53,795 --> 00:37:56,034 was taken at what is actually the problem 1038 00:37:56,034 --> 00:37:58,514 we're trying to solve, and a very different 1039 00:37:58,514 --> 00:38:00,695 solution direction came out of that. So 1040 00:38:01,074 --> 00:38:03,875 it's not a one to one replacement of 1041 00:38:03,875 --> 00:38:06,989 technologies that we've used up until this point. 1042 00:38:07,849 --> 00:38:09,230 But it it does, 1043 00:38:10,410 --> 00:38:10,910 substitute, 1044 00:38:11,690 --> 00:38:13,070 or overlap with 1045 00:38:13,369 --> 00:38:16,010 the intentions of the technologies that we should 1046 00:38:16,010 --> 00:38:16,750 be deprecating. 1047 00:38:17,210 --> 00:38:18,809 I think the other thing is, Tom, is 1048 00:38:18,809 --> 00:38:21,394 that instead of you expecting your customer to 1049 00:38:21,394 --> 00:38:23,494 do the carrot dollar sign 1050 00:38:23,954 --> 00:38:24,454 configuration 1051 00:38:25,315 --> 00:38:26,775 on all of their routers, 1052 00:38:27,074 --> 00:38:29,715 this is actually simpler for them. They just 1053 00:38:29,715 --> 00:38:31,815 say, yes, I'm a customer. In fact, 1054 00:38:32,114 --> 00:38:33,655 as a provider in Upstream, 1055 00:38:34,039 --> 00:38:36,199 you can say, I know you're my customer. 1056 00:38:36,199 --> 00:38:38,219 I'm not accepting your advertisements. 1057 00:38:38,760 --> 00:38:41,420 I'm not accepting your beer your peering session 1058 00:38:42,199 --> 00:38:44,380 until I see this capability negotiated 1059 00:38:45,559 --> 00:38:48,119 and and stuff. So that so that actually 1060 00:38:48,119 --> 00:38:49,500 gives you a bit more control. 1061 00:38:50,775 --> 00:38:52,795 And it's incrementally deployable. 1062 00:38:53,094 --> 00:38:53,594 So, 1063 00:38:55,175 --> 00:38:55,675 about, 1064 00:38:56,054 --> 00:38:58,375 eighteen months ago, I I reported on the 1065 00:38:58,375 --> 00:39:01,974 North America network operator list, that a Internet 1066 00:39:01,974 --> 00:39:05,320 exchange to Internet exchange leak was prevented. Thanks 1067 00:39:05,320 --> 00:39:05,820 to 1068 00:39:06,199 --> 00:39:06,940 the OTC, 1069 00:39:07,400 --> 00:39:09,500 the the nine two three four attribute. 1070 00:39:10,440 --> 00:39:12,920 Because what had happened is that the Internet 1071 00:39:12,920 --> 00:39:14,619 exchange in Calgary, Canada, 1072 00:39:16,039 --> 00:39:17,099 uses this mechanism, 1073 00:39:18,039 --> 00:39:18,539 and, 1074 00:39:19,335 --> 00:39:21,835 France IX had also deployed this mechanism. 1075 00:39:22,215 --> 00:39:24,614 And then there was a a large global 1076 00:39:24,614 --> 00:39:27,675 carrier that was connected to both Internet exchanges. 1077 00:39:27,734 --> 00:39:29,655 And for some reason, it was picking up 1078 00:39:29,655 --> 00:39:30,155 routes, 1079 00:39:30,934 --> 00:39:32,635 from the Paris Internet exchange 1080 00:39:33,349 --> 00:39:35,670 and then propagating those to the Calgary Internet 1081 00:39:35,670 --> 00:39:37,050 exchange. So this would be 1082 00:39:37,349 --> 00:39:39,690 a a lateral leak. So it's a peer 1083 00:39:40,390 --> 00:39:42,869 to peer. It it violates the the valley 1084 00:39:42,869 --> 00:39:43,269 free, 1085 00:39:43,909 --> 00:39:45,050 conceptual model. 1086 00:39:45,989 --> 00:39:47,130 But because the Calgary 1087 00:39:48,344 --> 00:39:49,804 route server recognized 1088 00:39:50,184 --> 00:39:52,664 the attributes and it recognized that it came 1089 00:39:52,664 --> 00:39:53,385 from a, 1090 00:39:54,105 --> 00:39:55,164 a peering participant, 1091 00:39:56,264 --> 00:39:59,164 it could recognize that the path was implausible 1092 00:39:59,739 --> 00:40:01,739 and it would reject the routes and the 1093 00:40:01,739 --> 00:40:02,719 loop stopped. 1094 00:40:03,180 --> 00:40:05,660 So on the global Internet, at that point 1095 00:40:05,660 --> 00:40:06,880 in time, only 1096 00:40:07,340 --> 00:40:10,640 a handful of ASs had implemented this technology, 1097 00:40:10,700 --> 00:40:12,800 and it was already giving us benefits. 1098 00:40:13,474 --> 00:40:15,335 And I think that's really cool when 1099 00:40:15,635 --> 00:40:17,175 even with limited deployment, 1100 00:40:17,795 --> 00:40:18,775 a new technology 1101 00:40:19,155 --> 00:40:21,494 immediately gives benefits to the participants. 1102 00:40:23,074 --> 00:40:25,394 If you gotta wait for everybody to do 1103 00:40:25,394 --> 00:40:25,894 something, 1104 00:40:26,269 --> 00:40:27,949 then you're gonna be waiting for a long 1105 00:40:27,949 --> 00:40:28,449 time. 1106 00:40:28,909 --> 00:40:30,929 Especially on the global Internet. Yeah. 1107 00:40:31,230 --> 00:40:33,809 Right. Right. So don't get it. So it's 1108 00:40:34,190 --> 00:40:36,349 Is there any data there or anything that 1109 00:40:36,349 --> 00:40:38,750 you wanted to talk about as far as 1110 00:40:38,750 --> 00:40:40,530 how common these still are? 1111 00:40:41,054 --> 00:40:43,074 Are are people still using these 1112 00:40:43,855 --> 00:40:47,234 AS sets? Oh, the AS sets? Yeah. 1113 00:40:49,295 --> 00:40:50,815 Yes. I I I think they are still 1114 00:40:50,815 --> 00:40:52,175 in use for lack of a better, 1115 00:40:52,655 --> 00:40:54,559 mechanism, and everybody kinda has to come up 1116 00:40:54,559 --> 00:40:55,699 with some way to, 1117 00:40:58,559 --> 00:41:00,840 yeah, mitigate the the issues. Go ahead, Darren. 1118 00:41:01,039 --> 00:41:01,940 Better mechanisms, 1119 00:41:02,800 --> 00:41:05,619 but we have not yet reached global community 1120 00:41:06,159 --> 00:41:06,659 consensus 1121 00:41:06,960 --> 00:41:07,940 what those mechanisms 1122 00:41:08,239 --> 00:41:10,195 are. So it feels like there's 1123 00:41:11,054 --> 00:41:13,715 an issue of education and outreach like, 1124 00:41:14,335 --> 00:41:17,474 hey. Instead of using IRR based, 1125 00:41:19,135 --> 00:41:21,954 deployment strategies, you should be looking into 1126 00:41:22,269 --> 00:41:26,050 the OTC attributes, into RPKI original validation. Maybe 1127 00:41:26,349 --> 00:41:29,090 start a pilot with ASFA filtering or, 1128 00:41:29,710 --> 00:41:33,070 use maximum prefix limits or like, there's all 1129 00:41:33,070 --> 00:41:33,570 these 1130 00:41:33,950 --> 00:41:36,369 steps you can take to improve the security 1131 00:41:36,430 --> 00:41:38,605 posture of your network, but you have to 1132 00:41:38,605 --> 00:41:39,345 be aware 1133 00:41:39,885 --> 00:41:40,385 that 1134 00:41:41,244 --> 00:41:44,385 new approaches exist and that they are overtaking 1135 00:41:44,525 --> 00:41:45,505 the older approaches. 1136 00:41:46,045 --> 00:41:48,364 So it sounds it sounds to me like 1137 00:41:48,364 --> 00:41:49,105 we need 1138 00:41:50,364 --> 00:41:51,905 an an Internet scale 1139 00:41:52,445 --> 00:41:52,945 routing 1140 00:41:53,739 --> 00:41:55,920 routing control plane protocol. Right? 1141 00:41:56,619 --> 00:41:58,460 Because, like, we have all these things in 1142 00:41:58,460 --> 00:42:00,380 different chunks. We have these IRR things with 1143 00:42:00,380 --> 00:42:02,380 ASX. We have these features that are coming 1144 00:42:02,380 --> 00:42:03,980 along. Like, it I don't know. It almost 1145 00:42:03,980 --> 00:42:05,280 seems like we need a, 1146 00:42:06,135 --> 00:42:07,515 a a management plane 1147 00:42:08,695 --> 00:42:10,614 sort of solution that you can Another plane 1148 00:42:10,614 --> 00:42:12,335 on top of the plane. Yeah. Yeah. Or 1149 00:42:12,375 --> 00:42:13,894 You encode all this stuff in it. Replace 1150 00:42:13,894 --> 00:42:16,474 or replace BGP with something more rational. 1151 00:42:18,215 --> 00:42:21,219 Okay. There's a there's a rational thought, Russ. 1152 00:42:21,760 --> 00:42:23,599 That's another that's another podcast, I think. 1153 00:42:24,960 --> 00:42:26,319 I I guess I guess what gives me 1154 00:42:26,319 --> 00:42:28,159 hope like, I think we're we're we're we're 1155 00:42:28,159 --> 00:42:30,480 talking about problems this whole time just about, 1156 00:42:30,799 --> 00:42:34,159 here, you know, the OTT OTC discussion of 1157 00:42:34,159 --> 00:42:37,275 this, solutions of just, like, how, you know, 1158 00:42:37,275 --> 00:42:39,594 how how difficult this is. You've gotta 1159 00:42:40,235 --> 00:42:42,074 anything you're gonna try to waste onto the 1160 00:42:42,074 --> 00:42:43,195 entire Internet is, 1161 00:42:43,914 --> 00:42:45,914 is a really difficult challenge. But I would 1162 00:42:45,914 --> 00:42:46,655 say that, 1163 00:42:47,510 --> 00:42:50,170 the experience with getting RKIROB, 1164 00:42:51,670 --> 00:42:53,829 adopted to the state that it's in, I 1165 00:42:53,829 --> 00:42:55,109 feel like it gives us hope. And a 1166 00:42:55,109 --> 00:42:56,550 lot of credit goes to Job for all 1167 00:42:56,550 --> 00:42:57,289 of his evangelism, 1168 00:42:57,750 --> 00:42:59,845 on that, touring the world trying to get 1169 00:43:00,005 --> 00:43:01,545 convince people to to do this. 1170 00:43:02,565 --> 00:43:05,204 And, and I think it is it is 1171 00:43:05,204 --> 00:43:06,965 possible, but it takes a lot of effort 1172 00:43:06,965 --> 00:43:08,985 and a lot of time to get, 1173 00:43:10,445 --> 00:43:12,130 a lot of the Internet, a sufficient amount 1174 00:43:12,130 --> 00:43:13,349 of the Internet to, 1175 00:43:13,730 --> 00:43:16,289 adopt a new thing, but, I don't know. 1176 00:43:16,289 --> 00:43:17,829 I I feel like my role in this, 1177 00:43:18,449 --> 00:43:20,369 at least last year was like, alright. Let's, 1178 00:43:20,849 --> 00:43:22,929 let's just let's start our conversation around this, 1179 00:43:23,170 --> 00:43:24,610 as an issue because I feel like everybody 1180 00:43:24,610 --> 00:43:26,644 kinda knows this, who's who's knowledgeable, 1181 00:43:26,945 --> 00:43:29,445 but there's not, like, a a open conversation. 1182 00:43:29,985 --> 00:43:31,744 I feel like that was helpful. I didn't 1183 00:43:31,744 --> 00:43:33,505 have a ton of solutions. Job's got a 1184 00:43:33,505 --> 00:43:35,684 lot of ideas here about o t OTC 1185 00:43:35,744 --> 00:43:38,144 and then even ASPA. So where an AS 1186 00:43:38,144 --> 00:43:38,804 would assert 1187 00:43:39,170 --> 00:43:40,550 who are its transit providers, 1188 00:43:41,090 --> 00:43:43,590 in a in a RTI, you know, system. 1189 00:43:44,210 --> 00:43:46,050 Others could then use that to pick pick 1190 00:43:46,050 --> 00:43:47,969 out a value free violation of AS path, 1191 00:43:47,969 --> 00:43:50,289 reject that route. So that's those are those 1192 00:43:50,289 --> 00:43:52,950 are two approaches to deal with these adjacency 1193 00:43:53,010 --> 00:43:53,510 leaks. 1194 00:43:54,130 --> 00:43:56,394 The ASPA thing is is very early on, 1195 00:43:56,394 --> 00:43:56,875 and, 1196 00:43:57,275 --> 00:43:58,974 you know, we'll we'll see how it goes. 1197 00:43:59,914 --> 00:44:00,315 But, 1198 00:44:00,954 --> 00:44:03,355 I don't know. There's there's there's hope. It 1199 00:44:03,355 --> 00:44:05,835 just doesn't take, it's not gonna be, you 1200 00:44:05,835 --> 00:44:08,469 know, in a year that you you get 1201 00:44:08,469 --> 00:44:11,190 a year. Feel that. But, you know, the 1202 00:44:11,190 --> 00:44:13,429 listeners of this podcast, if you're doing a 1203 00:44:13,429 --> 00:44:16,389 greenfield deployment, you you're getting some new modern 1204 00:44:16,389 --> 00:44:19,670 equipment. And with modern, we mean anything produced 1205 00:44:19,670 --> 00:44:20,889 in the last few years, 1206 00:44:21,614 --> 00:44:23,474 and you have OTC support, 1207 00:44:24,255 --> 00:44:25,394 I would use that 1208 00:44:26,255 --> 00:44:26,755 anytime, 1209 00:44:27,775 --> 00:44:28,574 instead of, 1210 00:44:29,295 --> 00:44:31,635 recreating an IRR based workflow. 1211 00:44:32,335 --> 00:44:36,114 Because with OTC, the configurations become much smaller, 1212 00:44:36,900 --> 00:44:38,039 much more precise, 1213 00:44:39,059 --> 00:44:39,880 much safer 1214 00:44:40,179 --> 00:44:43,139 even though it's a smaller configuration. So maybe 1215 00:44:43,139 --> 00:44:44,819 some old timers will be like, hey. That's 1216 00:44:44,819 --> 00:44:45,319 counterintuitive. 1217 00:44:45,859 --> 00:44:46,839 We thought safety 1218 00:44:47,380 --> 00:44:50,019 comes from having a multi megabyte configuration, but 1219 00:44:50,019 --> 00:44:50,519 that's 1220 00:44:50,875 --> 00:44:52,494 that's that's not the case anymore. 1221 00:44:53,835 --> 00:44:54,335 And 1222 00:44:54,715 --> 00:44:55,215 using 1223 00:44:55,835 --> 00:44:56,974 the modern approach, 1224 00:44:57,275 --> 00:44:58,094 it's gonna 1225 00:44:58,635 --> 00:44:59,515 it's gonna give, 1226 00:45:00,155 --> 00:45:02,094 provide less risk to the business. 1227 00:45:02,474 --> 00:45:04,309 So if we, you know, popularize 1228 00:45:04,690 --> 00:45:06,469 this this knowledge and 1229 00:45:06,930 --> 00:45:09,570 and tell people, like, you know, don't don't 1230 00:45:09,570 --> 00:45:12,690 start new IRR projects. Like, really evaluate. Are 1231 00:45:12,690 --> 00:45:15,030 there other approaches I could be using? 1232 00:45:15,730 --> 00:45:17,744 Then the answer to that is yes. 1233 00:45:18,144 --> 00:45:19,605 In in 2025, 1234 00:45:19,825 --> 00:45:22,704 people should be hesitant to invest in IRR 1235 00:45:22,704 --> 00:45:23,445 based solutions. 1236 00:45:23,905 --> 00:45:25,204 And 2026. 1237 00:45:25,664 --> 00:45:26,945 And 2026. 1238 00:45:26,945 --> 00:45:28,765 Yeah. Sorry. Yeah. Yeah. Yeah. 1239 00:45:29,345 --> 00:45:31,664 Twenty one days into 2026, 1240 00:45:31,664 --> 00:45:34,000 and I still haven't fully settled in this 1241 00:45:34,000 --> 00:45:34,500 year. 1242 00:45:35,759 --> 00:45:37,619 All your checks have been the wrong day. 1243 00:45:39,359 --> 00:45:41,139 So yeah. Cool. Awesome. 1244 00:45:41,440 --> 00:45:41,940 Okay. 1245 00:45:42,400 --> 00:45:42,900 So 1246 00:45:43,199 --> 00:45:45,359 get the word out. Don't use IRR based 1247 00:45:45,359 --> 00:45:48,159 solutions. Use other solutions. There are solutions out 1248 00:45:48,159 --> 00:45:50,454 there. I think that's a good that's an 1249 00:45:50,454 --> 00:45:52,074 actually a very, very good, 1250 00:45:52,614 --> 00:45:55,114 place to leave this and to get people 1251 00:45:55,175 --> 00:45:55,675 to, 1252 00:45:57,255 --> 00:45:58,554 to do what they 1253 00:45:58,855 --> 00:46:00,934 need to do, to just, you know, get 1254 00:46:00,934 --> 00:46:03,880 things done and get security increased on the 1255 00:46:03,880 --> 00:46:04,699 global Internet. 1256 00:46:05,880 --> 00:46:06,380 So 1257 00:46:07,320 --> 00:46:09,559 anything else before we wrap up, Tom? Any 1258 00:46:09,559 --> 00:46:11,179 more questions you wanna ask? 1259 00:46:11,480 --> 00:46:13,239 No. No. It's been really fun. Thanks, guys. 1260 00:46:13,239 --> 00:46:15,420 Alright. And Doug, any final 1261 00:46:15,800 --> 00:46:18,155 comments? Any I think that covers it. I 1262 00:46:18,155 --> 00:46:20,235 think we did. We covered it well. Alright. 1263 00:46:20,235 --> 00:46:21,054 And, Jobe, 1264 00:46:22,074 --> 00:46:23,934 come to Amsterdam. Right, Jobe? 1265 00:46:24,875 --> 00:46:27,054 Come to Amsterdam and let's deprecate 1266 00:46:27,755 --> 00:46:28,735 IRR together. 1267 00:46:31,809 --> 00:46:33,829 Yeah. That would be great. Alright. 1268 00:46:34,210 --> 00:46:36,050 So, Tom, where can people reach you if 1269 00:46:36,050 --> 00:46:36,869 they want to? 1270 00:46:38,690 --> 00:46:39,190 LinkedIn. 1271 00:46:39,969 --> 00:46:42,609 The the length of the pause varies. Depends 1272 00:46:42,609 --> 00:46:44,690 on the mood that Tom is in. And 1273 00:46:44,690 --> 00:46:46,434 we're not really sure what the the correlation 1274 00:46:46,434 --> 00:46:49,234 is yet, but there's We need more data. 1275 00:46:49,234 --> 00:46:50,214 We need more data. 1276 00:46:51,635 --> 00:46:52,135 Alright. 1277 00:46:52,514 --> 00:46:54,114 Doug, where can people reach you if they 1278 00:46:54,114 --> 00:46:55,635 want to or get in touch with you 1279 00:46:55,635 --> 00:46:57,155 or follow your blog? So, yeah, you can 1280 00:46:57,155 --> 00:46:58,855 find me on, on LinkedIn. 1281 00:46:59,234 --> 00:47:00,375 I'm still on 1282 00:47:01,019 --> 00:47:04,140 x and Blue Sky and Mastodon. I'm in, 1283 00:47:04,380 --> 00:47:05,980 I put a lot of stuff in all 1284 00:47:05,980 --> 00:47:08,380 those channels. Do you blog over at Kentech? 1285 00:47:08,380 --> 00:47:09,739 I think you do. Yes. I do. Thank 1286 00:47:09,739 --> 00:47:11,659 you for mentioning. I I, my company would 1287 00:47:11,659 --> 00:47:13,019 be disappointed if I didn't mention it. But, 1288 00:47:13,019 --> 00:47:14,940 yeah, so I wrote it. I, I write 1289 00:47:14,940 --> 00:47:17,255 quite a bit for, for the Kentic, 1290 00:47:17,954 --> 00:47:18,855 blog about, 1291 00:47:19,315 --> 00:47:19,815 both 1292 00:47:20,114 --> 00:47:21,795 this this kind of topic of kinda, like, 1293 00:47:21,795 --> 00:47:22,434 the routing, 1294 00:47:22,835 --> 00:47:24,694 kinda gearhead, I call it, 1295 00:47:26,114 --> 00:47:29,030 technical, parts of the Internet as well as, 1296 00:47:29,489 --> 00:47:31,170 a lot of stuff that goes on, at 1297 00:47:31,170 --> 00:47:34,150 the intersection between geopolitics and and Internet technology. 1298 00:47:34,210 --> 00:47:37,809 So Cool. Might find that interesting. Awesome. And, 1299 00:47:37,809 --> 00:47:39,650 Joe, where can people follow you if they 1300 00:47:39,650 --> 00:47:40,304 want to? 1301 00:47:41,585 --> 00:47:42,885 Just go to my website, 1302 00:47:43,744 --> 00:47:45,764 b s d dot n l, 1303 00:47:46,065 --> 00:47:46,464 and, 1304 00:47:46,864 --> 00:47:49,264 send me an email, joe@bsd.nl, 1305 00:47:49,264 --> 00:47:50,964 if you wanna talk shop about 1306 00:47:51,344 --> 00:47:51,844 strategies 1307 00:47:52,224 --> 00:47:55,019 to improve security and reduce business risk. 1308 00:47:55,739 --> 00:47:55,980 I, 1309 00:47:56,539 --> 00:47:58,480 I really like tinkering with large 1310 00:47:58,780 --> 00:48:01,579 networks and finding solutions that work well for, 1311 00:48:02,059 --> 00:48:04,380 for everyone. And, you know, if that means 1312 00:48:04,380 --> 00:48:06,300 that the entire world needs to change a 1313 00:48:06,300 --> 00:48:07,099 little bit, then, 1314 00:48:07,925 --> 00:48:09,385 hell yeah. Let's do it. 1315 00:48:11,525 --> 00:48:13,704 Not afraid of the world. That's Job. 1316 00:48:15,204 --> 00:48:17,045 Alright. Cool. Well, I'm Russ White. You can 1317 00:48:17,045 --> 00:48:18,244 find me here at the hedge at rule 1318 00:48:18,244 --> 00:48:19,304 eleven dot tech, 1319 00:48:19,844 --> 00:48:22,619 on LinkedIn, sometimes on x, Who knows where? 1320 00:48:22,619 --> 00:48:24,139 I'm really easy to find if you wanna 1321 00:48:24,139 --> 00:48:25,839 find me, so just go look around. 1322 00:48:26,139 --> 00:48:27,819 We know we live in an attention driven 1323 00:48:27,819 --> 00:48:28,319 economy 1324 00:48:28,619 --> 00:48:29,760 and that your attention, 1325 00:48:30,380 --> 00:48:32,139 is important in the world, and we thank 1326 00:48:32,139 --> 00:48:33,900 you for listening all the way to the 1327 00:48:33,900 --> 00:48:36,074 bitter end of this episode of The Hedge. 1328 00:48:36,714 --> 00:48:38,734 Turn off your IRR based stuff, 1329 00:48:39,034 --> 00:48:41,034 and we will catch you next time. And 1330 00:48:41,034 --> 00:48:44,494 turn on OTC. Like, don't go flying without 1331 00:48:44,714 --> 00:48:45,614 any precautions.