1 00:00:01,760 --> 00:00:03,839 Join us as we gather around the hedge, 2 00:00:03,839 --> 00:00:05,219 where we dig into technology, 3 00:00:05,519 --> 00:00:08,240 business, and culture with the finest minds in 4 00:00:08,240 --> 00:00:09,220 computer networking. 5 00:00:20,914 --> 00:00:23,494 Hello, Tom. Did you turn that plan around? 6 00:00:24,515 --> 00:00:25,015 Nope. 7 00:00:26,329 --> 00:00:27,929 I'm gonna in the same spot. I'm gonna 8 00:00:27,929 --> 00:00:29,370 have to drive all the way out to 9 00:00:29,370 --> 00:00:29,870 Austin. 10 00:00:30,730 --> 00:00:33,689 Yeah. Fix that plant. You should. That might 11 00:00:33,689 --> 00:00:34,969 be the only way it might be the 12 00:00:34,969 --> 00:00:36,109 only way it gets done. 13 00:00:39,484 --> 00:00:41,484 I don't know. Or maybe I call the, 14 00:00:41,884 --> 00:00:43,344 the local plant patrol, 15 00:00:45,405 --> 00:00:47,804 the wonder pets. I'll call them, you know, 16 00:00:47,804 --> 00:00:49,405 the phone the phone is ringing, and they 17 00:00:49,405 --> 00:00:50,844 can come over to your house and turn 18 00:00:50,844 --> 00:00:51,824 the plant around. 19 00:00:52,204 --> 00:00:52,704 Nice. 20 00:00:54,629 --> 00:00:56,469 If you see a raccoon wandering around your 21 00:00:56,469 --> 00:00:57,770 house, you know what happened. 22 00:01:00,710 --> 00:01:03,030 So today we are joined by Danny Wade. 23 00:01:03,030 --> 00:01:04,810 So, Danny, where are you physically? 24 00:01:05,349 --> 00:01:06,090 What what? 25 00:01:06,924 --> 00:01:09,085 Yeah. Russ, thanks for having me on. So 26 00:01:09,085 --> 00:01:11,665 I'm located in Maryland in The United States. 27 00:01:11,965 --> 00:01:12,465 Okay. 28 00:01:13,805 --> 00:01:15,585 Cool. So East Coast, 29 00:01:16,125 --> 00:01:18,365 nearest the bay and the ocean and all 30 00:01:18,365 --> 00:01:19,105 that stuff. 31 00:01:19,805 --> 00:01:21,325 It's a cool little spot up there in 32 00:01:21,325 --> 00:01:21,825 Maryland. 33 00:01:22,710 --> 00:01:24,490 They all do tend to get the occasional 34 00:01:24,549 --> 00:01:25,049 hurricane 35 00:01:25,430 --> 00:01:27,530 that comes up the coast and the nor'easters, 36 00:01:28,310 --> 00:01:30,250 which are brilliantly horrible. 37 00:01:30,870 --> 00:01:32,549 When I lived in New Jersey, we hated 38 00:01:32,549 --> 00:01:33,049 nor'easters. 39 00:01:33,750 --> 00:01:36,170 Like, oh, please stop it with the nor'easters. 40 00:01:37,585 --> 00:01:39,665 Yeah. Because they they come over the mountains 41 00:01:39,665 --> 00:01:41,505 cold and pick up all the water from 42 00:01:41,505 --> 00:01:43,505 all the lakes and the ocean, and then 43 00:01:43,505 --> 00:01:45,265 they run up the coast and, like, just 44 00:01:45,265 --> 00:01:47,204 dump snow everywhere. It's terrible. 45 00:01:48,145 --> 00:01:48,645 So 46 00:01:49,890 --> 00:01:51,890 let's see. So today, we are talking about 47 00:01:51,890 --> 00:01:55,189 Pytest and network automation and test driven 48 00:01:55,569 --> 00:01:58,069 or, yeah, automated driven testing, 49 00:01:58,689 --> 00:02:00,229 I think, is the topic. 50 00:02:00,609 --> 00:02:02,629 So I guess we should start with, 51 00:02:04,114 --> 00:02:04,935 now, PyATS 52 00:02:05,394 --> 00:02:07,634 is a Cisco only thing. Is that correct? 53 00:02:07,634 --> 00:02:08,854 Or are there 54 00:02:09,394 --> 00:02:12,435 non open source implementations as well? Or is 55 00:02:12,435 --> 00:02:12,754 this like 56 00:02:13,955 --> 00:02:16,215 Yeah. So it was developed by Cisco. 57 00:02:16,740 --> 00:02:18,740 So there's kind of two parts to it. 58 00:02:18,740 --> 00:02:20,840 So there's PyTS, the testing framework. 59 00:02:21,219 --> 00:02:22,199 And then there's PyTS, 60 00:02:22,500 --> 00:02:24,120 what they call the PyTS library, 61 00:02:24,419 --> 00:02:27,219 or formerly known as Genie. And so Genie, 62 00:02:27,219 --> 00:02:28,520 you may have heard of because 63 00:02:28,819 --> 00:02:29,800 it has the 64 00:02:30,224 --> 00:02:31,525 parsers and APIs 65 00:02:32,064 --> 00:02:33,044 and kind of the tools 66 00:02:33,344 --> 00:02:33,844 for 67 00:02:34,145 --> 00:02:34,805 the toolset. 68 00:02:35,664 --> 00:02:37,925 So the Genie portion, a lot of the 69 00:02:38,224 --> 00:02:40,164 parsers and everything are open source. 70 00:02:40,625 --> 00:02:42,784 It is a vendor agnostic tool, so it 71 00:02:42,784 --> 00:02:44,324 does work outside of Cisco. 72 00:02:45,120 --> 00:02:47,780 The testing framework, the PyTS piece, 73 00:02:48,479 --> 00:02:49,539 the testing framework 74 00:02:51,039 --> 00:02:52,500 is closed source. So 75 00:02:53,840 --> 00:02:55,699 it's still internally developed by Cisco. 76 00:02:56,560 --> 00:02:59,120 And the purpose around that and the actually, 77 00:02:59,120 --> 00:03:02,314 the framework itself is it was used internally 78 00:03:02,455 --> 00:03:04,294 at Cisco until it was publicly released, 79 00:03:04,935 --> 00:03:07,094 and I guess it still is used to 80 00:03:07,094 --> 00:03:07,594 test 81 00:03:07,895 --> 00:03:10,135 all their different flavors of software for their 82 00:03:10,135 --> 00:03:11,435 network operating systems. 83 00:03:13,014 --> 00:03:13,514 Okay. 84 00:03:14,139 --> 00:03:14,639 So, 85 00:03:16,300 --> 00:03:17,819 so the so there are parts of it 86 00:03:17,819 --> 00:03:19,340 that are internal to Cisco only. There are 87 00:03:19,340 --> 00:03:20,639 parts of it that are not. 88 00:03:22,219 --> 00:03:23,740 Yeah. That's a good way to summarize it. 89 00:03:23,740 --> 00:03:25,659 Yeah. Okay. And so the open source parts, 90 00:03:25,659 --> 00:03:27,819 are these something that the average network engineer 91 00:03:27,819 --> 00:03:29,360 could go download and, 92 00:03:30,754 --> 00:03:31,414 you know, 93 00:03:32,435 --> 00:03:34,354 use? Or do you need the internal parts 94 00:03:34,354 --> 00:03:35,495 as well-to-do this? 95 00:03:37,155 --> 00:03:37,955 So the 96 00:03:38,594 --> 00:03:40,354 so it really does come kind of in 97 00:03:40,354 --> 00:03:42,614 a bundled package with the testing framework. 98 00:03:43,189 --> 00:03:46,230 But from a contribution standpoint, you can anyone 99 00:03:46,230 --> 00:03:48,550 can go to GitHub today and look at 100 00:03:48,550 --> 00:03:49,930 the Genie parser library 101 00:03:50,469 --> 00:03:51,450 and contribute 102 00:03:51,830 --> 00:03:53,129 to new parsers, 103 00:03:53,590 --> 00:03:54,330 new APIs. 104 00:03:55,764 --> 00:03:57,784 There is actually a package 105 00:03:58,324 --> 00:03:59,944 of PyTS. I 106 00:04:00,485 --> 00:04:02,344 think it's called PyTS dot contrib, 107 00:04:03,525 --> 00:04:05,064 c o n t r I b. 108 00:04:05,604 --> 00:04:07,465 And you're able to create, 109 00:04:08,710 --> 00:04:09,990 I don't want to say plugins, but you're 110 00:04:09,990 --> 00:04:12,710 able to create ways to integrate with other 111 00:04:12,710 --> 00:04:14,790 tooling. So for example, a popular one that 112 00:04:14,790 --> 00:04:15,770 I like to use 113 00:04:17,110 --> 00:04:18,709 is, they have the concept of what they 114 00:04:18,709 --> 00:04:20,230 call a test bed, but essentially, it's an 115 00:04:20,230 --> 00:04:22,569 inventory file for the PyITS tool. 116 00:04:23,464 --> 00:04:24,125 You can 117 00:04:24,585 --> 00:04:25,964 actually part of that contrib 118 00:04:26,264 --> 00:04:26,764 package 119 00:04:27,225 --> 00:04:28,845 is a way to integrate with NetBox. 120 00:04:29,305 --> 00:04:31,305 So it's a CLI tool where you can 121 00:04:31,305 --> 00:04:34,045 connect to NetBox and generate basically a dynamic 122 00:04:35,305 --> 00:04:37,805 inventory file versus statically 123 00:04:39,149 --> 00:04:40,370 handwriting all your 124 00:04:41,069 --> 00:04:43,330 network inventory into a YAML file. 125 00:04:44,670 --> 00:04:46,910 Okay. And so does this work with non 126 00:04:46,910 --> 00:04:49,389 Cisco devices? I'm assuming that the Genie part 127 00:04:49,389 --> 00:04:49,889 does. 128 00:04:50,509 --> 00:04:51,970 Yes. Yeah. So, 129 00:04:52,430 --> 00:04:53,250 both do. 130 00:04:53,665 --> 00:04:56,324 Okay. Yeah. The testing framework really is 131 00:04:56,785 --> 00:04:59,504 if you're familiar with Python testing frameworks, there's 132 00:04:59,504 --> 00:05:01,605 one called unit test that's baked in 133 00:05:03,025 --> 00:05:04,165 to the standard library. 134 00:05:04,865 --> 00:05:06,564 It's very reflective of that. 135 00:05:07,025 --> 00:05:07,525 So 136 00:05:08,550 --> 00:05:11,029 the testing the PyTS side, the testing framework 137 00:05:11,029 --> 00:05:13,029 is agnostic. It's really just like any other 138 00:05:13,029 --> 00:05:14,490 software testing framework. 139 00:05:14,790 --> 00:05:17,589 The Genie side or the PyTS library side 140 00:05:17,589 --> 00:05:20,069 is the part that's open source and vendor 141 00:05:20,069 --> 00:05:22,714 agnostic that you can contribute to. So So 142 00:05:22,714 --> 00:05:24,354 it kind of feeds into that testing library 143 00:05:24,354 --> 00:05:25,175 or the framework. 144 00:05:25,555 --> 00:05:28,935 Awesome. Okay. Have you seen other vendors contribute 145 00:05:29,794 --> 00:05:30,294 to 146 00:05:30,834 --> 00:05:33,654 make their products usable with PyTS? 147 00:05:35,074 --> 00:05:37,654 So I've personally contributed to 148 00:05:38,439 --> 00:05:41,319 the Genie parser sign and, it it was 149 00:05:41,319 --> 00:05:44,360 a Cisco command. I've seen most I haven't 150 00:05:44,360 --> 00:05:46,040 looked through all the contributors, but most of 151 00:05:46,040 --> 00:05:48,600 them are individual contributors that have a problem 152 00:05:48,600 --> 00:05:50,759 and they fix it by creating a parser 153 00:05:50,759 --> 00:05:51,259 for, 154 00:05:51,665 --> 00:05:53,045 whether it be BIG IP 155 00:05:53,504 --> 00:05:54,245 or Junos. 156 00:05:55,745 --> 00:05:56,884 You can look through their 157 00:05:57,345 --> 00:05:58,805 parser library. But 158 00:05:59,345 --> 00:06:00,865 I see a lot of times it is 159 00:06:00,865 --> 00:06:02,004 like individual contributors. 160 00:06:03,264 --> 00:06:03,764 Okay. 161 00:06:04,330 --> 00:06:06,569 So this would so from what you're saying, 162 00:06:06,569 --> 00:06:09,050 this came out of, Cisco's own internal product 163 00:06:09,050 --> 00:06:09,550 testing. 164 00:06:10,009 --> 00:06:11,790 So what is the what is the motivation 165 00:06:12,009 --> 00:06:12,509 to, 166 00:06:13,210 --> 00:06:15,210 offer this to users? Do you find network 167 00:06:15,210 --> 00:06:17,770 operators doing it for network testing then instead 168 00:06:17,770 --> 00:06:20,004 of just product testing? Or what's the what 169 00:06:20,004 --> 00:06:22,185 was the motivation for releasing this? 170 00:06:23,845 --> 00:06:25,685 So I I I've never worked at Cisco, 171 00:06:25,685 --> 00:06:27,045 so I don't I don't know. I can't 172 00:06:27,045 --> 00:06:28,824 talk on behalf of Cisco. But, 173 00:06:29,605 --> 00:06:32,485 my assumption is it it's a it's an 174 00:06:32,485 --> 00:06:33,694 all around framework in 175 00:06:34,319 --> 00:06:36,259 for not just testing, but 176 00:06:36,800 --> 00:06:38,500 you build out an inventory file. 177 00:06:38,959 --> 00:06:39,860 And essentially, 178 00:06:40,560 --> 00:06:42,560 it creates an abstraction. Once you have an 179 00:06:42,560 --> 00:06:43,939 inventory file, each 180 00:06:44,479 --> 00:06:46,639 of those devices in your inventory file becomes 181 00:06:46,639 --> 00:06:48,099 basically a Python object. 182 00:06:48,479 --> 00:06:48,979 And 183 00:06:49,574 --> 00:06:51,975 through that, you have methods associated with it. 184 00:06:51,975 --> 00:06:53,735 So you can use like a there's a 185 00:06:53,735 --> 00:06:55,735 configure method and you could pass in a 186 00:06:55,735 --> 00:06:56,235 string. 187 00:06:56,615 --> 00:06:58,615 If you were to just go to conf 188 00:06:58,615 --> 00:06:59,675 t with Cisco, 189 00:07:00,615 --> 00:07:02,935 you could do it straight using Python instead 190 00:07:02,935 --> 00:07:05,459 of CLI scraping or anything like that. 191 00:07:07,199 --> 00:07:10,240 So my thought and the way I think 192 00:07:10,240 --> 00:07:11,599 a lot of people are thinking about it 193 00:07:11,599 --> 00:07:12,099 is 194 00:07:12,719 --> 00:07:14,800 just to be able to have a batteries 195 00:07:14,800 --> 00:07:17,675 included framework to not just test the test 196 00:07:17,675 --> 00:07:20,475 the, network, but also configure it and then 197 00:07:20,475 --> 00:07:20,975 retest. 198 00:07:21,675 --> 00:07:23,694 So it's really meant to, 199 00:07:24,074 --> 00:07:26,574 hey, I'm not just looking for all this, 200 00:07:26,714 --> 00:07:27,535 you know, 201 00:07:27,835 --> 00:07:30,220 operational data using show commands. What I'm trying 202 00:07:30,220 --> 00:07:31,660 to do is I want to configure a 203 00:07:31,660 --> 00:07:32,160 feature 204 00:07:32,540 --> 00:07:34,399 in a lab or closed off environment, 205 00:07:35,420 --> 00:07:37,580 see do run their show commands, see what 206 00:07:37,580 --> 00:07:40,240 changes, whether it be routing, layer two, whatever. 207 00:07:40,699 --> 00:07:43,264 And then I want to remove that configuration 208 00:07:43,264 --> 00:07:45,604 and see if everything resets that baseline. 209 00:07:46,544 --> 00:07:48,704 So when I say testing it, I see 210 00:07:48,704 --> 00:07:49,604 it as a larger 211 00:07:49,985 --> 00:07:53,024 testing of feature once being configured versus just 212 00:07:53,024 --> 00:07:54,404 running a bunch of show commands. 213 00:07:54,979 --> 00:07:58,199 So Sure. Sure. Yeah. Hopefully, answers your question. 214 00:07:58,579 --> 00:08:01,220 Cool. So so almost halfway to intent based, 215 00:08:01,220 --> 00:08:03,379 it almost sounds like, or the original generation 216 00:08:03,379 --> 00:08:05,620 of intent based. Like, let me go try 217 00:08:05,620 --> 00:08:08,019 this, see what happens, and if it doesn't 218 00:08:08,019 --> 00:08:10,194 work, pull it out, or at least verify 219 00:08:10,194 --> 00:08:11,875 that it's doing what I think it should 220 00:08:11,875 --> 00:08:12,615 be doing 221 00:08:13,235 --> 00:08:15,555 based on, like, looking at show output and 222 00:08:15,555 --> 00:08:16,375 stuff like that. 223 00:08:16,995 --> 00:08:19,475 Yeah. Yeah. And and within the testing framework, 224 00:08:19,475 --> 00:08:22,215 there's even ways to abstract, you know, parameters. 225 00:08:22,435 --> 00:08:24,535 So you could have a 226 00:08:24,879 --> 00:08:27,920 a non developer fill out a YAML file, 227 00:08:27,920 --> 00:08:30,819 call it, something very basic human readable 228 00:08:31,360 --> 00:08:33,860 of test parameters like I'm expecting 229 00:08:34,240 --> 00:08:37,519 this AS number for this BGP process. I'm 230 00:08:37,519 --> 00:08:38,955 expecting these number of routes. 231 00:08:39,434 --> 00:08:41,035 And you fill out a parameter file, and 232 00:08:41,035 --> 00:08:42,875 then you give it to the developer to 233 00:08:42,875 --> 00:08:45,115 run all the tests or configure those tests 234 00:08:45,115 --> 00:08:46,254 and then run them. 235 00:08:46,955 --> 00:08:49,195 So, yeah, you could definitely make it intent 236 00:08:49,195 --> 00:08:49,695 based. 237 00:08:50,554 --> 00:08:51,754 I don't know. I don't want to say 238 00:08:51,754 --> 00:08:53,835 it's the first generation, but it definitely is 239 00:08:53,995 --> 00:08:56,350 you could you know, deployed as an intent 240 00:08:56,350 --> 00:08:57,250 based framework. 241 00:08:57,789 --> 00:09:00,350 Right. Yeah. Yeah. Because I'm just thinking I 242 00:09:00,350 --> 00:09:02,909 was thinking, like, early generation only because that's 243 00:09:02,909 --> 00:09:03,409 where, 244 00:09:03,789 --> 00:09:05,870 like, a lot of the intent based products 245 00:09:05,870 --> 00:09:07,549 started out was doing that type of work. 246 00:09:07,549 --> 00:09:08,450 Now they've expanded 247 00:09:09,024 --> 00:09:11,584 to other things since then, and I assume 248 00:09:11,584 --> 00:09:12,804 that using this framework, 249 00:09:13,345 --> 00:09:15,584 you could actually stick your own LLM behind 250 00:09:15,584 --> 00:09:18,004 it and do things that are more dynamic 251 00:09:18,065 --> 00:09:20,144 if you ever really wanted to. But this 252 00:09:20,144 --> 00:09:21,985 would give you just the base library to 253 00:09:21,985 --> 00:09:23,159 do that kind of thing. 254 00:09:23,639 --> 00:09:25,240 It seems like you would be able to 255 00:09:25,240 --> 00:09:26,600 interact with it in that way if you 256 00:09:26,600 --> 00:09:27,419 wanted to. 257 00:09:27,959 --> 00:09:29,959 Yeah. And I haven't been catching up with 258 00:09:29,959 --> 00:09:32,139 him recently, but, John Capobianca, 259 00:09:32,440 --> 00:09:35,159 the guy I co coauthored the book with, 260 00:09:35,399 --> 00:09:37,240 the PyTS book, it, 261 00:09:37,845 --> 00:09:40,245 he's doing something along those lines where he's 262 00:09:40,245 --> 00:09:40,745 backing 263 00:09:41,284 --> 00:09:43,064 this framework with an LLM 264 00:09:43,365 --> 00:09:44,664 and doing some AI. 265 00:09:45,845 --> 00:09:48,245 I'm not too deep into the AI stuff 266 00:09:48,245 --> 00:09:49,845 right now. So but, 267 00:09:50,269 --> 00:09:52,509 from things I've seen he post online, he's 268 00:09:52,509 --> 00:09:54,610 definitely trying to take it to that extent. 269 00:09:54,990 --> 00:09:56,909 Well, that's really cool. Yeah. So this this 270 00:09:56,909 --> 00:09:58,690 acts as almost a middleware between 271 00:09:59,389 --> 00:10:01,009 the LLM and the devices. 272 00:10:01,950 --> 00:10:04,110 So let's so let's walk through, like, a, 273 00:10:04,110 --> 00:10:06,465 like, a fictional example. Say that I have, 274 00:10:07,024 --> 00:10:09,585 a reasonably sized OSPF network. I've got, I 275 00:10:09,585 --> 00:10:11,044 don't know, two or 300 routers. 276 00:10:12,144 --> 00:10:14,544 Pretty standard. Maybe it's a single area. Tom, 277 00:10:14,544 --> 00:10:16,384 you just dream so small, but go ahead. 278 00:10:16,384 --> 00:10:18,225 I know. Well, you gotta keep it real. 279 00:10:18,225 --> 00:10:20,245 Right? How many people run thousand router 280 00:10:20,649 --> 00:10:21,230 network? So, 281 00:10:22,330 --> 00:10:24,990 anyway, you got this network reasonably sized, and, 282 00:10:25,769 --> 00:10:28,570 it's deployed and things are working, but you 283 00:10:28,570 --> 00:10:29,470 want to see, 284 00:10:30,570 --> 00:10:32,649 what effect a certain change will have on 285 00:10:32,649 --> 00:10:33,309 your network. 286 00:10:33,674 --> 00:10:35,434 What how does how do you hook Py 287 00:10:35,595 --> 00:10:37,514 PyATS into it, and how do you how 288 00:10:37,514 --> 00:10:38,875 do you use what are the what's the 289 00:10:38,875 --> 00:10:41,134 concrete steps for making you get an answer 290 00:10:41,274 --> 00:10:43,195 from, oh, here's this thing on GitHub until 291 00:10:43,195 --> 00:10:44,634 now I know what's gonna happen. Like, what 292 00:10:44,634 --> 00:10:46,095 are the what's the workflow? 293 00:10:46,850 --> 00:10:48,529 Right. So I guess the first thing would 294 00:10:48,529 --> 00:10:51,009 be and this is a luxury, but having 295 00:10:51,009 --> 00:10:52,710 some sort of test or dev network, 296 00:10:53,170 --> 00:10:54,870 whether it be virtualized or physical. 297 00:10:55,330 --> 00:10:56,470 So under that assumption, 298 00:10:57,009 --> 00:10:59,490 you would to install PyATS, it's just a 299 00:10:59,490 --> 00:11:02,305 Python package. So using pip, you pip install 300 00:11:02,305 --> 00:11:03,845 pyts in the library. 301 00:11:05,184 --> 00:11:07,825 And then from there, what you do is 302 00:11:08,225 --> 00:11:10,225 I mentioned unit tests earlier as part of 303 00:11:10,225 --> 00:11:12,004 the standard library within Python. 304 00:11:12,384 --> 00:11:15,285 You basically use Python classes to create 305 00:11:15,690 --> 00:11:18,490 test cases, individual test cases. And in each 306 00:11:18,490 --> 00:11:20,830 test case, you would have an individual test. 307 00:11:21,370 --> 00:11:23,450 You actually could have a setup, a test, 308 00:11:23,450 --> 00:11:26,269 and a cleanup within one test case. So 309 00:11:26,409 --> 00:11:28,190 if you wanted to do something like OSPF, 310 00:11:29,794 --> 00:11:31,794 you would have an initial, what they call, 311 00:11:31,794 --> 00:11:32,615 common setup, 312 00:11:33,075 --> 00:11:34,855 which would connect to all the devices, 313 00:11:35,315 --> 00:11:37,495 right? So just SSH connections out. 314 00:11:37,875 --> 00:11:39,315 And then you would jump into your test 315 00:11:39,315 --> 00:11:42,500 case of maybe running a show command as 316 00:11:42,799 --> 00:11:43,460 a setup. 317 00:11:43,840 --> 00:11:46,019 So you're collecting maybe some data beforehand. 318 00:11:46,559 --> 00:11:48,080 And then you would go into the testing 319 00:11:48,080 --> 00:11:49,779 of configuring the device. 320 00:11:50,399 --> 00:11:52,000 And you could break that out into a 321 00:11:52,000 --> 00:11:53,519 test, or you can include that in the 322 00:11:53,519 --> 00:11:55,325 setup, however you want to do it. 323 00:11:56,105 --> 00:11:58,345 But the setup piece is really to add 324 00:11:58,345 --> 00:12:00,745 your configuration, add all your changes, and then 325 00:12:00,745 --> 00:12:01,725 run your test 326 00:12:02,664 --> 00:12:04,205 with the changes implemented. 327 00:12:05,225 --> 00:12:07,065 So you'd run the test, and then you 328 00:12:07,065 --> 00:12:07,725 would use 329 00:12:08,480 --> 00:12:09,139 the cleanup 330 00:12:09,680 --> 00:12:11,920 step to go through and wipe out your 331 00:12:11,920 --> 00:12:14,559 config and do, I guess, call it your 332 00:12:14,559 --> 00:12:15,059 baseline, 333 00:12:15,600 --> 00:12:16,580 making sure everything's 334 00:12:17,040 --> 00:12:19,840 where it's supposed to be. So that's kind 335 00:12:19,840 --> 00:12:21,360 of how you would run through it is 336 00:12:21,360 --> 00:12:22,259 creating that 337 00:12:23,095 --> 00:12:24,615 test case, and then you would run the 338 00:12:24,615 --> 00:12:27,414 test script across your inventory. And it would 339 00:12:27,414 --> 00:12:29,034 go out and take care of the connections, 340 00:12:29,575 --> 00:12:31,495 all the parsing of the show commands, and 341 00:12:31,495 --> 00:12:33,115 then the configuration as well. 342 00:12:34,054 --> 00:12:36,375 So what does the southbound, northbound interface look 343 00:12:36,375 --> 00:12:39,059 like on this thing? Are you working primarily 344 00:12:39,059 --> 00:12:41,960 with screen scraping? Are you primarily working with 345 00:12:42,980 --> 00:12:45,539 Yang models? Or is it are there multiple 346 00:12:45,539 --> 00:12:48,100 ways of communicating to the devices? Or how 347 00:12:48,100 --> 00:12:49,639 does how does that bit work? 348 00:12:50,095 --> 00:12:52,174 Yeah, that's a good question. Yeah, so they 349 00:12:52,174 --> 00:12:52,674 have 350 00:12:53,375 --> 00:12:56,195 most popular still is SSH, your common denominator. 351 00:12:56,414 --> 00:12:59,475 But there are options to connect via REST. 352 00:13:00,495 --> 00:13:03,134 Netconf is also so you're able to extract 353 00:13:03,134 --> 00:13:06,179 those Jing data models and and create or, 354 00:13:06,240 --> 00:13:09,519 you know, perform transactions using something like NetConf 355 00:13:09,519 --> 00:13:12,000 and then also Rest Comp as well. So 356 00:13:12,000 --> 00:13:14,980 there are alternative ways besides just screen scraping. 357 00:13:16,399 --> 00:13:19,214 Okay. Alright. That's good because that always is 358 00:13:19,214 --> 00:13:21,054 an issue, you know, with these with these 359 00:13:21,054 --> 00:13:22,274 types of things is 360 00:13:23,214 --> 00:13:25,534 and how do you handle yang models between 361 00:13:25,534 --> 00:13:25,934 different 362 00:13:26,735 --> 00:13:29,214 are there translation modules in there that you 363 00:13:29,214 --> 00:13:30,894 can say, okay. I'm working on a Juniper 364 00:13:30,894 --> 00:13:33,139 device. I'm working on a Cisco device. I'm 365 00:13:33,139 --> 00:13:35,700 working on, well, Bird doesn't have Yang. I'm 366 00:13:35,700 --> 00:13:37,080 working on FR routing. 367 00:13:37,539 --> 00:13:39,860 Do you have translation models? How do you 368 00:13:39,860 --> 00:13:40,919 deal with the different 369 00:13:42,259 --> 00:13:42,759 implementations 370 00:13:43,299 --> 00:13:44,919 of Yang or different 371 00:13:45,379 --> 00:13:46,919 different ways of dealing with Yang? 372 00:13:48,144 --> 00:13:48,884 So I mean, I think 373 00:13:49,585 --> 00:13:51,605 so built into the library? No. 374 00:13:51,985 --> 00:13:54,304 There's really not. You would have to either 375 00:13:54,304 --> 00:13:57,125 create those translations or use Python logic 376 00:13:59,825 --> 00:14:02,730 to I'm keeping the example simple. But you 377 00:14:02,730 --> 00:14:05,470 could say, hey, if device. Os equals 378 00:14:06,730 --> 00:14:09,950 iOS, then maybe look at these end capabilities. 379 00:14:10,090 --> 00:14:12,509 Or if it equals fr, do these capabilities. 380 00:14:12,570 --> 00:14:13,850 So you could 381 00:14:14,454 --> 00:14:17,014 using the device attributes assigned to the device 382 00:14:17,014 --> 00:14:19,975 objects, you could clearly delineate some of those 383 00:14:19,975 --> 00:14:20,875 different capabilities. 384 00:14:21,495 --> 00:14:23,034 That might be one way to do it. 385 00:14:23,414 --> 00:14:24,554 Okay. Interesting. 386 00:14:25,174 --> 00:14:27,254 So there's no way to consume a Yang 387 00:14:27,254 --> 00:14:27,754 model 388 00:14:28,294 --> 00:14:29,034 per se. 389 00:14:32,509 --> 00:14:34,269 Right? I'm I'm assuming there's no way to 390 00:14:34,269 --> 00:14:35,950 consume a Yang model from outside that would 391 00:14:35,950 --> 00:14:36,929 be rather sophisticated 392 00:14:37,790 --> 00:14:39,330 machinery in the software 393 00:14:39,790 --> 00:14:41,550 and, like, use it. You would just have 394 00:14:41,550 --> 00:14:44,350 to introduce possibly another Python library that can, 395 00:14:45,105 --> 00:14:46,304 take a look. Like I said, it would 396 00:14:46,304 --> 00:14:47,605 be outside the scope 397 00:14:48,144 --> 00:14:49,904 of the Pyts, but you could implement it, 398 00:14:49,904 --> 00:14:51,825 right? Because we're talking just Python at this 399 00:14:51,825 --> 00:14:53,605 point. Okay. All right. 400 00:14:54,544 --> 00:14:55,044 And 401 00:14:57,039 --> 00:14:59,279 any like, because it's in Python I mean, 402 00:14:59,279 --> 00:15:01,679 why Python? I'm always curious about this because 403 00:15:01,679 --> 00:15:03,600 different people do do different things in different 404 00:15:03,600 --> 00:15:04,100 languages. 405 00:15:04,480 --> 00:15:06,639 Like, is there a reason for Python instead 406 00:15:06,639 --> 00:15:09,519 of, like, Go or Rust or anything else? 407 00:15:09,519 --> 00:15:11,279 Is it just because it's such a common 408 00:15:11,705 --> 00:15:13,384 or is it because the open source stuff 409 00:15:13,384 --> 00:15:15,705 was already there, Genie was already there in 410 00:15:15,705 --> 00:15:18,284 that format, or was there a specific reason? 411 00:15:19,784 --> 00:15:22,184 Not so once once again, I don't I 412 00:15:22,184 --> 00:15:24,664 was on the development team, but my guess 413 00:15:24,664 --> 00:15:26,044 is just based on 414 00:15:26,509 --> 00:15:29,490 what I've seen, the amount of network automation 415 00:15:29,549 --> 00:15:32,049 focused libraries, Python seems to be pretty dominant. 416 00:15:32,350 --> 00:15:34,210 Ghost is catching up pretty significantly. 417 00:15:35,629 --> 00:15:37,789 But yeah, Python, I think, just 418 00:15:38,190 --> 00:15:39,549 and when it comes down to it, we're 419 00:15:39,549 --> 00:15:39,870 just 420 00:15:40,754 --> 00:15:42,914 what we're looking at is data parsing and 421 00:15:42,914 --> 00:15:43,735 data manipulation. 422 00:15:44,434 --> 00:15:46,674 And Python is still king, I think, in 423 00:15:46,674 --> 00:15:49,254 the data science world and just kind of 424 00:15:49,715 --> 00:15:51,955 be able to organize data. So that would 425 00:15:51,955 --> 00:15:53,414 be my guess. Okay. 426 00:15:53,875 --> 00:15:54,375 Yeah. 427 00:15:54,835 --> 00:15:56,054 Cool. That's good. 428 00:15:56,909 --> 00:15:58,269 Yeah. I mean, guesses are fine. I'm just 429 00:15:58,269 --> 00:15:59,629 trying to figure it out. Right? Like, is 430 00:15:59,629 --> 00:16:01,649 there anything else that's that's that's there? 431 00:16:02,589 --> 00:16:03,250 Yeah. Okay. 432 00:16:03,870 --> 00:16:06,370 So is there Danny, have there been any, 433 00:16:06,830 --> 00:16:08,669 have you come across any real world use 434 00:16:08,669 --> 00:16:10,450 cases where this approach 435 00:16:10,944 --> 00:16:13,924 saves somebody time, saves errors and trip configurations, 436 00:16:14,225 --> 00:16:17,044 saves something, anything in that area? 437 00:16:18,225 --> 00:16:19,044 Yeah. So, 438 00:16:19,985 --> 00:16:21,345 yes. I I work for, 439 00:16:22,144 --> 00:16:25,904 and I basically, a consulting service, and we 440 00:16:26,559 --> 00:16:27,379 we've implemented 441 00:16:27,920 --> 00:16:29,860 this framework at a couple of our clients. 442 00:16:30,240 --> 00:16:30,740 And 443 00:16:31,680 --> 00:16:33,920 one that I've actually was hands on with 444 00:16:33,920 --> 00:16:35,940 is being able to 445 00:16:37,519 --> 00:16:38,019 validate 446 00:16:39,424 --> 00:16:41,184 the team that I was working with was 447 00:16:41,184 --> 00:16:43,044 responsible for doing hardware refreshes 448 00:16:43,424 --> 00:16:44,565 at multiple sites, 449 00:16:45,504 --> 00:16:46,004 sometimes 450 00:16:46,625 --> 00:16:49,184 multiple within the same night or couple of 451 00:16:49,184 --> 00:16:50,084 days. So 452 00:16:50,784 --> 00:16:52,565 they're pretty fast churn here. 453 00:16:52,945 --> 00:16:55,670 So what you know, part of the refresh 454 00:16:55,670 --> 00:16:56,809 was obviously the configuration 455 00:16:57,190 --> 00:16:59,509 piece, which I think that's the number one 456 00:16:59,509 --> 00:17:02,470 use case is configuration management, generating configs, moving 457 00:17:02,470 --> 00:17:02,970 configs. 458 00:17:03,590 --> 00:17:06,630 But after the fact, once every once they 459 00:17:06,630 --> 00:17:08,664 swapped out all the hardware, they had to 460 00:17:08,664 --> 00:17:11,005 do all their validation with routing, layer two, 461 00:17:11,305 --> 00:17:13,964 even environmental factors like your CPU memory, 462 00:17:14,505 --> 00:17:16,904 power. Right? So making sure that even the 463 00:17:16,904 --> 00:17:18,684 power supplies were in a healthy state. 464 00:17:19,224 --> 00:17:20,825 And we were able to use the PyETS 465 00:17:20,825 --> 00:17:22,500 framework to validate all of that. 466 00:17:23,539 --> 00:17:26,019 So basically, the tool would run. It would 467 00:17:26,019 --> 00:17:26,519 have 468 00:17:27,380 --> 00:17:30,519 multiple artifacts. So it'd have pass fail results 469 00:17:30,660 --> 00:17:33,220 along with connection logs. So you could see 470 00:17:33,220 --> 00:17:35,140 the raw show output and everything that it 471 00:17:35,140 --> 00:17:36,039 did on the device. 472 00:17:36,740 --> 00:17:37,799 And it ran within 473 00:17:38,345 --> 00:17:39,884 one to two minutes versus 474 00:17:40,424 --> 00:17:42,585 running show commands that could take up to 475 00:17:42,585 --> 00:17:43,884 twenty, twenty five minutes. 476 00:17:44,744 --> 00:17:47,625 So, yeah, we we've definitely found we have 477 00:17:47,625 --> 00:17:51,119 found that the network validation and testing piece, 478 00:17:51,279 --> 00:17:53,599 not many people think of because, you know, 479 00:17:53,599 --> 00:17:56,160 it's not really that fun. Like, it's not 480 00:17:56,160 --> 00:17:58,160 it's not a top use case for many. 481 00:17:58,400 --> 00:18:00,579 It doesn't really create any change. It doesn't 482 00:18:00,880 --> 00:18:02,960 have any sort of progress in some network 483 00:18:02,960 --> 00:18:05,539 engineers' minds because it's not making those changes 484 00:18:05,599 --> 00:18:07,005 in solving their use cases. 485 00:18:07,565 --> 00:18:09,724 But it has so much valuable data that 486 00:18:09,724 --> 00:18:11,265 you might not know about your network. 487 00:18:11,644 --> 00:18:12,384 Right? So, 488 00:18:13,005 --> 00:18:14,845 it helps like I like to say, it 489 00:18:14,845 --> 00:18:16,865 validates the design that was implemented, 490 00:18:17,805 --> 00:18:19,079 or invalidates it. 491 00:18:20,119 --> 00:18:22,440 So your network architects create a really nice 492 00:18:22,440 --> 00:18:25,079 design, say, all right, engineers go implement this 493 00:18:25,079 --> 00:18:26,220 or ops go implement. 494 00:18:26,839 --> 00:18:29,159 And everyone just assumes it's working as expected 495 00:18:29,159 --> 00:18:31,079 until there's a problem. But what if you 496 00:18:31,079 --> 00:18:32,140 could just constantly 497 00:18:32,744 --> 00:18:35,144 run tests against your network to validate that 498 00:18:35,144 --> 00:18:37,325 design is implemented and running as expected? 499 00:18:38,345 --> 00:18:40,025 Yeah. This is something that I think is 500 00:18:40,025 --> 00:18:40,525 is 501 00:18:41,144 --> 00:18:42,285 really, really underrepresented 502 00:18:42,744 --> 00:18:45,144 in the industry today. Lots of people wanna 503 00:18:45,144 --> 00:18:46,980 talk about, oh, I took my my intent 504 00:18:46,980 --> 00:18:49,460 was translated into configurations, and I was able 505 00:18:49,460 --> 00:18:51,140 to make the little soldiers march, and they 506 00:18:51,140 --> 00:18:52,340 all did the things that I told them 507 00:18:52,340 --> 00:18:54,100 to do. And, you know, we wanna talk 508 00:18:54,100 --> 00:18:56,420 about, you know, monitoring bits per second on 509 00:18:56,420 --> 00:18:57,000 an interface. 510 00:18:57,380 --> 00:18:59,619 But it seems like there's almost no voices 511 00:18:59,619 --> 00:19:01,575 out there talking about, well, is the network 512 00:19:01,575 --> 00:19:02,394 actually working? 513 00:19:02,855 --> 00:19:04,875 And that seem it seems to me that 514 00:19:05,015 --> 00:19:07,194 this is is a strange gap. 515 00:19:07,654 --> 00:19:10,394 Like, why why aren't more people talking about 516 00:19:10,694 --> 00:19:12,535 is the network in the state that you 517 00:19:12,535 --> 00:19:13,654 wanted it to be in? Are all those 518 00:19:13,654 --> 00:19:15,549 routing tables correct like they should be? Are 519 00:19:15,549 --> 00:19:17,149 all the interface states the right I think 520 00:19:17,149 --> 00:19:18,669 people think of this as a monitoring problem, 521 00:19:18,669 --> 00:19:20,750 but it's not. Validation is a is a 522 00:19:20,750 --> 00:19:23,230 different area. Yeah. And and yeah. In fact, 523 00:19:23,230 --> 00:19:24,509 I would say that was one of my 524 00:19:24,509 --> 00:19:26,769 first things when I encountered intent based networking 525 00:19:27,069 --> 00:19:29,775 was that people tend to think, I'm gonna 526 00:19:29,775 --> 00:19:30,994 go check the configuration 527 00:19:32,015 --> 00:19:34,095 Right. Every five minutes and make sure the 528 00:19:34,095 --> 00:19:36,755 configuration hasn't changed. And I think, 529 00:19:37,055 --> 00:19:39,455 yeah, but the configuration, like, unless somebody's going 530 00:19:39,455 --> 00:19:41,795 in on the box and doing it, configuration 531 00:19:41,934 --> 00:19:44,674 isn't gonna change. What changes is, 532 00:19:45,059 --> 00:19:46,740 do I have all the BGP peers I 533 00:19:46,740 --> 00:19:48,579 expect? Do I have the, all the USPF 534 00:19:48,579 --> 00:19:50,039 neighbors do I expect? 535 00:19:50,579 --> 00:19:52,819 Am I recursing into the correct place? Is 536 00:19:52,819 --> 00:19:54,679 this route following that particular, 537 00:19:54,980 --> 00:19:57,220 you know, going off the right interface? Is 538 00:19:57,220 --> 00:19:59,335 that what I expect? Or is there something 539 00:19:59,335 --> 00:20:00,794 different going on in the network? 540 00:20:01,095 --> 00:20:02,615 And I need to go figure out why 541 00:20:02,615 --> 00:20:05,414 it is this particular route is following that 542 00:20:05,414 --> 00:20:07,654 thing over there instead of this thing over 543 00:20:07,654 --> 00:20:08,154 here. 544 00:20:08,855 --> 00:20:11,575 Yeah. Configuration management, like, if we focus on 545 00:20:11,575 --> 00:20:13,809 that to the exclusion of other things, it's 546 00:20:13,809 --> 00:20:15,890 it's like you you're seeing, like, 2% of 547 00:20:15,890 --> 00:20:17,590 the entire problem space. 548 00:20:18,289 --> 00:20:19,570 In in in my world where I do 549 00:20:19,570 --> 00:20:21,890 a lot of product testing, configuration is a 550 00:20:21,890 --> 00:20:22,630 mere suggestion. 551 00:20:22,930 --> 00:20:25,490 Like Yeah. That is not a command. It 552 00:20:25,490 --> 00:20:27,555 is not a a priority or authority of 553 00:20:27,555 --> 00:20:29,315 anything. It is a, here's a good idea 554 00:20:29,315 --> 00:20:30,755 that you might wanna do a system, but, 555 00:20:30,755 --> 00:20:32,434 you know, you're it's the system's gonna do 556 00:20:32,434 --> 00:20:34,755 its something. And Yeah. Yeah. Well, and and 557 00:20:34,755 --> 00:20:37,494 that's and that's a thing too, right, that 558 00:20:37,875 --> 00:20:40,134 we tend to think of configuration, 559 00:20:40,515 --> 00:20:41,414 and this is 560 00:20:42,329 --> 00:20:44,490 an argument throughout the entire world. In fact, 561 00:20:44,490 --> 00:20:46,089 I use this as a I use this 562 00:20:46,089 --> 00:20:48,509 as an interview question. Are recipes 563 00:20:49,609 --> 00:20:50,109 suggestions? 564 00:20:51,529 --> 00:20:53,769 Are they patterns to follow, or are they 565 00:20:53,769 --> 00:20:54,829 chemical formulas? 566 00:20:55,484 --> 00:20:57,805 And quite often, we think of a recipe 567 00:20:57,805 --> 00:20:58,625 or a configuration 568 00:20:59,164 --> 00:21:01,265 as a formula. You go do this. 569 00:21:01,724 --> 00:21:02,625 When in reality, 570 00:21:03,164 --> 00:21:04,785 you're trying to solve a problem, 571 00:21:05,565 --> 00:21:07,085 right, which is to make a really good 572 00:21:07,085 --> 00:21:08,605 cake or a really good biscuit or a 573 00:21:08,605 --> 00:21:10,880 really whatever it is. And so 574 00:21:11,580 --> 00:21:13,420 you may not have all the same ingredients 575 00:21:13,420 --> 00:21:14,619 or whatever, and you might need to go 576 00:21:14,619 --> 00:21:15,980 change things, or you may just like it 577 00:21:15,980 --> 00:21:17,339 a little bit different, or you might have 578 00:21:17,339 --> 00:21:19,599 a, you know, whatever it is. 579 00:21:20,059 --> 00:21:22,299 And I think that we tend to, particularly 580 00:21:22,299 --> 00:21:24,400 in the world of network validation and configuration, 581 00:21:24,964 --> 00:21:27,224 we think of things as being very fixed. 582 00:21:27,684 --> 00:21:29,365 There's one right way to do it or 583 00:21:29,365 --> 00:21:30,424 whatever it is. 584 00:21:30,884 --> 00:21:33,525 The reality is there's not one right way 585 00:21:33,525 --> 00:21:34,265 to do it. 586 00:21:35,684 --> 00:21:37,829 There's a thousand right ways to do it. 587 00:21:37,909 --> 00:21:39,909 And what's important is that you come out 588 00:21:39,909 --> 00:21:40,730 in the end 589 00:21:41,269 --> 00:21:43,210 getting done what needed to get done. 590 00:21:43,589 --> 00:21:44,970 That's really what's important. 591 00:21:45,750 --> 00:21:47,829 And so it sounds to me that's kind 592 00:21:47,829 --> 00:21:48,809 of what you're 593 00:21:49,269 --> 00:21:51,769 you're addressing in this space is 594 00:21:52,724 --> 00:21:53,704 not so much, 595 00:21:55,125 --> 00:21:56,964 just like what the configuration looks like, but 596 00:21:56,964 --> 00:21:59,045 like what the result is, what what what 597 00:21:59,045 --> 00:22:00,424 is coming out of this. 598 00:22:01,444 --> 00:22:01,944 So 599 00:22:02,404 --> 00:22:04,005 so so, Danny, why, 600 00:22:04,484 --> 00:22:06,279 why would I I think I know the 601 00:22:06,279 --> 00:22:07,320 answer, but I would like to hear your 602 00:22:07,320 --> 00:22:07,820 perspective. 603 00:22:08,200 --> 00:22:09,900 Why would someone use PyTS 604 00:22:10,200 --> 00:22:11,580 instead of just getting, 605 00:22:12,119 --> 00:22:15,000 get setting up Pytest with test infra for 606 00:22:15,000 --> 00:22:17,019 SSH into the devices and, 607 00:22:17,799 --> 00:22:19,160 you know, because you can you can pretty 608 00:22:19,160 --> 00:22:21,345 quickly make Pytest do most of this. 609 00:22:21,825 --> 00:22:23,664 What's the what's the use case? Why why 610 00:22:23,664 --> 00:22:25,204 is Cisco specific product? 611 00:22:26,464 --> 00:22:28,404 Yeah. That's a that's a good point. So, 612 00:22:28,865 --> 00:22:31,585 and and I've had other projects where I've 613 00:22:31,585 --> 00:22:32,085 used, 614 00:22:32,704 --> 00:22:34,799 pytest as well. Right? So, 615 00:22:35,200 --> 00:22:37,759 I definitely understand, like, this this is a 616 00:22:37,759 --> 00:22:40,259 problem that not pytest can only solve. 617 00:22:41,039 --> 00:22:43,440 Why, you know, between the two, I'd like 618 00:22:43,440 --> 00:22:45,360 to compare it to, like, if you're familiar 619 00:22:45,360 --> 00:22:47,565 with Python web frameworks, Django and Flask. 620 00:22:48,125 --> 00:22:50,524 So I've started my journey with Flask. And 621 00:22:50,524 --> 00:22:53,024 for those that aren't unfamiliar, Flask is very, 622 00:22:53,164 --> 00:22:53,664 very 623 00:22:55,004 --> 00:22:57,424 rudimentary. It has just the basics 624 00:22:57,884 --> 00:23:00,865 to just do requests and response to 625 00:23:02,480 --> 00:23:03,859 web requests and responses. 626 00:23:04,640 --> 00:23:05,140 And 627 00:23:05,599 --> 00:23:07,380 then you have to kind of use extensions 628 00:23:07,759 --> 00:23:09,359 to Hey, I want to talk to a 629 00:23:09,359 --> 00:23:10,740 database. Alright. I'll install 630 00:23:11,200 --> 00:23:12,099 Flask database. 631 00:23:12,400 --> 00:23:13,779 Hey, I want to talk to this. 632 00:23:14,240 --> 00:23:16,000 Okay. I got to learn about SQLAlchemy. I 633 00:23:16,000 --> 00:23:18,315 want to do this. So you kind of 634 00:23:18,315 --> 00:23:20,795 branch out very slowly as you need features, 635 00:23:20,795 --> 00:23:21,775 whereas Django, 636 00:23:22,234 --> 00:23:25,295 it's a batteries included framework where it has 637 00:23:25,355 --> 00:23:27,595 a database, it has an ORM, which, you 638 00:23:27,595 --> 00:23:29,434 know, you're able to communicate with that database. 639 00:23:29,434 --> 00:23:32,009 And so that's kind of where PyTS is. 640 00:23:32,009 --> 00:23:34,349 PyTS, I would say, is the Django of 641 00:23:34,490 --> 00:23:35,390 network testing. 642 00:23:36,329 --> 00:23:38,650 You have you have to define a test 643 00:23:38,650 --> 00:23:40,730 bed, but everything else is taken care of 644 00:23:40,730 --> 00:23:42,329 for you. If you want to configure the 645 00:23:42,329 --> 00:23:45,049 device, you're one method away from that. If 646 00:23:45,049 --> 00:23:46,565 you want to parse something, 647 00:23:46,944 --> 00:23:49,444 you call the dot parse method, right? So 648 00:23:49,664 --> 00:23:51,444 everything is just kind of built in. 649 00:23:52,304 --> 00:23:54,384 Obviously, there's more there's more to learn about 650 00:23:54,384 --> 00:23:56,005 it and there's a lot of documentation 651 00:23:56,464 --> 00:23:57,825 and now a book out there for you 652 00:23:57,825 --> 00:23:59,444 to understand the framework. But 653 00:23:59,904 --> 00:24:01,880 it really is just it's batteries included. You 654 00:24:01,880 --> 00:24:03,640 don't have to figure out all the different 655 00:24:03,640 --> 00:24:05,340 pieces, especially if you're green. 656 00:24:06,119 --> 00:24:07,019 But if you're 657 00:24:07,320 --> 00:24:10,600 experienced and you have some familiarity with pytest 658 00:24:10,600 --> 00:24:13,320 and some of the constructs there and you're 659 00:24:13,320 --> 00:24:16,039 familiar with other Python libraries like NetMico or 660 00:24:16,039 --> 00:24:17,019 Norni or Scrapply, 661 00:24:18,015 --> 00:24:20,195 where you could do the device connection and 662 00:24:20,735 --> 00:24:21,235 device 663 00:24:21,934 --> 00:24:23,634 being able to get data off the devices, 664 00:24:24,255 --> 00:24:26,414 then go for it. But yeah, I think 665 00:24:26,414 --> 00:24:28,815 the batteries included piece of IETF, I think, 666 00:24:28,815 --> 00:24:30,355 is what can draw 667 00:24:30,654 --> 00:24:32,115 a lot of people into it. 668 00:24:33,039 --> 00:24:33,859 Yeah. That's 669 00:24:34,240 --> 00:24:35,539 that's that's great. I 670 00:24:35,920 --> 00:24:37,920 the and I like I like that you 671 00:24:37,920 --> 00:24:39,059 use Django because, 672 00:24:39,519 --> 00:24:41,119 so I've I've no expert, but I've used 673 00:24:41,119 --> 00:24:42,799 Django a couple of times. There's a fair 674 00:24:42,799 --> 00:24:45,039 learning curve just learning how Django sees the 675 00:24:45,039 --> 00:24:47,384 world, and it's Django. It's not Python. It's 676 00:24:47,464 --> 00:24:49,464 Django. How does Django see the world? But 677 00:24:49,464 --> 00:24:52,184 then once you understand Django's perspective, things are 678 00:24:52,184 --> 00:24:53,945 so fast. You can do you can do 679 00:24:53,945 --> 00:24:57,545 enormously complex things, in very little time. It's 680 00:24:57,545 --> 00:24:58,700 just that you have to learn how Django 681 00:24:58,700 --> 00:25:00,720 sees the world, and, I can see how 682 00:25:00,720 --> 00:25:02,559 that would apply. And and with Pytest, you're 683 00:25:02,559 --> 00:25:04,079 you're totally right. Like, you have to invent 684 00:25:04,079 --> 00:25:06,160 all of the pieces yourself, and it's not 685 00:25:06,160 --> 00:25:07,680 terrible. I mean, if you can write Python, 686 00:25:07,680 --> 00:25:09,059 you can do all this stuff. But, 687 00:25:09,386 --> 00:25:11,119 that that's great. I think that's a great 688 00:25:11,119 --> 00:25:11,619 comparison. 689 00:25:11,920 --> 00:25:14,559 Are there any restrictions or trade offs for 690 00:25:14,559 --> 00:25:15,059 choosing 691 00:25:15,734 --> 00:25:17,575 something that gives you more of the bits 692 00:25:17,575 --> 00:25:18,714 and pieces prebuilt? 693 00:25:21,015 --> 00:25:23,414 That's always a question, right? Yeah. That that 694 00:25:23,414 --> 00:25:24,234 is. Yeah. 695 00:25:24,694 --> 00:25:27,194 Yeah. So, I mean, with with PyATS, 696 00:25:27,894 --> 00:25:30,720 I think it's especially if you're you're already 697 00:25:30,720 --> 00:25:32,420 one that's hesitant with Python, 698 00:25:33,200 --> 00:25:34,799 py ATS can get you up and running 699 00:25:34,799 --> 00:25:37,140 pretty quickly without having you 700 00:25:37,519 --> 00:25:39,940 without you having to understand the internals, 701 00:25:40,240 --> 00:25:42,079 right? So there's a lot of, I would 702 00:25:42,079 --> 00:25:42,819 say, abstraction, 703 00:25:43,335 --> 00:25:45,734 and there's some features actually within the framework. 704 00:25:45,734 --> 00:25:47,034 There's a feature actually called 705 00:25:47,414 --> 00:25:48,394 PyATS Blitz, 706 00:25:49,095 --> 00:25:51,095 that you don't have to write any Python, 707 00:25:51,095 --> 00:25:53,894 it's all YAML. It's a lot like Ansible 708 00:25:53,894 --> 00:25:54,634 in that way, 709 00:25:55,335 --> 00:25:57,990 where you can you still have the constructs 710 00:25:58,069 --> 00:26:00,230 of PyTS, but everything is defined in a 711 00:26:00,230 --> 00:26:02,869 YAML file instead of, you know, Python classes 712 00:26:02,869 --> 00:26:03,609 and functions. 713 00:26:04,390 --> 00:26:06,329 So there's I would say there's more abstraction 714 00:26:06,710 --> 00:26:08,390 on top of everything kind of being within 715 00:26:08,390 --> 00:26:10,569 the framework, the inventory system, 716 00:26:11,035 --> 00:26:12,494 the testing, the reporting, 717 00:26:12,954 --> 00:26:15,134 the data parsing, the data models, 718 00:26:15,835 --> 00:26:18,575 because there's a function within PyTS called, 719 00:26:19,035 --> 00:26:20,255 or the PyTS library 720 00:26:20,555 --> 00:26:22,335 called learn, where essentially 721 00:26:22,795 --> 00:26:25,035 you can learn about a network feature like 722 00:26:25,035 --> 00:26:25,535 OSPF 723 00:26:25,990 --> 00:26:27,829 and you can run the learn command against 724 00:26:27,829 --> 00:26:28,970 your entire testbed 725 00:26:29,349 --> 00:26:32,569 regardless of the operating system, whether it be 726 00:26:32,630 --> 00:26:33,609 iOS, Junos, 727 00:26:34,069 --> 00:26:36,710 anything supported by the framework. It will come 728 00:26:36,710 --> 00:26:39,109 back with a vendor agnostic, a data model, 729 00:26:39,109 --> 00:26:40,809 essentially, that you can query 730 00:26:41,154 --> 00:26:43,894 all those facts about that specific feature. 731 00:26:44,434 --> 00:26:46,674 And the scheme is documented and so forth 732 00:26:46,674 --> 00:26:48,454 in the Pytest documentation. 733 00:26:49,394 --> 00:26:50,994 So all of that is out of the 734 00:26:50,994 --> 00:26:51,494 box 735 00:26:51,954 --> 00:26:53,015 there for you. 736 00:26:54,289 --> 00:26:56,369 If you looked at an alternative route like 737 00:26:56,369 --> 00:26:56,869 iTest, 738 00:26:57,250 --> 00:26:58,630 you could build some of that, 739 00:26:59,730 --> 00:27:02,130 being able to get all the command all 740 00:27:02,130 --> 00:27:04,210 the show commands and being able to normalize 741 00:27:04,210 --> 00:27:06,595 that data set and then using a you 742 00:27:06,595 --> 00:27:08,695 know, data validation tool like a Pydantic. 743 00:27:09,555 --> 00:27:12,055 So you could do all of that manually, 744 00:27:12,355 --> 00:27:14,595 and it does provide more flexibility in that 745 00:27:14,595 --> 00:27:16,595 way. But like I said, if if you're 746 00:27:16,595 --> 00:27:18,615 coming from just traditional network engineering, 747 00:27:19,960 --> 00:27:21,320 you know, you don't I don't think many 748 00:27:21,320 --> 00:27:23,799 network engineers wanna get into data model and 749 00:27:23,799 --> 00:27:24,440 data validation, 750 00:27:25,400 --> 00:27:27,720 you know, day one. So I think there's 751 00:27:27,720 --> 00:27:29,320 there's a lot to be had with having 752 00:27:29,320 --> 00:27:30,299 that full framework 753 00:27:31,080 --> 00:27:32,059 at your fingertips. 754 00:27:34,855 --> 00:27:37,974 Okay. How does how does how does Pyet 755 00:27:38,055 --> 00:27:40,055 yes. So have you noticed in terms of 756 00:27:40,055 --> 00:27:42,394 performance, are there are things like concurrency 757 00:27:43,095 --> 00:27:44,795 built in if you have a large inventory 758 00:27:45,015 --> 00:27:45,914 that's all handled? 759 00:27:46,579 --> 00:27:47,240 Yeah. So, 760 00:27:47,700 --> 00:27:48,920 yeah. So there's a feature 761 00:27:50,259 --> 00:27:50,740 called, 762 00:27:51,220 --> 00:27:52,820 p it's called p call, but it's short 763 00:27:52,820 --> 00:27:55,640 for parallel calls. So, essentially, they do parallelism 764 00:27:55,940 --> 00:27:56,759 under the hood. 765 00:27:57,859 --> 00:27:59,480 So it's not necessarily 766 00:27:59,859 --> 00:28:02,894 your true concurrency, but it's parallelism. So you're 767 00:28:02,894 --> 00:28:03,714 doing multiprocessing 768 00:28:05,534 --> 00:28:07,375 to to hit all a bunch of different 769 00:28:07,375 --> 00:28:08,654 devices. So you could basically 770 00:28:09,294 --> 00:28:11,554 under the hood, you create, you know, multiple 771 00:28:11,774 --> 00:28:13,554 processes to connect to each device. 772 00:28:14,640 --> 00:28:15,539 And normally, 773 00:28:16,160 --> 00:28:18,160 if you don't introduce p call, you're just 774 00:28:18,160 --> 00:28:19,460 doing basically serial, 775 00:28:20,160 --> 00:28:22,240 hitting each device and going through the pecking 776 00:28:22,240 --> 00:28:22,740 order. 777 00:28:23,599 --> 00:28:25,940 So here's here's one one example where, 778 00:28:26,400 --> 00:28:28,275 if you don't want to deal with concurrency 779 00:28:28,275 --> 00:28:30,355 and threads and stuff, you know, this is 780 00:28:30,355 --> 00:28:31,715 where I yes. It would it would be 781 00:28:31,875 --> 00:28:33,715 like, large inventory. Great. Problem solved. One of 782 00:28:33,715 --> 00:28:35,575 the problems I've had to solve is concurrency. 783 00:28:36,035 --> 00:28:37,315 And so what do you do? Well, we 784 00:28:37,315 --> 00:28:39,475 turn to the Python threads library or whatever 785 00:28:39,475 --> 00:28:41,600 else, you know, whatever library. And, you know, 786 00:28:41,600 --> 00:28:43,680 you're smiling a little bit there, Danny. That 787 00:28:43,680 --> 00:28:45,519 comes with a certain amount of suffering. Right? 788 00:28:45,519 --> 00:28:47,140 Yeah. Yeah. The executor, 789 00:28:47,519 --> 00:28:48,820 the pulling. Yeah. 790 00:28:49,279 --> 00:28:50,100 Yeah. Yeah. 791 00:28:50,960 --> 00:28:53,279 And so so this probably also intersects with 792 00:28:53,279 --> 00:28:55,724 concurrency in a different way. Right? Sometimes you 793 00:28:55,724 --> 00:28:57,805 need to configure things in a certain order 794 00:28:57,805 --> 00:28:59,404 to make them work correctly, and you gotta 795 00:28:59,404 --> 00:29:00,144 make sure 796 00:29:00,605 --> 00:29:02,765 that step a was done before you do 797 00:29:02,765 --> 00:29:05,484 step b, where, otherwise, things don't come out 798 00:29:05,484 --> 00:29:07,565 correctly. And I assume there's a way of 799 00:29:07,565 --> 00:29:09,105 handling all of that 800 00:29:09,769 --> 00:29:11,389 type of stuff in PyATS. 801 00:29:12,809 --> 00:29:15,149 Yeah. Yeah. So and and that's, 802 00:29:16,409 --> 00:29:18,009 you're able to do that. A lot of 803 00:29:18,009 --> 00:29:19,849 a lot of that kind of logic that 804 00:29:19,849 --> 00:29:23,450 that logic handling is with the, the testing 805 00:29:23,450 --> 00:29:24,990 framework. So you can choose, 806 00:29:25,505 --> 00:29:28,545 hey, if this TestCase fails, which incorporates this 807 00:29:28,545 --> 00:29:29,045 logic, 808 00:29:29,505 --> 00:29:31,424 then we're skipping to the end and closing 809 00:29:31,424 --> 00:29:34,065 out the testing. Everything else fails, right? Or 810 00:29:34,065 --> 00:29:36,404 jump to this point. There's actually an interesting 811 00:29:37,839 --> 00:29:39,779 way I believe it's an argument. 812 00:29:40,159 --> 00:29:41,379 Whenever you 813 00:29:41,919 --> 00:29:42,980 do pass fail, 814 00:29:43,440 --> 00:29:45,200 because everything's in a class, you call on 815 00:29:45,200 --> 00:29:47,440 self. So it's like self. Pass, self. Fail, 816 00:29:47,440 --> 00:29:48,339 self. Whatever. 817 00:29:49,039 --> 00:29:50,099 In that argument, 818 00:29:51,039 --> 00:29:53,214 you could provide a string value as a 819 00:29:53,214 --> 00:29:55,615 reason for the result, why it passed, why 820 00:29:55,615 --> 00:29:57,694 it failed. But there's actually an argument called 821 00:29:57,694 --> 00:29:58,434 go to. 822 00:29:58,734 --> 00:29:59,474 And just 823 00:30:00,335 --> 00:30:01,934 like programming back in the day where you 824 00:30:01,934 --> 00:30:04,994 could jump around to different functions and use 825 00:30:05,134 --> 00:30:06,034 the go to 826 00:30:06,929 --> 00:30:09,029 functionality within, I think it was C. 827 00:30:09,569 --> 00:30:11,409 Yeah, C, C plus plus tells us Go 828 00:30:11,409 --> 00:30:12,849 to. I don't have too much experience with 829 00:30:12,849 --> 00:30:14,049 it, so that's why I was saying that. 830 00:30:14,049 --> 00:30:14,950 Yeah, that's fine. Yeah. 831 00:30:15,250 --> 00:30:16,849 But yeah, so you can actually use that 832 00:30:16,849 --> 00:30:18,529 and say Go to, and then you provide 833 00:30:18,529 --> 00:30:19,990 the name of the test case. 834 00:30:21,005 --> 00:30:23,404 So let's say you have five test cases, 835 00:30:23,404 --> 00:30:26,045 and if this device fails on test case 836 00:30:26,045 --> 00:30:28,605 two, skip three. You can either skip three 837 00:30:28,605 --> 00:30:30,525 or you could say go to test four, 838 00:30:30,525 --> 00:30:31,884 and then it will kind of keep going. 839 00:30:31,884 --> 00:30:34,205 So it's kind of interesting that there are 840 00:30:34,205 --> 00:30:35,025 some ways 841 00:30:35,404 --> 00:30:37,609 to handle that during the testing, 842 00:30:38,009 --> 00:30:38,509 phase. 843 00:30:39,929 --> 00:30:40,829 Yeah. Cool. 844 00:30:41,130 --> 00:30:42,970 So I'm assuming that if people wanna play 845 00:30:42,970 --> 00:30:44,490 with this, they could do it with something 846 00:30:44,490 --> 00:30:48,009 like container lab. Right? Like, doesn't require physical 847 00:30:48,009 --> 00:30:48,509 hardware. 848 00:30:48,970 --> 00:30:50,429 You can actually bring up 849 00:30:51,115 --> 00:30:52,174 virtual boxes 850 00:30:52,875 --> 00:30:55,595 using, I don't know, whatever whatever your poison 851 00:30:55,595 --> 00:30:56,974 is for virtual labbing 852 00:30:57,515 --> 00:30:58,015 and, 853 00:30:59,115 --> 00:31:00,875 play with this. Right? Like, this is not 854 00:31:00,875 --> 00:31:01,934 something that requires 855 00:31:02,634 --> 00:31:03,615 physical hardware. 856 00:31:04,109 --> 00:31:05,150 No, not at all, 857 00:31:05,710 --> 00:31:08,829 because there's nothing that's super data nothing intensive 858 00:31:08,829 --> 00:31:09,569 on the actual 859 00:31:09,869 --> 00:31:10,369 network 860 00:31:11,309 --> 00:31:14,049 devices themselves. We're just at it's either SSH 861 00:31:14,109 --> 00:31:15,009 or NETCONFRESCONFR 862 00:31:15,710 --> 00:31:16,769 connecting the device, 863 00:31:17,390 --> 00:31:19,884 and extracting data. I will say with it 864 00:31:19,884 --> 00:31:20,384 being 865 00:31:20,765 --> 00:31:23,164 a coming from Cisco, obviously, there's a lot 866 00:31:23,164 --> 00:31:24,865 of support for Cisco devices. 867 00:31:25,884 --> 00:31:27,724 And as you guys probably know if you've 868 00:31:27,724 --> 00:31:30,445 ever tried doing virtual labs, Cisco is it's 869 00:31:30,445 --> 00:31:32,765 interesting compared to the rest of the network 870 00:31:32,765 --> 00:31:33,904 vendors with containerization. 871 00:31:35,429 --> 00:31:37,750 Now you can use ways to convert the 872 00:31:37,750 --> 00:31:40,069 VMs to containers and so forth, and that's 873 00:31:40,069 --> 00:31:41,869 kind of what I do. But they get 874 00:31:41,990 --> 00:31:43,210 they can get kind of 875 00:31:43,829 --> 00:31:46,009 resource intensive if you spin up too many. 876 00:31:46,230 --> 00:31:46,730 So 877 00:31:47,029 --> 00:31:48,789 that's just just a warning if you do 878 00:31:48,789 --> 00:31:50,884 use something like a container lab. I do 879 00:31:50,884 --> 00:31:53,105 a lot of my testing just within CML. 880 00:31:54,365 --> 00:31:56,764 So once again, it is it's another Cisco 881 00:31:56,764 --> 00:31:58,784 product, but it works out really well. 882 00:31:59,244 --> 00:32:00,625 And they have a cool integration 883 00:32:01,724 --> 00:32:02,224 where, 884 00:32:02,684 --> 00:32:04,684 I'm sorry, it's really just a feature where 885 00:32:04,684 --> 00:32:05,984 if you build out a, 886 00:32:06,820 --> 00:32:08,759 a CML lab and 887 00:32:09,059 --> 00:32:11,299 do all your testing and so forth, you 888 00:32:11,299 --> 00:32:13,700 actually can there's a I think it's a 889 00:32:13,859 --> 00:32:15,380 I don't know if there's a UI feature, 890 00:32:15,380 --> 00:32:17,000 but there's an API endpoint 891 00:32:17,299 --> 00:32:19,799 to export the whole lab as a PyTS 892 00:32:19,859 --> 00:32:20,600 test bed. 893 00:32:21,285 --> 00:32:23,045 So you can kind of add that as, 894 00:32:23,045 --> 00:32:24,265 you know, if you're thinking 895 00:32:24,565 --> 00:32:26,265 down the line of, like, doing a CICD 896 00:32:26,325 --> 00:32:28,424 pipeline, you can call on CML, 897 00:32:28,884 --> 00:32:30,965 get a certain lab by its ID, and 898 00:32:30,965 --> 00:32:34,265 then call on that endpoint to generate 899 00:32:34,805 --> 00:32:35,384 a dynamic 900 00:32:36,059 --> 00:32:37,039 PyTS testbed. 901 00:32:37,580 --> 00:32:38,860 And then that way, you don't have to 902 00:32:38,860 --> 00:32:40,640 worry about Jinja templates 903 00:32:41,420 --> 00:32:43,420 or dynamically figuring out your own logic to 904 00:32:43,420 --> 00:32:45,420 create that testbed file. You can just use 905 00:32:45,420 --> 00:32:46,640 a CML topology 906 00:32:46,940 --> 00:32:48,080 to create that testbed. 907 00:32:49,820 --> 00:32:50,320 Interesting. 908 00:32:50,700 --> 00:32:51,200 Okay. 909 00:32:53,045 --> 00:32:55,285 Awesome. I don't. Do you have anything else, 910 00:32:55,285 --> 00:32:56,825 Tom, before we wrap up? 911 00:32:57,285 --> 00:32:59,205 No, it's been really great. Thank you. Anything 912 00:32:59,205 --> 00:33:00,805 for you, Danny? Is there some part of 913 00:33:00,805 --> 00:33:02,325 this that you wanted to cover that you 914 00:33:02,325 --> 00:33:02,825 didn't? 915 00:33:04,724 --> 00:33:08,269 No. Not really. I mean, I definitely it's 916 00:33:08,269 --> 00:33:10,190 a it's a big frame it's a big 917 00:33:10,190 --> 00:33:10,690 library. 918 00:33:11,710 --> 00:33:14,369 There's multiple features. I mentioned PyTS Blitz. 919 00:33:15,150 --> 00:33:16,369 There's some other interesting 920 00:33:16,750 --> 00:33:18,369 features within the 921 00:33:19,325 --> 00:33:21,484 within the library. So definitely take a look 922 00:33:21,484 --> 00:33:22,305 at the documentation. 923 00:33:23,805 --> 00:33:25,884 And I would just say, like, it's kind 924 00:33:25,884 --> 00:33:27,184 of a final word, like, 925 00:33:28,444 --> 00:33:31,085 explore network testing and validation, whether it's with 926 00:33:31,085 --> 00:33:33,589 PyTS or whether it's with Pytest or whether 927 00:33:33,589 --> 00:33:35,589 it's outside of Python and it's using Go 928 00:33:35,589 --> 00:33:37,929 or another testing framework. If you're able 929 00:33:38,309 --> 00:33:38,809 to 930 00:33:39,190 --> 00:33:41,190 get the data from the network, the testing 931 00:33:41,190 --> 00:33:41,690 part 932 00:33:42,069 --> 00:33:43,769 is pretty easy. So 933 00:33:44,085 --> 00:33:45,525 being able to just you have to just 934 00:33:45,525 --> 00:33:46,884 remember the network at the end of the 935 00:33:46,884 --> 00:33:49,464 day is just data. So whether it's configuration 936 00:33:49,924 --> 00:33:50,424 or, 937 00:33:50,724 --> 00:33:53,224 you know, the operational state data, 938 00:33:53,765 --> 00:33:55,525 getting that data out of your network. And 939 00:33:55,525 --> 00:33:56,964 once you have that data, there's a lot 940 00:33:56,964 --> 00:33:58,644 of really cool things you can do because 941 00:33:58,644 --> 00:34:00,730 the software development world has been doing it 942 00:34:00,730 --> 00:34:01,849 for years. So 943 00:34:02,170 --> 00:34:04,410 Yeah. Yeah. I would just say take take 944 00:34:04,410 --> 00:34:06,329 a look at network testing and validation as 945 00:34:06,329 --> 00:34:08,670 your next step with with automation. 946 00:34:09,929 --> 00:34:12,090 Yeah. That's really cool. Well, thanks for coming 947 00:34:12,090 --> 00:34:13,789 on. And the book is PyATS. 948 00:34:14,090 --> 00:34:14,474 Right? 949 00:34:14,954 --> 00:34:17,675 Correct. Yeah. Cisco Py ATS, network tests and 950 00:34:17,675 --> 00:34:18,735 automation solution. 951 00:34:19,434 --> 00:34:21,355 Awesome. Great. I'll try to put a link 952 00:34:21,355 --> 00:34:23,515 to it in the show notes if I 953 00:34:23,515 --> 00:34:24,015 remember. 954 00:34:24,715 --> 00:34:26,894 I never remember these things, so who knows? 955 00:34:28,299 --> 00:34:29,739 I always say my memory is like a 956 00:34:29,739 --> 00:34:32,400 steel sleeve. Only the big chunks stick. 957 00:34:32,699 --> 00:34:33,199 So 958 00:34:34,779 --> 00:34:37,339 that's the way it is. Alright. So, Danny, 959 00:34:37,339 --> 00:34:39,440 where can people follow you? Are you Twitter 960 00:34:39,739 --> 00:34:41,739 or X or LinkedIn or I don't know 961 00:34:41,739 --> 00:34:44,234 wherever. Where do you blog someplace? Yeah. So 962 00:34:44,234 --> 00:34:45,534 I have my own 963 00:34:45,914 --> 00:34:47,054 blog, DevNet Dan. 964 00:34:47,674 --> 00:34:49,614 That's also my Twitter 965 00:34:50,714 --> 00:34:52,894 account as well. It's also DevNet Dan, 966 00:34:53,194 --> 00:34:54,414 and I'm also on LinkedIn, 967 00:34:54,954 --> 00:34:57,614 not under DevNet Dan. You can find me, 968 00:34:58,000 --> 00:35:00,340 Daniel c Wade. It's just my full name, 969 00:35:00,880 --> 00:35:01,619 on LinkedIn. 970 00:35:02,079 --> 00:35:05,119 But, yeah, I'm most active probably on LinkedIn 971 00:35:05,119 --> 00:35:07,680 and Twitter these days. I gotta catch up 972 00:35:07,680 --> 00:35:09,619 on on the blog at some point. 973 00:35:10,559 --> 00:35:13,194 Yeah. Well, that's okay. I'm I'm the next 974 00:35:13,194 --> 00:35:15,135 person I ask is gonna be Tom, and 975 00:35:15,835 --> 00:35:16,734 go ahead, Tom. 976 00:35:17,114 --> 00:35:19,214 Tom's pitiful compared to you, Danny. 977 00:35:19,755 --> 00:35:22,074 Link LinkedIn. That's it. Just search for me 978 00:35:22,074 --> 00:35:22,655 on LinkedIn. 979 00:35:25,034 --> 00:35:26,574 Oh, I don't know. Alright. 980 00:35:27,570 --> 00:35:29,170 Okay. Well, I'm Russ White. You can find 981 00:35:29,170 --> 00:35:31,170 me here at the hedge @rule11.techat 982 00:35:31,170 --> 00:35:32,230 rule eleven academy 983 00:35:33,090 --> 00:35:33,989 on LinkedIn 984 00:35:34,610 --> 00:35:35,110 and 985 00:35:36,130 --> 00:35:37,650 on Twitter. I don't know. I check I 986 00:35:37,650 --> 00:35:39,010 check x a couple of times a day 987 00:35:39,010 --> 00:35:40,309 or whatever now. So 988 00:35:40,684 --> 00:35:42,125 feel free to DM me on any of 989 00:35:42,125 --> 00:35:44,385 those, come over to the academy or whatever. 990 00:35:44,684 --> 00:35:45,184 And 991 00:35:45,644 --> 00:35:47,244 we know that you live we live in 992 00:35:47,244 --> 00:35:49,344 an attention driven world and that, 993 00:35:49,644 --> 00:35:51,484 your intention is probably the most valuable thing 994 00:35:51,484 --> 00:35:52,684 in the world right now. In fact, if 995 00:35:52,684 --> 00:35:54,945 you could sell it, you'd probably be wealthy, 996 00:35:55,085 --> 00:35:56,929 but you can't, say you aren't. 997 00:35:58,109 --> 00:35:58,929 So, anyway, 998 00:35:59,949 --> 00:36:01,230 thanks for listening all the way to the 999 00:36:01,230 --> 00:36:02,589 end of this episode of The Hedge, and 1000 00:36:02,589 --> 00:36:04,130 we will catch you next time.