1:25 Introductions

1:49 Hello, and welcome. Hello, Taylor. How are you? Doing alright. How about yourself, David? Yeah. I'm very well. Thank you. It is getting towards the end of my day, so it's a nice way to end the day with a nice look at Kubernetes and spend some time with yourself. How are you today? Doing well. It's yeah. Just just kicking off my day personally here. So I promise I know you're at the end of the day, but I'm gonna try and bring that caffeine and some of that good morning energy to to the stream today. Awesome. I definitely appreciate it. That's for sure.

2:23 So you are a developer advocate at HashiCorp, and very relevant for today, you were the one nineteen Kubernetes release team lead. Do you wanna add anything to that and Absolutely. So so one nineteen, I just kind of, you know, I'll I'll I'll just kinda jump in and out of some topics on that front. But howdy, everyone. My name is Taylor Dolozol. I'm a senior developer advocate at HashiCorp, and really, really excited to talk to you today. Mostly gonna be covering the 01/19 release and what that experience was like, some of the fun new features around that, And

3:00 then just that whole release process, honestly, David, was was was a wild experience. Beforehand, I was working at Walt Disney Studios on on the SRE team and and and, you know, helping with all the efforts on that front. So the 01/19 release, you know, was one I I believe the longest release that we've had thus far between all of the Kubernetes releases that we've seen. And so kind of navigating, jumping between jobs in that, all the the wild things happening in the world, you know, it's just been it was a marathon release is what most of the

3:35 team had said around that, which is absolutely true. Yeah. Definitely. So I I wasn't actually aware that this was the the longest release, but I guess, you know, there's been a lot of different factors in 2020 that may have contributed to that. I believe you've been under a release team a while now. Right? How many releases? Absolutely. So I started on the one fourteen. It was, I believe, the first release that I was on. I saw the shadow survey and then jumped in and got started with the communications team. So for those of you that aren't aware, the release

4:11 team, it is one team, but it's broken into multiple different verticals. So we have, you know, one one grouping. It takes care of communications. Another one takes care of enhancements, for example, another documentation, CI signal. There are different groups that focus on different things throughout the course of a release. And so, typically, the the release teams in, like, in the, like, lower mid thirties, I think we had about, if if I'm counting correctly, I think it was, like, 33, 30 four people on the one nineteen release. Yeah. I think people watching this will probably be a little bit surprised that, you know,

4:47 with the size of Kubernetes as an open source project, that there it requires a lot of people and time and effort to actually get that release out the door. It's it's interesting too because you you know, the there are different special interest groups within Kubernetes for, again, you know, for those of you that aren't aware. And their, you know, SIG group the special interest group just for the release handles that release and then those those subgroups. But there are other ones like SIG API machinery, SIG docs, you know, and and even more on that front too. So no shortage of

5:20 people to ping or reach out to you on Slack if you have a question about a feature that's gonna make it or or not. And how have you found the release teams now? Let's see. If you said one fourteen is your first, this is your fifth release. Right? So you've you've experienced a lot of the role, I guess? Yeah. So I took I did take off the the one seventeen release cycle. So that was and kind of got to, you know, get a good get some rest and relaxation away from that. Because it does it does

5:49 take a lot out of you. But really, really been fun to see how the group has evolved in the different ways in which we go about communicating with one another and seeing, you know, all of the documentation and handoff processes get, you know, more and more mature from release to release. But and and, honestly, it's it's to in in on my perspective, my bias, I it has been a lot easier from release to release and just kind of that, you know, talking with people and even just, you know again, it could've just been the stars aligning for me with the one

6:24 nineteen team, but it just was so easy to work with everyone. You know, everyone was just so kind. Everyone was so upbeat. And because it's an open source project, you know, that's you're not there's no financial incentive, hence open source. Everybody's passionate about the project, and so they're bringing that passion to work with them every single day that you're working on that release. And so that really you know, that passion really powered this release. That's where the accentuate the positive title came from. And, just it was fantastic group of people to work with and, and everyone

7:00 was just, you know, even if there was a little slippage here or, you know, getting some done early or late, everybody was just there and it very much had that family feel to it, for lack of a better term. That's awesome. And I was very curious about to accentuate the positives, so I'm glad that we kinda covered that. I think every release now that I've at least been visible to or I've been paying attention to. It's like, these themes are always great, and those come from the release team leads. Right? You're one that accentuated the positive here. Right? Oh, absolutely.

7:31 Yeah. Just to just to to just trying to get do a good thing, pay it forward. Yeah. Each each of the releases has a theme that will come with it, which is chosen by the release lead. And then, typically, it's well, I I don't think it's ever been disclosed before the release has gone live, but that's usually kept a secret even, you know not even told to the release shadows as well. And so, you know, sometimes they're they they'll be like, hey. What do you think? You know, there's that back and forth kind of workshopping on the idea or, you know,

8:01 just kinda double checking to make sure everything all the the everything lines up. But this one was definitely intended to be a surprise and was a lot of fun picking that out. That for for for anyone that's seen it, it's a Animal Crossing themed type logo just very much in the spirit of. And, you know, that game came out just as the 01/19 release kicked on and really started up. And then, you know, as as we've had to go through COVID, as we've had to, you know, just injure endure all these socioeconomic and other, you know, the natural disasters, just

8:39 all these crazy things that we've had happen to us around the world, no better time than to focus on one another, really just being upbeat, finding out how you can help one another, and just kind of, you know, it just, you know, from joining hands with people and just kinda figuring out how to meet them where they're at and just to be, you know, good, kind, uplifting. And I I feel we really did that in 01/19, giving people time as, you know, some weeks were a lot harder than others, taking a break from meetings, you know, definitely around KubeCon. We took a

9:13 break there as well, but we really focused on the community in this release. You know? If you over overexert the community and the people working on this project, you're not gonna have a lot of people left to to work on the project if if they're all burnt out. Right? So it was a lot of fun to to work with people on that and just really actively listen and and see what was going on in the community. Excellent. I just popped up the great kind of artwork there. It's definitely one of my favorite ones. I think everybody's been a little bit

9:42 Animal Crossing that this year, so I thought it was a very fitting name, and I think the sentiment that you just described there as well with the release is fantastic as well. So great job then. Thank you. I I don't wanna get this wrong, but I believe that shadow applications are open right now for 01/20. Is that correct? That that's correct. I believe that they're closing up soon. So if you're looking to get your start on Kubernetes and interacting with the open source community, by all means, please please please check that out. I'll I'll try and find the link.

10:15 I'll I'll try and reach retweet that out later. Try to say that five times fast and and share that with y'all. But I do believe that's closing up. So and if you have any doubts about getting involved with open source, if you're on the fence, if you're having that that you know, if you're if you're undecided on that front, just do it. I obviously, make sure you have enough time to do so, but there are rules for for everyone. And the biggest the biggest piece of feedback that I've heard there is that a lot of people are concerned about,

10:48 you know, being judged or just they don't really feel comfortable sharing their code or their contributions with the world. You know, that's a that is a lot of scrutin scrutiny, but it's you're working with fantastic people, and they're not going to, you know, let anything bad happen. So long as you work with people, you have that community mindset, and and that's the goal. That's the intent that you're walking in with. You're you're not gonna be disappointed. But by all means, make sure that you have time to do it. There there I think when I selected the

11:19 communications team, when I started on that with the Kubernetes release cycles, that that role fit really well. But, you know, something like CI or or CI signal, that one is very involved and it requires a lot of communication, between team members. And so you can kind of pick what works for you, which is really nice about the open source project. Yeah. I think, just to kind of touch on one thing you said there. Like, you don't even have to write any code, you don't have to be fearful of writing code. You can join the release team at the

11:47 shadow and and start to experience some of those different roles and positions and, you know, never write a single layer of code. I know I have never written a single layer of code with the Kubernetes project. So and it's still a very positive experience. I really enjoyed the the couple of release teams that I had as a part of too, and but you get to meet so many cool people and just all those interactions. And and even if you are a little bit shy and you're curious or on the fence, you can just go and lurk in the

12:11 second release channel. You can join the mailing list. There are loads and loads of ways where you can get the mailer of it first, but they have to jump straight into the shadow applications. But we really do sorry. I had go. No. That's it's such a great point, David. I I think that it's such it's most open source projects, yeah, I I I've felt like that where it's just like you kinda jump in. You don't really know what to expect. Kubernetes, there's a whole bunch of, like, rule books. There's that shadow process. So you don't, you know it's you're not you you

12:38 don't have to be, like, you know, just completely in it and you have to know everything before you do anything. That's the whole purpose of that shadow program is so that you can take the time to see how things work and then ramp up until you're at a point where you feel comfortable to take over something in a in a in a lead capacity. Typically, we've we've advised that, like, two cycles of being in a shadow position before getting to that lead position is, you know, typically how that works out. But, yeah, it's it's come come come lurk, come ask

13:07 questions, come enjoy the banter and the conversation. It's not always about tech. You know, sometimes we will talk about Animal Crossing. We'll pop up in a Zoom chat on a Friday and just kinda talk about life. You know? Like, it's a it's it it's it's a family. It's community. It's it's a lot of fun. It doesn't have to be there there's equal parts work and play when it comes to working on Kubernetes, and that's that makes it a lot of fun. Yeah. Definitely. Okay. That's that's great. We will tweet other links. I'll make sure they're in the comment section as well in

13:30 Kubernetes 1.19 major themes

13:36 the YouTube video. The release team is a great experience. Hopefully, we've done it justice. We sold it really well there. Feel free to get involved. Now we're gonna talk about Kubernetes one nineteen. So it's got it's got some major themes. Like, we we've got a list here. Do you want to read through them? Do you wanna just dive into each one after one? What's your preferred format here? What would you like to do? Yeah. So so I'd love to kind of hit on a lot of major themes for 01/19. Then just kind of, you know, if if

14:10 they didn't if they weren't on your radar, we'd just kind of love that opportunity to put them on your radar. I I and I'll of course, I'll I'll point out which ones are my personal favorites, but we can do we we we can just jump through them. So one of the the biggest items that, happens in 01/19 is increasing the Kubernetes support window to one year, whereas previously, it was at nine months, because every three months, typically, you have a new Kubernetes release. And so that allowed for, you know, just just two back as soon

14:25 Extending the Kubernetes support window to 12 months

14:42 as one was one was published. So now at any given point in time, starting with 01/19, there's going to be four that get supported there. There's a lot of feedback from the community saying that, you know, nine months is a is a, you know, sizable amount of time. However, we'd like a little bit more time. So because it is you know, it it does get a little bit dizzying to have to upgrade your cluster with every single release that comes out, and and most teams aren't able to do that. Some can't do that even within the nine months period of time. So

15:13 we we we hear you, we listen to you, and and we have one year support. So that's that's a lot of fun to see. Yeah. I I think that's a a change that's gonna be welcomed by by so many. You know, Kubernetes has been such an integral part of of people's production infrastructure. Even updating every, you know, three months can be very daunting, but then getting to the end of a a supported release, even even skittier. But I think that extra bit of buffer, I will really really help a lot of companies. It's it's really gonna be fantastic, and I

15:45 do hope that, you know, my my selfish want on on that one with that feature specifically is some some of the managed Kubernetes options that you have. You know, I think that most of them are are are have one seventeen supported right now, but I you know, my personal want is to see that a little bit closer to what's, you know, what's gone GA. Like, seeing one eighteen just a couple months after it comes out would be great to see, same with one nineteen just so that the the community is able to kind of work with those features, see

16:14 see if it's, you know, good fit for them and, you know, and so on. But happy to see that, you know, we have those manager clusters available as well. And then for any any of you playing at home, I'd absolutely recommend checking out the kind project by the fantastic Ben the elder that allows you to spin up Kubernetes in Docker, kind, k I n d. And it really makes it easy for if you want to test something, you know, work with the control plane, or just kind of spin up, you know, like that. That's how I I

16:45 checked and and looked at and started working with a lot of the one nineteen team features out of the out of the box out of the container. Yeah. Kinda is definitely the easiest way to start kicking the tires on some of these features, especially even ManyCube as well has really good support for both of these. So, you know, you can you can have your shot with any. I think OneMain team is available on both pretty much immediately after the release, which it just makes it so easy for this. Oh, absolutely. Agree. As far I I I

17:13 don't wanna ask you an opinion piece so early on in our call, but I'm gonna do it anyway. There has been many conversations over the last twelve months about whether Kubernetes the release cycle is too fast. Like, and and corporations and companies are having troubles with speed up. Is that something that you see changing maybe in the future? Do you think it will slow down as the product continues to mature, or do you think that maturity will contain anything encourage new contributors and maybe speed it up? That's something I have thought a lot about myself.

17:15 Is Kubernetes releasing too often?

17:44 And I think that I personally, I do think that that will become the case at some point in time and or, you know, will maybe we'll separate out into kind of, you know, core features and, you know, looking at how the process is set up right now such that you have you know, you introduce a feature with alpha, you get feedback from the community, and you get, you know, that conversation started, then it moves into beta after you have, you know, a good backing there, then finally GA on that front. So I like that process, but there have been

18:18 a lot of conversations that I've been in both inside of the community and, you know, in in the end user community. And there is, you know, it's the stability is constantly becoming, you know, a a cornerstone to every release that happens for Kubernetes. And being able to you know, there's talks about, like, can we release bug fixes in a different, you know just to release that asynchronously along with this core functionality, you know, will things get broken out into, you know, different types of providers and and you know, like, having core functionality and then other different, you know, veins to kind of

18:53 feed that into the project. I'm curious to see if that happens. And then there was one talk that was given at KubeCon San Diego by Tim Pepper and Steven Augustus, And they talked a lot about, you know, we don't really have distribution of Kubernetes out there right now. We ship the product, but we don't have this distribution with an opinionated, like, almost like an OS. Right? We don't have anything like that. So I think as time goes on, we'll start to see more, in both distributions or just methods of delivery of, you know, what channels are you following

19:24 for these features and kind of keeping track of that. I'm really, really I'd knowing the Kubernetes community, I know they're gonna they're gonna do it the right way. They're gonna talk with the community before anything happens, but I definitely want them tracking that. That's gonna be an interesting one for sure. Right. Keep an eye on that one definitely then. So we've got twelve months. Everyone should be happy. They've got the time, and hopefully, they need to stay on a supported version. And I think with you know, I think I love I don't wanna segue far too much in again. But

19:54 your previous role to HashiCorp, you said, was at Disney. Right? If I remember correctly, you were an SRE? Correct. Yeah. Ah, right. Did you run your own Kubernetes clusters? So mostly, we focused on just, you know, Disney is a big place, so so lots of technologies used. But, really, the team that I worked on, we mostly focused on managed Kubernetes. We we, you know, we're using EKS mostly. Before that, it it it was a that was a long road. We we definitely identified Kubernetes as the way we want it to go because it allowed us a common

20:31 language to use between our development teams and our systems teams. And, you know, it wasn't like every app wasn't the snowflake or this pet type of configuration that we'd set up. So we I liked that we kind of came together as as, you know, collective teams and decided, you know, Kubernetes is our future. We started with the COPS project and, you know, tried spinning our own clusters up that way. But when you do that, the the open source community is great, but it makes it harder for big organizations to, you know, call up and and ask questions and kind of get

21:05 engaged. Again, community is great, but it doesn't give you that, like, support contract, and that's what big enterprises look for. So we went with Rancher for a bit and using that as well as Terraform to stand up. You'll see you'll you'll see why I'm at HashiCorp soon. Love Terraform. But we used Terraform and Rancher to to stand up and and manage our clusters. So we were hosting our own control planes and hosting our own, worker nodes, and that worked really well for a while. But, for for anyone that's that's run a control plane in Kubernetes,

21:41 the, ETCD data store, if you're running that through any abstraction, you might have some problems from time to time just as life happens and and comes at you. We saw some issues there and just being able to kind of, delegate away all of those ETCD and control plane problems and scaling issues to to Amazon was was a very good move for our teams, and so we really enjoyed that. But that's mostly what we would use, and then that would give us the time. You know, it was it was, you know, a little bit behind the current

22:13 releases that were out. And so taking that time to be aware of, you know, like in 01/16, there were a lot of API changes. Taking the time to to work through those and have the time to adjust was really nice. And so, again, you know, that's that's that's why I'm excited about that one year time cycle for for some of the bigger end users and people that aren't able to quickly change everything over just because of the scale and the size that they're working at. Yeah. I'll just echo some of your points there. But every production I teach I've ever

22:46 had of a Kubernetes cluster has either been QTNS or an NCD. So if you can if you're fortunate enough to be able to use one of the managed services, then definitely you should be using that. And and if not, there's cluster API, which maybe we'll we'll talk about later. So back on track now. Right? I'm I'm I'm gonna try not to see if I segue far too much. But the next major theme on the Kubernetes release blog was the storage capacity tracking. What's that? So I'm gonna lump these to these these I'm reading ahead. So there were a lot of really interesting

23:00 Storage capacity tracking, generic ephemeral volumes, & CSI volume health monitoring

23:24 features added, you know, in alpha state in this release. So there is storage capacity tracking, generic ephemeral volumes, and CSA volume health monitoring. All of this really being that, you know, the expectation of Kubernetes clusters and and those use cases were kind of assuming that you had infinite availability to to storage or or just the volumes that you would actually use to store things then. So just overall storage. The thought was that that was infinite. Now we're able to with the CSI in place, container storage interface, we have the ability to be a little bit more intelligent at

24:02 looking at, you know, their like, they I'll just talk about Amazon just because that's where I've gotten all that's where I have a lot of experience. So when you take a look at, you know, EBS volumes versus EFS versus kind of all those storage classes that you have, you you also have, like, the I nodes and those have a really fast local disk storage, the NVIE, I believe. And so you're not able to really leverage that those volumes because, you know, you're either dealing with block storage, you're not really too focused on that local storage on that machine.

24:37 I can't remember which Kubernetes release it was, but I know that that, you know once that was kind of pointed out to the community, they said, hey. Can we just make use of the disk that we actually have hosting on these machines? And so that was added in. So this allows us to be a little bit more direct when it comes to how do we divvy up our, you know, storage classes and how do we how do we actually look at storage. So these these three features have really set the, set us up, for success in the future for,

25:08 identifying, you know, different types of storage classes instead of just, you know, throwing an EBS volume on everything, which is which is quite nice. The the health monitoring is also really good too because then you can actually see when that volume gets mounted to that machine and so on and so forth. So a little bit better reporting on that front too, which is which is quite helpful because, you know, troubleshooting things in the in the, troubleshooting things without a lot of information becomes very difficult. Oh, yeah. Definitely. That's for sure. So those three features then are the stories capacity tracking,

25:41 which exposes metrics about the utilization of the different CSI block devices, etcetera, that we have. I think the feature you mentioned there, was that the local provisioner from one seventeen, I think, that allowed you to actually access the bare metal disk? Like, I remember correctly. Exactly. Exactly. And then the the second storage one there is the generic ephemeral volumes. What what does that allow me to do then from a Kubernetes point of view? So what that allows you to do is kind of have, like, a working disk. So if you have something that you don't want,

26:19 you know, that is ephemeral that you don't need around for a long time, and you don't want to use your that local machine's, you know, volume, at that point in time, you can kind of give it a different disk than the root disk and then, you know, do whatever operations you might need to with that and then kinda throw it away, which is which is kinda nice. So it's a scratch pad, essentially. Oh, okay. So if I have a Kubernetes deployment and I want to leverage some of the CSI implementations, but for ephemeral things, I

26:49 can't even think like, say, want to do some of logs, but not under bare metal test, but on c CSI implementation, then I could request an ephemeral volume from that provider. Exactly. Yeah. And you might have different needs, you know, if you might need something with higher IOPS or something like that so you can get that that quick, you know, burstability, have it be ephemeral, and then and chuck it once you're done with it. Okay. I I I think I get that now. The the last part that we were talking about there was the CSI volume health monitoring. How

27:19 does that hook in then to those last two features, and what what am I getting on top there? So what that'll tell you is is so the, storage capacity tracking, that just kind of lets you know, like, hey. We're like, you're really leveraging, this volume over here. It might not make sense because you have a lot of disk pressure. It might, you know, you might not want to put things over there. The CSI volume health monitoring, what that really allows you to do is just kind of get a sense into the events that are going on.

27:48 So if you have, you know, a ceph cluster, if you have EBS volumes, you can kind of keep track of where they're at and how they're doing. So if you have, you know, a machine go offline, you're gonna wanna know about that. If for for anyone that's used EBS volumes, those are locked to one availability zone, and so that kinda makes things difficult when you want to have, you know, say, a Mongo DB cluster. And if that goes down, you know, and you need to bring it up into another AZ, you you can't. You have to stay

28:18 in that same AZ. You can fix that with EFS, which is multi AZ, but you're you're still gonna wanna have metrics on getting that rescheduled. I've I've had nightmares and real life experiences with EBS volumes and it taking upwards of, like, four, five minutes to rejoin a machine and to reconnect. And so having a little bit more tooling on, you know, where is this volume, how is this you know, what's going on, and having some more error codes to source from make that a little bit easier experience too so that you can kind of tune and

28:49 and set up the cluster exactly the way that you'd want when it comes to volumes. Nice. So one nineteen is a pretty good release then for as far as stateful applications and storage goes in. Oh, absolutely. Do you have any strong opinions on running stateful applications on Kubernetes? I'd so so abs I've I've I've been there. I I feel the pain when it comes to supporting, you know, stateful applications. I've been lucky enough that a lot of the applications I've worked with have been stateless, so not too much of a concern. I've run though, conversely,

29:25 I've run Elasticsearch. I've run MongoDB. I've I I've played around with running Postgres, you know. I'd I'd also, like, pull down RDS or something like that, but definitely work with Postgres and Helm chart. And it's I I really like that you're able to do things like that, but it just is it's, you know, it's it's something that you have to think about is are you okay with that? Because you will run into problems. You absolutely will no matter how much tooling is available to you. Managed services are always easier to a, set up and to b or b, you know,

29:59 have managed because they're managed services. And you have support. You can call people and and work with them on that front. But, you know, it's it's it's an extra layer that you really have to think about. You have to think about if that were to go down, you have to think about your backup process. You have to think about and and do disaster recovery scenarios. So if you're able to do that, I'm all for it. I think it makes a lot of sense. And it's a lot of fun because you don't normally get that level of insight into

30:27 the types of workloads and applications that you support. But at the same point in time, that is a lot of you know, that can be a very big time sync. And if you're, you know, if you're a smaller team, that becomes a little bit harder to do. So, you know, smaller teams or teams that support a lot of applications, I'd definitely lean towards suggesting a managed service. Otherwise, stateful sets. Let's let's go. Let's learn some things because that's that that can be beneficial too. That's its own, that's its own, payment kind of kind of plan.

30:56 Yeah. I think either you've been extremely lucky in your career. I'm a bit of a masochist. But you all the databases you mentioned there are distributed by default. With MongoDB, you know, already handles data working across multiple nodes. Whereas I seem to have been very unlucky in my career and work with databases that did not support that. This makes my life a lot more painful, difficult, and fun. Let's go with fun. I I I I maybe at some point in time, we can we can do some live coding and get, like, a cockroach DB cluster set up and something that's supposed to

31:28 be easy and support SQL native calls. Maybe we can we can find some middle ground between us at some point in time. Oh, yes. Definitely. I'd be up for that session for sure. Because I I mean, I work for a bare metal company, which means there are no managed services. So even though I've never worked with a distributed database previously, I'm now working for a company that doesn't offer managed services. So I maybe it's masochism as my okay. So volumes are cool. I really love that. I think anything that makes stateful applications in Kubernetes easier for people to consume is

32:00 Ingress GA

32:02 fantastic. The the next one on this list that we have shocked me a little bit, and I don't know if I'm just being really naive or not. Angus graduates to general availability. Was that not generally available? Right? That's I'd I'm not kidding you despite being, you know, this true, you know, taxicab confession. Yeah. I I thought the same thing walking into the 01/19 release. I I had had a double take the first time I saw it. I like, wait a minute. What? It was a a Gmail beta, you know, type so we're getting rid of the beta. It's

32:35 like, wait a minute. That was beta type moment for sure. When I I can't remember the and and forgive me. You know, I'm still still booting up with some caffeine here, but I can't remember which Kubernetes enhancement proposal it was. But there was one about kind of reevaluating and looking at that alpha beta GA type classifications because we would have the situation where something would go into it's a chicken and egg problem. When you have something go into alpha, you've got it into Kubernetes. That's fantastic. But most of the time, it's hidden behind the feature flag that you'll have to enable

33:10 or, you know, pass the cubelet and then that will get supported. And with anything alpha, you know, it doesn't really scream production, you know, nor should you in in many cases unless you're just you you're knowingly you've accepted the risks and you're testing out a new feature. Never turn on alpha. That's my my recommendation. I think the any lawyer is going to back that up too. But when so we get into this situation where all of these new alpha features are really cool. Like, look at the ones that we're talking about. They sound fantastic. But if you don't have that beta or

33:44 higher type of certainty that it's actually going to go live at some point in time, you you you you don't have a lot of certainty. And that's what we like when we're working with infrastructure. So the the language was changed there and just kind of that that process because a lot of people would come forward and say, hey. I have this thing in alpha. Can we just move it to beta? Because that will get people to use it. People are, you know, typically not not jumping to use alpha features. So that makes it hard to move to that beta stage.

34:14 Even if there is that level of interest, it can still be pretty difficult. So the language and just kind of that overall process was changed. Again, I can't remember the cap that that that one was around, but that's why the ingress, you know, working as it was, It just needed some final things to push it across the finish line so that we could deem it as stable in GA and then, you know, start start working with the community and more features. So that's that's one I'm pretty excited about. It's one of my personal favorites along with the longer support time is the

34:44 is the Ingress finally moves to GA. Woo. Yeah. I think it's one of those one of those things that because I'm gonna get this wrong. We are just so conditioned, I feel, as users of Kubernetes to v one beta one. That to me, that was GA, like, because we just had all those APIs for so long. And, of course, now we've got v one, but I just always assumed that ingress was one of those one of those objects. It was just available. I mean, every Kubernetes cluster I've had for years now has used ingress. And I never once

35:15 thought for a second it was something that was on beta. So Same. And it works so well. It's it's worked so well. So it's you know, just like Gmail did for eleven years, I think it was, before it went GA. You know? It's Ingress did did everything you needed it to. Yeah. At at least yeah. You're right. It's always work. So it's fine. It's good. And I think yeah. It's it's it's it's nice for people to understand how that new feature thing works from KEP to alpha to beta and to to GA. And I think you can just cover that in

35:46 a complete journey of a KEP, which is nice and cool. And and go visit the GA, you can start using it, I guess, you know, if you weren't going. I think that that they changed the the namespace as well. Was that the release or the previous one? I can't remember now. But I think that's when I first realized that things weren't as I always expect them to be. Cool. So we've got two more we got three more, but there's two more I think we're probably gonna group together now. The next two major themes are structured

36:10 Structured Logging & klog

36:18 logging and the keylog methods. What are those? So structured logging is going to be fantastic because well, it is. It's really fantastic because it allows us now we to to get standard logs out of Kubernetes. We could do things with standard out and standard error, and, you know, that made it made it easy when we had a log aggregation layer in our stack, like Elastic searcher or something else. And then, you know but it did become kind of a pain point when we started indexing those logs and because you were never really you got logs, but they just weren't

36:58 in, you know, a similar format. So that indexing became difficult and then sourcing what data you actually needed to see became difficult. That, you know, mutating those logs downstream before you put them into your login cluster, that that becomes an issue. So finally, you know, the the there's there's a feature now where we actually have everything time stamped and then you have, like, a logical standard out and standard error to actually source from. There's we'll we'll kinda go through a a quick demo on that one in here in a little bit, but you'll just be able

37:30 to see those lines are are now different because they start with that time stamp and it's just in the standard format. So even if you, you know, advise your teams to log in JSON or anything else, you're if you're just doing, you know, line by line type of logging, that's going to be consistent now and you don't need to worry so much about that or making crazy reg regular expressions to deal that and to capture those, you know. I'm sure everyone agrees with me that there's better time better better time spent than on regular expressions.

37:58 Yeah. Apologies to those of you that really enjoy it. I have wasted so much of my life writing regex and grok parsers for Logstash and Telegraph and all these other tools, but just the and it's JSON is such it just makes everything so much easier. It's I think the way that we consume all of these logs and like I said, we're gonna do a demo in a few minutes. People will hopefully just see right away the power that we get, the flexibility we're gonna get from this approach. I I had the spoilers. I was not

38:28 able to get the the the thing I really like, I just haven't been able to replicate on that front with the logging is that output in JSON. But so I'll show you what, you know, just just regular standard out and error looks like. But being able to once you're able to tap into that JSON, this feature, you know, both those features are in alpha now. But being able to work with that and see that get to GA is so exciting because, you know, like like David said, lost years of my life. And there's I I don't

38:54 see any right now, but at least, like, 15 gray hairs that have that will be coming out from a regular expressions one, I'm sure. Yeah. And I I don't wanna make any naive assumptions, but, you know, it's like it's more formality, right, for such a log in to colonize an alpha just because it's a new feature. I don't really see what would hold out from graduating to beta and stable other than formality of possession. Exactly. It's just kinda that that steep time that, you know, just time to work with the community. And then that new KLOG method they've

39:26 given, you know, the community the the project that with that feature, you're able to kind of take your time and then kind of lean everything over into that new logging library such that you can really leverage it, you know, despite despite what's what's coming out of the logs as it is right now. Sweet. Awesome. I'm looking forward to playing with that. Alright. We've got one more thing on our super duper list, which is the Kubelet client CLS certificate rotation. So this one is really cool. Really should have made my list. I don't know why I haven't talked more about it.

40:00 But I think mostly because it's it's been something that we've been used to and that has kind of been done, you know, behind the scenes. And that edit that is the the Kubelet client TLS certificate rotation. So that that did happen in the past, but that was happening within the outside of the cluster mechanism. So with this feature that's gone GA, that finally is brought into the cluster so that as needed, the control plane can kind of roll those client certificates, and it's just not something that you need to think about. So when the Kubelet calls

40:32 in and it needs that new one, it's it's just a lot more seamless. You reduce that level of latency, and, and, you know, everything just works trademark. There you heard it from Taylor himself. Everything is just going to work. Just got back to go. See you later. So so what we've covered I'm just sharing my screen here. It's pretty much the majority of this article, which is the release article that just tells you what's new. At the bottom, we do have a couple of things that we haven't touched on, and some of them are quite important. So

41:07 the first one is is that the set comp integration has now graduated to stable. Assuming things go well, we will take a look at that and how that works, I'm moving away from the old annotation approach to the new actual pod spec level stuff, which is really cool. And I'm not sure if there's anything else here that you wanna quickly run over, like the run multiple schedule and profiles. I mean, that sounds kind of important, but I don't think I fully understand it. Is that something you're comfortable talking about? I think with most of these so I'm

41:40 still honestly, I'm still working through a lot of these myself and just kind of trying to find, you know I'm having a lot of moments after after the release, getting to kind of see these three things worked through, see the tests that, you know, they've been put through in Pro. But it it's still working quite a quite a few of these and and and how they apply to my workloads. But yeah. So none none that I could talk to you right now with the TLBR, but very excited too. Yeah. Well, I mean, all of these are

42:09 links to the caps and issues so people can obviously go if if anything there makes you peaks your interest, then feel free to go click and take a look at it. The other thing we are gonna take a look at today, however, is the immutable secrets and conflict maps, which hopefully we can get that working and show you how cool that is or painful depending on your prerogative. And another one I really like, which is less of a major change, but I think it's been graduating over the last couple of releases, is the North Topology Manager, which actually

42:35 allows you to do Rawkode data center where service reading, which is really, really cool too. And maybe it's something that we will touch on another day. Okay. I think it's time to get hands on. You feeling brave? Always. Alright. So I'm not sure if I'm not sure if that's a good thing or a character flaw, but It's a good thing. It's a good thing. Okay. So we're gonna do the immutable secrets and contact maps, seccomp GA, and structured logging. Do you have a preference of where you would like to start? I can kick it off with yeah. So

43:00 Structured Logging Demo

43:10 if if if you wouldn't mind grabbing the immutable secrets and seccomp, I I I can kick it off kinda easy breezy with some structured logging for sure. Alright. Let's do it. Cool. Let me share my screen and click the right buttons, and then we should be there. Alright. So everyone should see my terminal. Let me get that at a decent size for everyone. So so I have spun up actually, let me let me get rid of this. So I'd say like, I'm gonna delete things with you live. Okay. So the kind project that I talked about a

43:56 little bit earlier is really fantastic because it allows for you to spin up a local Kubernetes cluster. So I'm actually gonna cat out this this configuration. So I was working a little bit earlier with the seccomp, which I've, you know, happily happily hot potatoed for the David for for one of our next demos. But you can see that you're I'm able to specify the 01/19 release here. And then working with some of those those profiles, you can see that there are different arguments and and ways in which you can define this, when you're spinning up your your Kubernetes cluster.

44:33 Please note that up here in the role, that's actually going to be the different machines, you know, the virtual machines here that I'm going to be spinning up and working with. So huge thank you to the the team that manages kind. It's it really makes it easy when you're working with, with local workloads, in addition to, you know, things like mini cube and Docker for Mac, Windows, Linux, and spinning up those Kubernetes cluster. But the benefit here is that it makes it very easy to go ahead and and grab the the latest version there. So so I've deleted that

45:04 out. I'm gonna do kind, and let's create a cluster. Also, like the tooling just because it's, you know, all the command line items just make a lot of sense. I also do very much like the emojis. You can see here is an image. You got machines here with the with the nodes, the package. I really like that. As you configuration, you gotta scroll. Control plane, you have game controller. Fantastic. But I think I think I figured out most of these, but I'm not a % sure. But we'll see we'll see how well my my skills of relation are as we

45:39 go through this. But the good thing about this too is that it doesn't really take that much time. I think it's the control plane step, and when it joins the worker nodes to that control plane, that's, like, when you're waiting for twenty, thirty seconds. It's it's typically not that long. So, hopefully, you don't have to eat my shoe, after after, saying that. But, so, yeah, you have, CNI, storage class. This is working on. It takes just a little bit of time. But then you're and then it writes that cube config, to your local file system,

46:12 and you don't need to worry about that either. So once this once this is done, I'll kinda just show you that we have a a working cluster, and then I'm just gonna tail the logs to cube d or, core DNS to to show you what those standard logs look like. Cool. So we have a cluster. Let me so I a couple things for those of you following along at home. I've aliased the cube control just to k. So if you see me typing k, I'll I might bounce back and forth depending on the commands, but

46:43 that's that's my alias and setup. I also use two two tools. One is cube context and cube namespace, and that just makes it a little bit easier for me to switch between clusters because, you know, I'll typically manage quite a few. So if we go ahead and we take a look at the nodes, kind switches over that context to the cluster that you just spun up so you don't need to switch to it, which is kinda nice. So we can see that we have four machines here. One is the control plane, and then we have these worker machines here.

47:15 If I jump into the oops. I jump into cube system here, make it a little bit easier for me to get these pods and see what's up. So we can see we have our whole control plane here and say, I want to go ahead and take a look at this I don't want to define that. Get out of I want to take a look at the logs here and follow along. We can see that, you know, here are some of the errors. That's not so hot, but, you know, I'm I'm assuming everything's working working now. We can see that we have

47:48 time stamps, and then we have these, you know, just the regular body of the message. Beforehand, you know, you would get the separate pieces of information and data, but if we let's take a look at another pod. Let's take a look at let's take a look at this one. But you can see here, you know, now everything is standard in the fact that we can always rely on that timestamp being first and then getting that that, you know, downstream data in the body of that message later. Again, I apologize. I was not able to get the, the JSON view,

48:22 up for for this. I think there's a couple, flags I needed to double check and take a look at my setup. But you are able to export this as JSON if you specify the right commands, and that is going to be that's gonna obviously be a lot easier to kind of look through and then filter on as you're try trying to tail logs and things like that. So really excited to see how this evolves over time and as we're able to, you know, filter on, like, I only want to see error logs and, you know, the biggest one being that it's

48:51 so difficult to kind of link together if you have a deployment that's three pods to kind of tile those logs together so that you get a comprehensive view as to what's going on without having having to stand up your own login cluster. So that's that's structured logging. Nice. And because I am feeling brave and I have been working on the JSON stuff, I'm going to get that go. So and I'll I'm using many cube instead of cane. Oh, and I I will I'm not feeling good because I can't even spell. I will feel brave in the lead

49:28 night. Yes. And so the way this would work, I feel particularly brave. What up? Let's jump over here. So I have a I guess it's maybe too brave, but I've been working on a branch of many cubes to make this a bit easier. So let's let's look at what this would work. Let's let's look at how this would work normally. So is that bigger? Hold on. There we go. So many kubelers to pass dash dash extra config, which allows you to specify extra parameters that can go to each of the Kubernetes components. But what I could do here is I

50:09 can say that I wanna pass something to the kubelet, and the key that I wanna modify is gonna be the logging format. And I can specify that JSON like so. Unfortunately, I I could do that for every component. Requires doing this and saying, hey. Let's do the scheduler too. And then that have to do what else? That would be the API server. That would be the proxy. That would be core DNS. Like, that lane would get super painful. And I'm a very lazy developer. So what I started doing with this ability where we could just specify dash dash login format

50:42 equals JSON, and you'll see that I've added some crappy characters to the end because I actually don't know how to write tests for many cubes. I wanted to make sure it didn't work as well, and that was the way that I did it. So you can see here that it just says, hey. This is not a supported login format. So let's just do that again with JSON. Many cubes should take I mean, I look bad, but I'm I'm gonna assume it's gonna be okay. It should just take oh, it's not okay. It's there to delete.

51:15 Maybe I had one cluster and a half set up state there. If that doesn't work, was gonna be able to wait. But it should just take roughly about one minute, maybe a little bit less, get us a working note. And what we'll see following on but we can already see the parameters being added now to the different components, so schedule, the kubelet, the controller manager, and the API server. And we should be able to pull out those logs, much missing by the tail of that, only ours will be in adjacent representation. The cool thing about that is it's gonna

51:44 be really easy for that to be parsed by long stash, elastic search, telegraph, vector, whatever tooling you have in your setup. Cool. It didn't crash this thing, so I'm happy. So if I run, I do not have the fancy tool like you do, cube namespace. But I could do key dash n cube system, and I'm gonna pull out the API server logs. And when I do that, it's not JSON. So that's my I knew it was different. That just means that my fork didn't work. So let's just quickly do this the all the way there, and I'll just do the

52:29 API server. That was really unfortunate. I was so confident. Was there. I swear it's a it's as soon as you do a live demo, that's a there's something that happens. You lose your magical powers. Don't know what it is, but, yeah, being alone, you know, and just that solitude, that focus really really brings it out of the demo. Yeah. I don't know what I did wrong there because I have tested this. And I've told her on the API server, and I did enable it on the API server. Oh, well, it doesn't matter. So what I did there is API server. Now

53:04 if this doesn't work again, that means the API server just ignored it. Oh, it did. Okay. So you can see here, we now actually have JSON for every single line. And and if I just pick up on a keynote page for j q we have JSON. It's easy to parse. I can use other tools with it. Much good. That's it. A structured login is not one of those things that has a substantial amount of change. I think most of the changes came from you reusing the different key log methods within the instrumentation. But nice. Good.

53:44 I'm not gonna ask script anymore. It's just I'm not brave. I'm not not at half past six on a Monday. Okay. So structured logging. Very cool. We like it. We're looking forward to see tools, take advantage of them, and store logs with good debugging stuff. What did we say was next? What is next? I think it was the I'm not sure if you wanna save the oh, this this one should be pretty easy. Immutable secrets and config maps. Yes. Right. Okay. So at pop open Code and let's deploy I'm not gonna do We're doing a config

53:45 Immutable ConfigMap Demo

54:28 map. And we'll call this RTFM, and we can just have any arbitrary value like so. I guess we should just show and maybe talk about how this works currently without making any modification because I don't think people really understand what happens and why this is a bit of a pain point. So if we do apply this, then we can do a two p I'm in bash, and then I'm not getting my Elliot. Config map r t f m. If I modify this value in fact, I didn't show the value. You can see here, key is 1545.

55:17 Now if I change this and reapply it, I run git, we have this updated value. That could be considered good or bad depending on what you're doing with your Kubernetes cluster. So I'll speak from my own experience, and then maybe you wanna chip in with how you feel. I don't think of I I I don't know if we both have the same opinion or not. But there's this weird thing that everyone in their Kubernetes journey has to deal with at some point, and that's when you make a modification to a contact map. You have running pods inside

55:49 of your cluster, and there's actually no notification or event that tells that pod that it needs to restart because the configuration changed. And I see it's so common that there are a lot of helm charts now that will actually put an annotation on the pod spec with the SHA of the config to force that thing change. But the other school bus thought this is the config maps shouldn't change anyway and that you should always deploy a new config map and then we have a new deployment that consumes it or an updated deployment that consumes it. Is that kind of a name of

56:22 your experience as well? Oh, absolutely. It's it's it's when you have something especially when you follow, like, you know, a GitOps type methodology where what you see in that repository or, like, when you set up a cluster and app something, you you have to have in my opinion, you have to have something that is static so that you can reproduce it. If you have a config map that's mutating or changing, secrets that are mutating or changing, you don't get that level of certainty. And so that makes it kind of difficult to troubleshoot, to reproduce issues that happen,

56:57 things of that nature. And this also, say this were, you know, a Bitcoin wallet, and, and I found out what David's was, I could quickly change that. And then, you know, we could we could use, his money instead of mine, and you you wouldn't want that, would you? No. So I guess yeah. And just to kind of really cement why this was a problem is that different if some of your pods in your deployment, if you get 20 pods and some of them restart, they could be consuming an older version of the config. Some could be modifying a new version. And

57:29 if you've actually applied three, four, or five changes to your contract map for whatever crazy thing you're doing, all of your pods could, in theory, be running a different configuration. Really weird. We had a comment. I think someone noticed the error with my, by the way. It was that I possibly wasn't actually using my locally compelled one, but I'm not gonna back to it. I'll hopefully get that pull request merged and everyone else can seal that. So one nineteen gives us this new thing, and it should I'm gonna say it should be as simple as adding a new key to the config

58:05 map that says this is immutable. K. So we now have that configured. If I just pull it back down you know, I've never actually modified an mutable conflict map to be immutable. And so let's let's see if this breaks or not. But you can see here, we have the value immutable true, and the value is 678. Now if I've done this correctly and I have to say that this should now be a b 1 2 3, when I apply this, we should get another message that says you cannot modify this conflict. It worked. Yes. So it's it's one of those really small

58:50 changes. The API surface is really small, what mutable true, and it's done. But I think the quality of life of people that are consuming and using Kubernetes and the deterministic nature of now that a deployment of a conflict map will always be in sync is really, really important to people. At least, I I think that would be a very welcomed addition to the API for sure. Absolutely agree. And and secrets are just as easy too. All you need to add is that immutable true flag, and you're good to go. But definitely agree with David on that front. It's e even

59:22 figuring out, you know, what what is tech what is a secret in Kubernetes, what is, you know, worthy of putting into ConfigMap. And at the end of the day, that's typically, like, anything you're okay with seeing in plain text is usually a pretty good smell smell test, you know, before you actually put it in there as well. So, you know, credit card's not a good config map entry to put in there. No. Alright. We got two follow ups in the chat there. So the first one being another big issue is if you have a config map managed in a separate repository, it's

59:55 a pain because of the Kubernetes native event. Yeah. So that's just reaffirming that deployment situation. And I think the use case here is if you've got, like, a platform GitOps repository that provides those environment information and then your applications are deployed separately, there's no way to connect the dots between that conflict map changing and what's in the cluster. Yeah. It's a a huge problem. Immutable conflict maps, just remove that because you always have to deploy a new one and update the surrounding thing. So, yeah, I think it's cool. The other question was, can we delete

1:00:27 can we delete an immutable config map? Yes. He says confidently. I did do this earlier, so it's gone. And there's nothing there to stop you deleting that. There's no protections from there's there's nothing there to stop you deleting something that's gonna be consumed by it. So, yeah, if that's a challenge, you're gonna end up with pods and a fail to restart container create loop if you delete one that is needed. But at least you can be sure the content's all very good. Yes. Absolutely. There there was one edit too because even though, like, we had said before, David

1:01:06 pointed out the fact that, yeah, when you change a config map, that doesn't automatically restart all of your pods, you know, unless you have some, you know, some some out of band tooling that that handles that. But Kubernetes native does not give you that that type of experience. However, Kubernetes keeps track of, you know, hey. This config map was updated. Right? Because it has to store that state. And so when you market as immutable, Kubernetes gets that very, you know and and I I believe in most cases, very small performance gain where it doesn't keep checking to

1:01:36 see if that config map has been updated because it knows it can't be. So there's there's also that, like, hidden benefit as well. You know, at the end of the day, I'm not sure how much how much performance it's going to net you, but just something that's cool. Might be a good trivia question or something to pay attention to with the all the cloud trivia that that that we've seen this. Yeah. So that that's a really actually good point because what you've just said has now put in a bit of ambiguity into the problem domain

1:02:02 that I was mentioning. So I think we should quick let's just clear this up. Now the contact maps are and I if I get this wrong, please correct me. When they're exposed as environment variables or fails, well, fails and environment variables are updated within the pod. The challenge is the process running normally doesn't know to take advantage of the reload, which why you end up with multiple processes consuming old information. It's not that Kubernetes doesn't update the file or the environment there. That's right. Yeah. Exactly. Exact. Cool. And I think what we should just do now is show off one other thing

1:02:36 because I'm sure there'll be a question on it later. But what if I deploy this as immutable and want to make it mutable? I'm just gonna say unimmutable, and I would have just left it like that. So we now have an immutable conflict map. And if I remember correctly, you cannot then make it mutable. This will not work. Okay. So if you make a conflict map immutable, there is no undoing it to do it online or hot updates. You always have to create a new config map. But I can't do it, so let's get rid of it.

1:03:12 That reminds me of Slack channels, how you can make it private, but you can't then turn a private channel back into a public one, which also makes sense. Yeah. I guess so. Yeah. I don't think there's anything else to cover on the immutability here. I think we've pretty much covered every potential edge case and question. So that means we have to move on to the difficult bit. So we're gonna talk about the let's close that. The promotion of the SET Comp API. Was the journal available? Yeah. I think it was. Yes. So I cheated. I

1:03:30 Seccomp Demo

1:03:57 have downloaded a seccomp profile. I am not brave enough to try and put that together during a livestream. And this is for NGINX. What is really cool is if you deploy the set comp operator, this profile is actually provided for you. I'm not going to show the set pump operator today. There's a stream on that last week. Feel free to check it out. What we're gonna look at today is just the raw Kubernetes aspect of it working with the API directly. So what does that mean we need? We need a deployment. I see that's Versus code extension where you

1:04:36 never have to actually type the YAML. I don't even know if I could type the deployment YAML anymore. Wow. So I'm just noticing I need to add that and save myself hours of time. Thank you, Dave. Yeah. So that's the the resources. I don't need the port, but I'm just because it'll annoy me, I'll add the port. Now what happens here is I can deploy this engine x resource. It's like any normal resource. It's a minute. Okay. Nothing wrong. Oh, I'm applying the JSON because I called it deployment. There we go. Which means I can now do

1:05:20 quick control, get parts. Let's just get that second. It's gonna be pulling down the NGINX image. Running on a lot. Of course, it's running. And now I could do exec NGINX, not my real shell, so I can't get all complete. And let's see then the next container, and I can drop back in. Nothing exciting there. All standard stuff. However, there is a certain attack vector without specifying a set comp profile, and that is if a malicious actor, sometimes when they just to get within the process, can they execute commands that we do not want them to execute? Like, me exit me

1:06:05 getting inside of a a pod seems harmless, but, you know, the ability to get inside of that namespace could be potentially dangerous. Right? So we can use the setcom profile here, which lets all of the syscalls that NGINX is expected to make and restricts it. So what we have to do is copy this, and then if I do many qb s s h so you have to be on each of the hosts or the nodes, and you have to put in a file. I'm gonna have to on RIP, and we're gonna add a directory to the

1:06:46 kubelet configuration. So that is where all the kubelet conflict lives. We can create a new deck, create a new directory called seccomp profiles, and need to do an app update and install them because I'm not gonna have it. I just realized. Magic stuff happens. I'm my Internet is not too bad. And we can go ahead and set seccomp profiles. Now from here, I can create nginx dot JSON, paste that, and leave it alone. So this would obviously be part of your configuration management setup. You just have Terraform, write this file, or SaltStack, or Puppet, whatever

1:07:33 you're using. Put all of your profiles into a single location on each of your workers so they can be considered by the pods. Or you can use the second operator, which does all that work. Anyway, now we wanna restrict the Cisco's engine x can make. And to do that, these changes to the API leverage the security contact API. So this already exists, allows you to do things like what? Force the UID, the GID. I think there's a few other things. But now we can specify second profile, and we have a type. So this is where I'm gonna start getting into

1:08:11 territory that I'm not entirely comfortable with, but I'll try and remember it. There are local host types and container runtime type. Run time I think it's runtime default. The documentation is gonna have that covered. But this is a profile that ships with your CRI implementation, which tries to remove some of the more dangerous syscalls, like, privilege escalation and such, but it's not really specific to applications you're gonna run. For that, we have to tell it to use a local host, which is gonna use the profiles in the directory we just provisioned, and then we can tell it which profile

1:08:52 we want to use. Assuming I've not messed this up, which is extremely likely, I should be able to do profiles engine x dot JSON, and that path is just profiles here, the directory that I created, and the engine x dot JSON here. We can apply that. So I drop out my mini cube SSH. It will do a cube control apply and redo the deployment. No. We won't. Oh, drop out of Brit, drop out of Minikube. Now I can do it. Okay. But if I've not messed this up, I should be able to run get pods and

1:09:39 see the pods running. Good. And we can confirm that it has and what we expect. So describe pods, pod name, and then we'll describe for account. You can see here, we are running this NGINX profile. And what that means is I'm gonna use a pod name again. As I try and exec and save at this, fails. Good. So what happened there is this test call get p process group, I think that is the syscall, failed because our set pump profile doesn't allow engine x to run that syscall, and I can no longer get in a container.

1:10:38 The good thing about that is if someone were able to exploit an NGINX vulnerability or CVE to get inside of that container, the chances of them being able to execute any commands that would potentially allow them to get back then to the host should hopefully be not done by it. Hopefully. I think that's it. Like, the API surface is nice and small. Unfortunately, writing the second profile is really difficult. So the Google will try and find ones that other people have written. And but the application from a Kubernetes point of view could not be simpler.

1:11:12 So, you know, a couple of lines on the security context and you're good to go. Is there anything you wanna add to that? No. I think just very much agreeing with you that that that writing a seccomp profile is is, you know, at least right now, difficult. I definitely want to learn more on that front as well. I think I'd feel more comfortable writing an autobiography before a seccomp profile as it stands today. But no. Absolutely. It's one nineteen's fun. Go check it out. If you have any questions about features, you know, please feel free to reach

1:11:45 out. If there's any, you know, demos or or, you know, you feel like there's a blog entry that's needed or tutorials, things like that, hit us up. You know? And it's it's kinda fun to see what people are using, to talk through those work streams and those workflows, and just get a sense as as to what everybody's interested in. Let's let's have a conversation. Awesome. Well, thank you so much for joining me today. This is is really good to just kinda sit down and go through some of the new features, you know, understand how they work and and dig in some a

1:12:15 little bit more. So thank you. I'm good. Thanks for having me. No. This was a blast. Hope you all have a wonderful rest of your Monday. And, yeah, thanks thanks for hanging out with us. Alright. Thanks again, Taylor. I'll speak to you soon. Bye, all. Thank you, Nikki.