1:13 Introduction and Guest

1:13 Hello and welcome to today's episode of Rawkode live. I am your host Rawkode. So as you can see I'm a floaty head today. Steve is doing the same. We're gonna have a little bit of fun with some infrastructure as code. Now before we get started and I introduce Steve, I would encourage you please subscribe and click the bell on YouTube so you get notifications of more awesome episodes and also you'll notice that on most of the scenes we're using today there as a discuss discord embed which discord have currently got broken and you'll see your name as object object but I'm

1:47 hoping they fix soon. So please feel free to join us in the discord chat. And now I would like to introduce Steve. Steve Wade, an independent consultant and active member of Kubernetes and get up communities. Hey, man. How's it going? Hey. I'm good. How are you? Yeah. I'm very well. I'm looking forward to what we're gonna be kicking the tires on today, having a bit of fun and and generally just chatting gets up in automation. Yes. It's it's gonna be good. Right? We've we had that question that someone was posting on Twitter, and now we're gonna give a

2:19 demo of a possible way to fix it. So everyone everyone's the winner, and you're showing me some new technologies that I've never used. So Yeah. Mean, I guess if I was smart, would probably have had that tweet ready in advanced and been able to show on the screen but I am not. So we'll talk about the prompt space as we tackle them together. Why don't we give a quick summary of what we wanna do? Why don't you go first and then I'll try and add in anything else that I think is interesting. Sure. So the the plan here is to

2:40 What do we want to do?

2:51 essentially start from scratch and look at how we can bootstrap infrastructure and also enable deployments straight out of the gate for your workloads that are running within Kubernetes. So, essentially, we're we're starting with that bootstrapping problem. So that initial cluster up plus how we hook that into a number of other repositories in in this scenario, it's gonna be one so that we can start to bootstrap workloads. So this is a common problem that, you know, everyone that's creating Kubernetes clusters may or may not have, but in a kind of GitOps paradigm where you wanna drive everything from

3:25 Git, having that initial bootstrapping from Git is really the way that we wanna go. That's gonna be the gold standard. Nice. Perfect. Alright. Well, before we get started, let's tackle a couple of comments. So our friend Duffy has left us a message. Set the bar high there mate but we do appreciate the support. Yeah, no question now. Simon, hey man, how's it going? Alright. So what we're gonna do is we're trying, we're gonna try and split this into like two stages basically. Mean, you've already kind of covered us but I'm gonna summarize it for my

4:00 Outline: Pulumi for Infra, Flux for Workloads

4:04 own sanity here but what we we're gonna use Pulumi today. Now we're doing a skill swap here essentially right like I don't think you've ever used Pulumi before you're not familiar with it So I'm gonna try and impart my Pulumi knowledge on you and we're gonna do that to get the cluster up and running and I'm gonna cheat and I'm only gonna deploy a cane cluster and the interest of speed today. I mean, we don't wanna be spelling up that we don't want to be spinning up multiple nodes. We just really, that's not the interesting part, the

4:34 code, the automation, the process is the interesting part and then we're gonna flip it over and then you're gonna be driving forward and showing us how to adopt flux and get ups and apply all of that stuff to our cluster and we should have a trade tight feedback loop by the end of this and that we can just push changes to the get repository and automatically see that all deployed in our cluster. That's the plan. Right. Good. I like it where we're on the same page. So let's just get started then. There's no point in messing around. So I

5:05 Starting the Pulumi Demo

5:08 have fought a repository that you sent me earlier. This is a get ups example that I think you've got that uses customize. Is there if anyone wants to take a look at it, play with it, etcetera. But first we need to get some machines running. We have a shared Versus code terminal. I can type, Steve can type too hopefully. Say hello. There we go. Guest user. Wow, you're not even logged in. Okay. And we have an empty perimeter entry. We have nothing here whatsoever and we have that same repository that I have fought here. So step one,

5:51 as we're gonna create a new Pulumi project. I'm gonna walk through the process, explain the decisions that I'm making as I go. Just prod me with questions. Anything that I've not explained or whatever, you know, fire to me. And of course, if you're watching at home, please use either the discord chat or the YouTube comments to throw your questions and we'll do our best to tackle them as we go. And it would probably help if I made the discord chat visible. There we go. Hey, Duffy's joined us there too. Nice. Okay. Let's see if I can work out this

6:22 Pulumi thing. So if I remember correctly, we could do a Pulumi new and it's got like this nice little CLI based wizard that runs a certain new project. Now one of the questions I always get from people is why am I using Pulumi and why am I not using terraform? Really I should have queued up for you to ask me that right but doesn't matter. That my go to question. Okay so there are a couple of reasons I really like Pulumi, one of them is I get to use my programming language of choice and I think

6:32 Why Pulumi? (vs Terraform)

6:54 that is super valuable and you know as much as we, we want our infrastructure provision and to be a simple API where I just say give me a machine as it generally we're doing a little bit more with that and having a programming language available for loops and filters and maps etcetera gets really handy. The other really secret amazing just not secret, the other really super awesome thing about using the real programming language is that I can then take common bits of logic through them and encapsulate them in a function and then we use it. I do that all the

7:27 time and you know I'm just gonna keep throwing more reasons out because they're popping into my head but Pulumi also has the ability to like wrap terraform providers as well so you don't really lose any functionality by using Pulumi. If there's a terraform provider out there, you can generate the Pulumi wrapper that allows you to consume it as well, which is great. And there's that wonderful discord bug where object objects is now talking. So So is is there a is there a kind of command that will allow you to be able to take a existing provider and and wrap it?

7:57 Is that built into the Pulumi CLI? It's not built into the Pulumi CLI unfortunately. There is let me just pull it up. Right? This is the wonderful thing about the Internet. Everything is a few clicks away. So there is a tf something. Oh yeah, so there's a few projects here that's probably worth mentioning. There's a tf to Pulumi so you can actually generate Pulumi code from Terraform HCL then there's the provider boilerplate so this is the repository that allows you you'd basically just fork this and then tell it the Terraform provider and then do some Go code and it

8:35 generates all the bits and pieces that you need and it kind of just works. I'm sure I'm oversimplifying it, it does take a little bit of effort but it does pay back handsomely. Alright, so today we are gonna do I've forgotten who I work for. Sorry. Equinix metal. So yeah Equinix metal is in this weird stage where the providers always called packet and we've not renamed everything yet to Equinix metal because obviously that's really tough to rename packages that are being openly available for so long. So what I'm actually gonna do is just create a very generic TypeScript one and

8:58 Setting up Pulumi Equinix Metal Provider

9:14 then I'll I'll drop in the Equinix metal provider. Okay, so project name we'll call this get ops not high ops description I'm not fussed about and I realized I've made a mistake already. Pulumi login local. Let me explain this too. So Pulumi and Terraform both have state files, Hopefully I don't need to kind of explain that to anyone like when you create resources it has some state that understands what resources were created, what condition they're in and how to reconcile them. Pulumi provides a managed service much like HashiCorp do where they can store the state for

9:52 you and handle locking and you know our back and stuff like that for today I'm not gonna use that so that's why I'm just doing a login local and I'm gonna then just recreate my project and then now I've got a non empty thing so off to a crack and start. This is how every project starts. Right? You know, the initial bootstrapping, that's what we're talking about. Exactly. So text script, get ops. Yes. My stack name, we're just gonna go straight into production and because another really cool thing about Pulumi is it enables get ops by default and that it

10:32 Pulumi Secrets Management

10:34 already ships web secret support. So when I provide a password here, this is actually gonna be used because I'm in Pulumi local mode, gonna be used to encrypt all the secrets that we store inside of the repository. If I was just in Pulumi cloud, sorry they call it Pulumi teams or if I had another cloud provider like AWS, GCP or Azure available or even vault, I can use those as KMS instead of a password here but those all take time to configure and we're doing this the hacky way today. So password something probably shouldn't say that out.

11:17 Anyway. Unless it's actually the word something. No it was not that it was password and then wasn't one two three, wasn't four five six or seven eight nine you know, I'm not that much of a cowboy but it's close. So let's see what we're dealing with now. We have a Pulumi. Yaml that describes our Pulumi project. We have a stack file, so Pulumi has stacks much like Terraform, this is our production stack file, this is where we can store a configuration and secrets for that stack. We also have an index dot TS which was generated for its upfront which just pulls

11:34 Exploring Pulumi Project Structure

11:56 on the Pulumi module, now I can begin to create Pulumi resources. So is this stack in this regards kind of like a module in Terraform or is it a is it an encapsulation of many modules? Can you say that again? So in in Pulumi world is a stack. Does a stack equal a module in Terraform, or does it equal the execution of one or more modules that kind of come together before create your infrastructure or whatever that thing is that you're trying to configure? Yeah. So a stacks like an environment dev station production, it can consume multiple modules, multiple

12:11 Pulumi Stacks vs Terraform Workspaces

12:35 Terraform modules, multiple multiple Pulumi modules, etcetera. And Paul's corrected me. Paul works. That's the one I was looking for. Yeah. Terraform has work species. I'm just so embedded in the Pulumi way of life now that I don't even remember how to Terraform to be fair. Anyway, we also have a couple more. So we got a hello heritage. Don't know why Duffy is in RimRath. I don't know if that's important but Let's get back to it. See if I can remember how to Pulumi now. Now the first thing and I'm glad Paul's watching because I do this in every stack and it's

13:11 so annoying. I don't want my code to be an index. Ts in my root project right? I know user go developers but it's messy so I always move it into a source folder and update the Pulumi. Yaml to tell it that the main file lives in there and it's such this is one of those wee small things that always create something in every new project. That's my mild rant done. So now we're in position, in fact we could probably just run pulumi up with nothing good someone's at my door, my dog is going wild but I'm sure someone else will

13:47 handle it. And we can let go and this is just creating a stack, there's no resources and nothing for it to actually do here. It's all good. So let's make it do something. What we wanted to know is add a provider. So when we created a project, all we selected was TypeScript. We never told any providers that we actually want access to. So we can come to type metal and this is the Equinix metal polymer provider. And I am just going to copy. I don't even know if I have yarn or NPM installed and I don't wanna create a next dot

13:56 Adding Provider Dependencies & API Token

14:39 shell so we're just gonna hope. Yeah, NPM looks green there so I think we're okay. So I'm gonna add the provider. The next thing I need to do is get my off token which is also just in the docs here and I'm not really doing anything magic here but we have to provide us so that knows how to speak with the API. Now I'll show adding the secret to our stack through the galaxy fashion but obviously I'm not going to show you my API token. So I'm gonna jump over to my other monitor and see a Pulumi

15:16 token. Alright. So because we got secrets management here we can do Pulumi config set secret Equinix metal off token and that should just match what we have here. Yep. I don't think I've messed that up. We can hit return. I should be able to paste this in and we need the password so that gets encrypted correctly. So does Pulumi not preset your password that you set the first time when we started the local? Do you have to add it every time? So I can export it to my shell Yeah. Which would avoid me typing it every

15:59 time but yeah, it's not a big deal. Like, I don't mind typing it. It's it's not too bad. So what does that look like in our stack file? Well, we can open this up and we can see here we have some secure value, which is our encrypted Equinix metal off token. If we look at our package dot JSON, we have our Equinix metal provider. So what changes read me TS config should be alright. So now we're gonna write some preliminary code, let's actually spin up a device. So this is TypeScript, the reason I like using this is it's

16:31 Writing Pulumi Code: Creating a Device

16:39 really easy to kinda get strict types that I can pass around and functions and you know get some sort of safety guarantees but at the same time working with JSON is not as painful as as we go. We're gonna create a new I said how day? I I really like go for some things right but juggling JSON or any kind of structured config is not as strong so I hear you. Yeah. I've forgotten how to code. Oh yeah. We need to import our provider. So let's get import as metal from, we pull in our provider.

17:17 I can now do a new metal device see and again nice auto complete here. I've never really had successful auto complete on Golang to be fair but that's a rant for another day. So this is our get ops machine and that takes a device args, I have no idea what device args is yet, so I'm just gonna pop open an empty JSON object and then trust the IO complete to lead me through this. So bell and cycle will be Bell and Cycle and all the types are here as well. This is another reason I really like the TypeScript SDK.

17:56 It's just all there like I'm, I'm not actually coding, I'm really just all completing my way through this. Really suck if my boss found out I can't actually code so I'm just gonna speed up a little bit because it's not particularly exciting. Let's see. Throw in all these required fields. C three medium should be alright hopefully. Project ID we don't have yet, so we can talk about config. What else do I need? User data. Alright. So that's pretty much it. I'm gonna tweak the last few things as I talk about them. So first of all we've got this thing called

18:30 Using Pulumi Configuration

18:48 a project ID. So what we actually want here is maybe something that we would want to configure at the stack level. Excuse me. Like we may want to deploy production to a different Equinix metal project than we would our staging environment etcetera. So let me pull up the Equinix metal console. Hopefully it logged me into the right one. Yep. There's my Rawkode project. I don't know if there's anything here. No, we're good. Okay. And I'm gonna copy this project ID. Now because I want this to be configurable, I'm gonna do a config set. It's not a secret so I don't need

19:30 to pass in the secret thing here. I'm just gonna call this project ID. And all this does is drop it in here it uses our Pulumi project get ops as a prefix to you know providers use their own key here and one of the things I also really like about Pulumi and I won't spend, don't want to shell it too much right but it's really cool is that these take arbitrary objects as well like you know I could just do like metal, I could have my project ID here, I could have my instant size, I could really make all of this configurable

20:12 and then just load this object as a single type inside of Pulumi but don't need to do that right So we've got a question from Paul, he's asking why is the person from Pulumi asking me a question about why I have Pulumi? I guess it's good to understand why users do things. Okay so why set the value rather than making a request to look up the project? Yeah. That's a really good point. The reason I don't do it is what what Paul is suggesting is that I could do project name and just put Rawkode and then actually do

20:48 a lookup using the SDK to get the ID rather than the store an ID to me they're much the same and so I just avoid a network quest. That's it. No real brain or reason to be fair. Okay, so now we need to get our project ID here, so what we need to do is first get a config from the Pulumi object and then we can define a type, so we'll call this my config and it's gonna have a project ID which is a string and no, I didn't use an object. Sorry. That's what I normally do. I'm just gonna do

21:30 project ID. This is the pressure, that's what it is and that's just a required field. So I can say my project ID is what I want, nice and simple. You'll see if we take a look at the config thing here, you know there's gets for things that are optional, there's require billions like if I want to do some sort of type enforcement, there's require object if I am doing something a bit more advanced and then there's secret helpers too. So So can can you do can you do a get and it works out in the background if it's a secret

21:59 or not not a secret? Or do you have to explicitly pass the get command if like do you have to pass get secret if you're hoping to return a secret or you're looking up a secret or can you just use the generic get? No. I I, you know, I've never tried it but I don't think it would work. Maybe if Paul's watching, Paul's still watching he can answer. Okay, so we have our project ID here now, project ID now so we can just drop that in. We want a host name so I'm just gonna call this get ops, it's not very

22:30 interesting and then the way that we provision the machine is gonna be through user data here. So we're, we're just gonna hack this today because it's not the most exciting thing. So I'm literally just gonna do run a script grab my favorite helper of all time. Get Docker. Io. I preferred it when they gave you the pipe to shell version but whatever. And then we will run get docker. So that will install docker. We then need kind. Docker kind Kubernetes. I'm assuming I just need to grab the release from the GitHub page actually unless it's got something here.

22:40 User Data Script for Machine Setup

23:23 So Paul was saying that you can use git but ideally you use git secret and they're gonna change that now. Nice. I'll pop that on the screen. Okay. So we can on Linux copy this and we're gonna need to keep control. Yep. And this is of course not the best way to be doing any of this. This is just to keep things moving at a sensible pace this afternoon so you can imagine really what this would be doing is like because I'm yes. Thank you William. There we go. I've got everybody's telling me now so William

24:19 you were first you you can win bragging rates and then no results so quickly in as well. So yeah, trim off the script we run it. I'm gonna pull down control. I'm not sure where that's putting it. Probably just the local directory. So we'll probably want to move that. And then kind which does this, I never even noticed that path that was wild. Okay. Was about to tell you we need to change that. Yeah. Now there's something I want to kind of explain here before we go any further like I'm just using user data to install

24:56 Docker and install Kube control and install Kine so that we can get across a cluster and then I'm gonna run Kine cluster create and just that is not what we would normally be doing in a production environment and normally you'd be using a managed Kubernetes or a cluster API or anything else that provides better ways to do this but again those are not the focus for today. So kind cluster. Done. Are we are we confident that's gonna work? Do we want a couple of of actual nodes in our kind cluster or do we wanna go with just a

25:34 single node? We can just go with one I think, like we're not deploying anything to Wales to it, right? So Let's do it. Alright. So I'm happy with that, you can see we've got Pulumi, we're using TypeScript. I'll just kinda show off again my favorite feature is that this could be in a library that I consume for NPM. This could be a create kind cluster thing that as a function that returns this object and I could then use create kind cluster and I think that just on itself is a really really good reason for using Pulumi

26:08 Pulumi Component Abstractions

26:17 over terraform. You're you're selling it to me now. Yeah, it's just those weird things really. Oh yeah, I can take these four different resources, group them together and provide an abstraction that's just create the thing and nobody else has to worry about it. I can end up in types to pass in as parameters to tweak the configuration and of course you can go down a rabbit hole and start getting the abstractions wrong and too much configuration and making it a bit of a spaghetti monster but still very cool. Anyway, sorry I digress. So I'm gonna run a Pulumi up and

26:47 Running Pulumi Preview & Up

26:47 what we're gonna see here is it's gonna try and work out what it wants to create and then hopefully we get a plan that we're happy with and we can approve it. So Pulumi app is can you do a Pulumi plan or an equivalent of just the plan or do you always do an app and the plan happens regardless? There's a Pulumi preview command as well. Alright. So we can see here it wants to create a new device. In fact, one thing I will do then is do a preview and we can also specify a def.

27:15 So we'll we'll actually see the user data and a whole bunch of other things. Yep. Good comment from Paul as well, very new feature in Pulumi is that you can write these abstractions in these libraries publish them in TypeScript or go or Python or .net and then consume them from any of those languages too. So even like Steve, could write a go library for creating kind customers on Equinix metal and I and my TypeScript thing could still use that which I think is really cool. That's nice. Alright, this is a resource we're gonna try and spin up a c three medium and

27:52 Amsterdam Six and hopefully get a cane cluster out of it. So I'm just gonna go for that and hope for the best. And now we wait. The chat is is really kicking off object object is having a party to themselves there. It's weird like you can hide it and then bring it back and like the names work. So Bala asked in the Discord chat on local does it store secrets on a file or some DB? I'm pretty sure Paul will be able to answer that much better than I. So one of the things Pulumi does is

28:06 Provisioning the Bare Metal Device

28:35 in my local home directory. I wonder if we can just take a look at that quickly. Pulumi and we have all of our stacks and we'll see my production one, we've got this like JSON and I don't know what the ATTRs is. I'm not even sure. Won't guess but all of our encryption information or secrets are all kind of stuffed in there and you can use KMS, you can use vault and you can use Pulumi teams which is a bit secure but you know as I think what's just really cool is that I mean I could just store a big

29:13 long password and one password and distribute it to my team and I can still commit all this to get and it's secure like you know it's a really quick way to get started to bring and get platform side rather than just the application workload side. Objects is object is awesome. All the same we need to approve. Clean me up. Did. I did. Did. It's running. Yeah. I left it sitting there for too long and I noticed it. I don't think anyone noticed but obviously Paul's keeping me honest here. Alright. Well, spinning up some bare metal on

29:49 Echorus metal takes about ninety seconds to hopefully two minutes worst case. Oh did I use a fast provision? What what I've been to today is? Alright, it may take four minutes. I should have used a bunch of 24 but still we'll we'll we'll get some at the end and I think just for the sake of today if you're alright with that Steve like if if I don't get a Kain cluster I'll probably just manually SSHN and create it so we can move on to the next step but I'm confident our user data that's I mean that's

30:24 some pretty solid IP there I think we could sell that. Quite like being a floating head especially when I'm got code behind me. Yeah. Hold on. You have to review my code for me. How's it look? Problem there. Let's see. Pulumi Pulumi. I guess we can go over to my dashboard. Yeah. We can see our Gallup's device here. We can actually just click on the table in. Okay. So it was pretty quick. We're just the servers being rebooted. We're just waiting for it to dial in. Probably find I'm getting pretty close to being able to

31:14 SSH into that. Not quite. So there's one other thing. We we we have a cane cluster, we've not deployed a flux to or anything yet. No. I can do that manually or I can show off a really cool feature of Pulumi and I'm trying to decide what I wanna do. I'll talk If if we're trying to bootstrap properly, then then that's let's use the tools for the job. So if I if I if I were if I was gonna bootstrap the the cluster, I would in in in Terraform world, which I'm familiar with, we deploy

32:02 flux using that. So it's it's standing up the cluster plus the initial bootstracking process. And it relate to you. Alright. We'll do it properly. We do have this as itch. So let's see when are we. Silly me. Okay. Docker container. Why did I not get my cluster? Oh, it's not cluster create. Oh, it's create cluster. Alright. So let's just do that. We'll fix it here. So when we push the code it works and we'll blame the chat because they should have been all over that. Yeah and we never chimaudied our acute control. Nice, okay. Now it's definitely a million dollar script.

32:20 Debugging the Machine Setup Script

33:12 So what I'm gonna do then so that we can still use Pulumi to apply flux to our cluster as I'm gonna actually just grab the cube config from this machine store it in the Pulumi config and then use it and the provider. Yeah. Okay. We have a working. Okay. Much better. Okay. Now we did also change the user data. I don't wanna force the device to change on me. I'm actually gonna revert these changes and I will fix it before I push the code to the Internet. I promise. And the reason why is that I wanna

33:15 Getting and Storing the Kubeconfig

33:56 be able to run Pulumi preview dev after I've added the cube. What? How dare you? Did I break it? What have I done? I can't use ignore changes, so Paul suggested I can just say ignored. Oh, I can't do that on the actual resource, that's a kill feature. He is a legend. Sorry I did to you. Okay. Make my changes again and you know you can tell me why Pulumi is telling me my password is incorrect if you've a minute to me. Chimaud plus x user Ben cube control. Okay. That doesn't help me does it?

34:57 That is a known bug I am working on as we speak. How do I fix it for just now? I'll wait. I'm gonna export my secret so hold on see that helps. Well I'm just gonna move this window over here. Yeah, he's told me I need to just export. Yeah. Oh and I tried the wrong tab. Okay. Let's go back. So export. Pulumi config passphrase equals. Alright. I mean it's not even a real password. I don't know why I'm going through so much effort to. Okay. So let's see. Preview. Now if I do a deaf now.

35:58 Yeah. Paul's told me to, no, I'm just gonna export to Paul because even though it's not a real password, there was a blob visible in my production dot YAML that if they had the password they could maybe correct with that salt be a lot of effort but I'm still not gonna do it. So we can see that with my fixes this wants to recreate this device. I do not want to do that and as Paul reminded me is that after our device RX we have some Pulumi RX one of the really cool things you can do

36:25 there is just tell it to ignore changes. String. Am I not using that right? Ignore changes to any of the specified properties. Yeah. So user data. So hopefully when I run deaf now, it doesn't want to change this resource and then we can bring in that cube config and user data with a capital D. No, no changes. Don't ever use that flag. Thanks. It was very special use case. So now I'm gonna grab this cube config. Run it locally. Where does kind store that I can do kind how do I get the cube configs on

37:26 this, do you know? Is it is it kind plastic conflict? Yeah. Okay. If anyone wants to take that out and have access to a cane cluster for the next area, you're more than welcome. I shouldn't say that Duffy's probably got some OCR running already that does it. Okay. So what am I gonna do? I am gonna store this as a secret called cube config config set. Woah. No. That didn't work. No. Let's base encode it. Base 64. I broke it. Yeah. That that looks like half of it. Okay. Copy. Who just thought I got stuck on the

38:36 cube config. Alright. There we go. Is that it? Yeah. Yeah. That's it. Okay. So let's do cat cube cube config. Pipe that into a Pulumi config set secret kube config and hope that just put the dash on this will do what I want. Looks promising. Success. Oh, you have three of the same thing, but Try that again. That can't be big enough for me. That can't be big enough. Right? Right. You know what? I'm not gonna store it as a secret. It's not a big deal, it doesn't matter. Let's just do get ups and just store it as a multi

40:09 liner. Yeah. That's alright. Yeah. That'll do. Okay. So let's get me finished in the next few minutes. So we need one more Pulumi provider, which is the Kubernetes one. Which I'm sure I could just have done NPM install with roughly the same thing. This gives us access to the Kubernetes thing. I'm gonna create a custom provider with my cube. Now this last five minutes is obviously the hacky approach where I'm trying to make it work. If you were using Pulumi it has been up JKE or EKS you would already have access to a cube config through

40:20 Adding the Pulumi Kubernetes Provider

41:00 programmatic means. I was just trying to speed things up and probably slow them down a little bit, but still. So now we can import our Kubernetes provider. I just called Kate. We go down here. We're going to create a Kubernetes provider. Give it a name Kubernetes. It should take a cube config as a resource and I'm going to do config dot require cube config and that should give me a Kubernetes provider that I can apply a resource to. So we're gonna be using flux one today. Is that correct? That's correct. Yeah. Flux one. Alright. So long story short is that I didn't

41:42 Preparing to Apply Flux (Pulumi & Manual)

41:52 have enough time to actually rework the whole entire repository over to Flux two. So maybe we should do this again once I've done that. Sounds good to me. I think Noel's trying to ruin my day today. Your Cane cluster won't be available on the internet. So let's try. Alright. Maybe we will be doing this manually. Docker. Is it actually published in the port? It only banks to 12700. Right. Yeah. Is that something I can change? Let's just do it from here. Yeah. Let's just apply flux one. So I'll show you what this would have looked like. I'm sorry

42:37 that I can't show you it for real but we could have done flux v one equals new Kubernetes deployment. Core, no apps. V two deployment. Give it a name. And then when we open up the braces here, we have access to all of the spec, the template, etcetera. Right. So you can flesh out your Kubernetes spec this way. Another really cool thing you can do is do a new Kubernetes dot yaml dot config file, give it a name, point it to a fail which actually accepts remote locations and so you can apply remote yaml. One last thing you can do is you

43:28 can do helm charts v three chart, give it a name and then provide all the details and the values here too. So like there's so much flexibility with the Kubernetes provider that allows you to do everything that you need to do as well. Alright. Let's just do it manually. So flux one, I just grabbed that from So what we what we could do is the majority of the scripts are actually in the repo. So if we if you're if the repository that you thought that you thought was is public, we can climb that whole repo onto the

43:51 Manual Flux Setup on the Machine

44:02 machine. Okay. No give us a good idea here. Instead of using kind I should have used case three d or case yes. That's actually a really good point. That would have been much better. I'm learning, learning much. So let's install git. Git clone. So this is SSH URL. Alright. What do you want me to run here? So there's a script inside the scripts directory. There's a flux in it. So if you run the flux ah, yeah. We're gonna need help. So of course. With all these little things we don't think about. Install. Quick start. Just give me a

45:16 curl out there. Oh, there's a you spelled your mark wrong. Oh, of course. Get her You can get that from the main flux repo. So flux c d slash flux. Give me a curl version. Alright. Release it. There we go. Oh, wait. Yeah. Flux one two. Okay. Linux. No. No. I don't know what. I know what you think. I think it's okay. I think it's fine. I've got we're we're we've got one more release left on us before that stops working. Yeah. It's just installing the helm operator now. So we'll give that a few seconds.

46:49 Running Flux Init and Adding Deploy Key

47:02 Okay. Cool. So, yeah, that's your Bitlab deploy key. So for for anyone that doesn't really understand how this process works, essentially, we're gonna do is Flux is the agent that's running inside of our client cluster. We're gonna have a repository that is gonna store essentially the definitions of the manifests and the resources that we want to deploy into the cluster. So we have to use this deploy key as a way for flux to be able to talk to that repository. So I just need to add this to this get ops repository then? Yes, please. Apply keys.

47:43 Go demo. Do we need to Yeah. Yeah. Do that. Click that. Alright. Cool. If you go back to the script now. And just hit the button? It it will wait. And as soon as it's seen that it's able to get connectivity, it will continue. Alright. So while we give that a second, I'm just gonna make sure you've got SSH ability via the shared terminal. Oh, come on computer. Seems to be happy with me today. Alright. So you should be able to modify that repository which we have open here. You've also got a terminal. Let's check on the status.

48:50 How often does it check? It hit enter. It should it should have checked by now. Oh, which repository is that using? It's using yours. Right? Yeah. We haven't made we haven't made the changes. There we go. Easy fix. It's not gonna change the deploy key, is it? It essentially, we're about to find out. I guess it's the best for everybody. Right? Don't just copy random random repairs off the Internet and expect them to work first time for you. Oh, Waleed has joined us. Hey, Wally. Hi, Wally. Who says he's getting TGIK episode deja vu. Alright. It looks like the cluster is now

49:43 ready. Cool. So now if you do kubectl get pods Feel free to type in the terminal. Why don't you take over? I cannot edit unless I sign in. Okay. Bear with me two seconds. I can see we've got the helm operator running and we have a flux namespace. Oh, my alias doesn't doesn't exist here. We should run-in flux, memcached and op editor. Oh, you have joined. Let me give you access to that terminal. Done. All yours. You may wanna alias keep control to key. Do you use that alias? No. Yeah. I yeah. I do. But I'm gonna get myself

49:58 Checking Flux Pods and Components

50:54 so confused if I do that. So yes. So get pods. So, again, got the default Helm operator in the default namespace, and then as you showed, the dash n Flux here. So we have when we deploy Flux, Flux comes as two main components. We've got the actual Flux agent itself that is gonna sit there and interact and reconcile our repository, and then we have a Flux mem cached pod that you can see here, which is actually gonna store or is used to store all of your image tags. So imagine you're doing continuous deployment, and you what we don't wanna do is

51:00 Flux Setup Explained

51:37 have to constantly keep updating our specifications in Git. We want a way to be able to configure that once, check that into our Git repository, and then essentially have Flux be able to determine and look at the image registry for new image tags, automate the deploy of those image tags, and then the kind of secret sources are already writing it back to Git. So Git is our centralized source of truth here for the workloads that are running within the cluster. Nice. And then kind of Flux's kind of right hand man essentially is the the Helm operator, and the job

52:17 of the Helm operator is to manage the life cycle of Helm releases. So with Flux, you have this a resource in Kubernetes called the Helm release. And, essentially, what that is is a Helm chart at a specific version with a specific set of values. So rather than you kind of in the old world having a home chart and different values dot YAML files, so you may have had, like, dev values dot YAML and staging values dot YAML. You essentially can define all of that as a single resource that can then be committed to Git. Flux will sync that. Flux doesn't have the

52:56 understanding of knowing what to do with it. It's just being used as a sync mechanism. And then the Helm operator itself will pick that up and actually be able to reconcile that. So the Helm operator is doing the Helm upgrades, the Helm installs, the Helm rollbacks, the Helm deletes, and Flux is just actually being used as syncing those into the cluster. Does that make sense to everybody? Yeah. That sounds good. Let's see the magic. Can you guys see me expanding out the the terminal there? No. But I can expand to that. That's okay. Okay. So if we take a look at sorry.

53:35 Not the terminal. The the the repository on the on the right hand side of the directories. Let me follow you. Right? Now anything you do, we will see. Cool. Okay. So with regards to how Flux is configured, especially specifically for this repository, we are using customize. So the way that customize works is essentially you have a base, and the base is really how I like to think about it is an a la carte menu of multiple resources that can be used within a specific cluster or within multiple in multiple clusters. So here, all we're talking about is the

54:17 the development cluster. So that we have a development cluster, and then that is going to inherit resources that currently sit within the base configuration. So the base configuration is gonna store all of the resources without the environment specific configuration. So think of things like URLs or maybe an API an API token or something that we're gonna deploy as a secret. Those are the diffs. So the diffs sit in your environment directory, and a large majority of your majority of your of your YAMLs essentially sits in base. So if we take an example, so inside base,

54:55 we have cluster. And, again, inside cluster, we have a a namespace here. So we can see the namespace in this example is called a demo. And then if we expand out into into the dev directory underneath cluster, you'll see that there's this customization. And for this customization, what we're doing is we're going back up to the base directory, and we are gonna in we're gonna basically consume everything within that cluster directory. So this allows us to be able to put a large majority of our configuration in base, reference to that configuration within our environment specific configuration,

55:44 and then have the ability to be able to override certain portions of that. So the best example really here is gonna be for our pod info. So pod info, again, has a namespace with the name pod info, pretty self explanatory. And then we have a helm release here. So this is a definition of a helm release. So the helm release is saying we gonna give it a name. So name being pod info. We're gonna deploy it to the namespace pod info. We have a chart, and Stefan has provided us a nice chart registry there that we can use at a specific version.

56:25 So here it can be can be a reference to git if we wanted to. It doesn't have to be a URL. It can even be local. And then a release name. So this release name is a release name for the specific instantiation of this helm release. So this name needs to be unique at the cluster scope. So one of the things that I'll kind of come on to is the CI portion of this. This is one thing you wanna guard against. So what we don't wanna do is be deploying the same multiple different helm releases

56:59 with the same helm release name. We really wanna be keeping these things unique. And then finally, the values file. So this is your kind of common dev dash values dot YAML. And all this is doing is going to take the default values that are in the pod info helm chart and then override with with this one here. So what we're saying is for any base configuration configuration in any environment, we want to deploy this version of the Helm chart with this tag. The way that kind of all gets bundled together is via a customization. So the customization here is saying,

57:40 for this specific customization, we want to add these two specific resources. So we wanna add the namespace, and we want to add the helm release. And then when flux actually gets hooked in, it essentially traverses the tree from bottom up. So if we go back to the dev directory, you'll see here that there is a Flux customization a customization, sorry, in the dev directory, and this points to all of the other directories within this top level directory for the environment. So we can see here actually, I'm gonna kind of gloss over pod info. In our Helm releases directory, again, we're just

58:17 going up to the top level Helm releases directory because we don't need to keep repeating the same configuration. We're always gonna be deploying a Helm operator the same way in every environment. Otherwise, this is not gonna be able to be reconciled. And then, again, the same thing with the customization for the cluster. So all we're doing here is essentially building up the overall makeup of our of our environment. So now what we have is and what I've shown you is simply just a a kind of rinse and repeat. Right? It's a copy. It's an exact copy of

58:56 base. But now what do we wanna do, or how can we make changes to base that are specific to our environment? And the way that works is by leveraging something called a a customized patch. So in pod info here, you'll see we have a customization again, and we are taking a resource, and that resource is that top level declaration that we have in our base directory. And then we are gonna perform strategic merge. And we're gonna take this strategic merge from the Helm release YAML in the same directory. And our Helm release YAML looks like this.

59:38 So what we could do if we wanted to is change this to six. Right? And imagine we had a lot of configuration in base. All it will do is it will blend what's in base with what we've got in here, perform the strategic merge, and then what flux will do is deploy the exact same chart but just with a different tag. Does that make sense? Yeah. It does. Sounds good. So because this is already checked into Git, we've actually already deployed this. So if we go back to the terminal and the kubectl get pods dash n pod info,

1:00:18 you'll see here that we've already got pod info deployed. This is where auto complete is gonna kill me. Yeah. It's annoying when you don't have your tools in front of your normal workstation tools available. You actually have to remember commands and stuff. Can I copy it? Yeah. I think you're about right. Okay. So what we can see here is that, actually, we've gone to version four dot zero dot six. However, at no point in this declaration have we ever specified the version four dot zero dot six. Even in our dev directory, we have specified the version four dot zero dot three.

1:00:56 Explaining Flux v1 and Image Automation

1:01:14 However, if you remember before, I said we need a way of being able to do continuous deployment. So I made a a major assumption that Stefan is already doing work whilst we're recording this and has created us a new image. And we use this flux annotation. So like everything in Kubernetes, we have this concept of annotations, and we are gonna set the Flux automation to true. So once we set this value to true, we are telling Flux that every every time you perform a sync with our registry, can you go and look for new image

1:01:52 tags that match the prefix or the semantic version that we have set below? So you can set this in flux five minutes, ten minutes, fifteen minutes, one minute. It doesn't matter. And every time it goes and looks at the registry, it will see if there's a new version. It will deploy that version for us, and then finally, will write it back to Git. Here is the slight difference between using customize as the way of implementing this and just kind of having a load of YAML files that you don't use customize on. So what you'll see here is there's a

1:02:31 Flux Writing Back Changes (Customize Patch)

1:02:31 flux patch directory sorry, a flux patch file within the dev directory. And now this is saying, okay. Well, the flux patch has specified that the image tag for this Helm release has gone to four dot zero dot six. So when you're looking for diffs between environments, you're not just looking at the YAML files. You're also looking at the flux patches as well. If you weren't using customize, it would actually make the changes to the YAML files directly, and you would we would just be able to go straight to to here and actually see the fact that there's a change to

1:03:00 Deploying an Update

1:03:12 six. That make sense? Uh-huh. They're just two different ways of being able to see it. So I don't think I can commit easily from here to your repository. Why don't I drive you through making a couple of changes and we can actually see this in action? Yeah. Of course. Alright. So go ahead. What's first? So let's start with something very basic. Let's just create a namespace. So we have multiple options here. We could add the namespace to base or we could add the namespace just to our dev cluster. It feels to me that we should maybe

1:03:30 GitOps Demo: Creating a Namespace (in Git)

1:03:55 add it to the base. Yeah. So we could add it to base and then inherit it in our our dev cluster. Definitely. Let's do So I can just call this the Rawkode namespace. I'm just kinda copying what you already have anyway in the demo one. And then namespace. It's a meta data or spec for namespace. I can never remember. I think it's meta data. I think that's valid. Let me let me check your demo one. API version. I should know better. That was poor. Metadata name. Yeah. I'm good. There we go. Okay. So key thing to note here is that you've

1:04:53 just created a namespace. Now we need to create a customization that actually inherits that namespace. But because you've added it to the namespaces directory, where we need to add that file is customization that sits under the namespaces directory. So stay where you are, and then underneath your namespace dot YAML, you'll see that customization dot YAML. Yep. Now we need to we need to add your namespace to here. Now could I just do namespaces star star star or do I need to very explicitly say Rawkode? Let let's try it and actually we can we can run a test script locally and

1:05:26 Testing Kustomization Build Locally

1:05:34 actually see what gets rendered. I'm just gonna do that. That that may be right. Okay. And now because so you've made that to base. Right? And if we go to the dev directory and we look at cluster, if you expand the cluster definition and you look at this one, we inherit everything from cluster. So you can naturally you're gonna get this. So now if you go back to your console, you should be able to run make and then yeah. You need to and then test dash dev. Am I gonna need flux control? Nope. Nope. Okay. So this is basically saying that it

1:06:26 Debugging Kustomization YAML

1:06:26 doesn't I don't think it Yeah. Don't think it like that. So I'll do the I'm gonna try a few more because I'm feeling brave. Okay. I'm not gonna do that then. Namespaces Rawkode namespace. Yeah. Cool. Okay. Let's try that again. Yeah. Much happier now. There we go. That's better. Okay. So then now if in your directory, if you go to there then there'll be an underscore test. I don't know if you wanna show it in Versus Code. Perfect. So you can see that if you now look for for Rawkode, you should be able to see

1:07:09 the namespace got added. So if we now commit that to git, we're gonna need to do kind of two we probably want two split terminals here because what we're gonna wanna do is do a get namespaces and have it in a watch, and then you're gonna wanna commit. Okay. So get namespace watch and then we're gonna pop over here. So what what's the how often I'm assuming that flux isn't pulling GitHub every one second. Like, what's the what's the Alright. So we we've configured it now to be I think we've configured it now to be about every every minute or every

1:07:30 Watching Namespace Creation in Kubernetes

1:07:44 couple of minutes. Get Pesh. I always type that. Alright. So that's the way we pop over here, and we wait for the the magic to happen. Yeah. So what we probably should show what we can show is the logs from Flux. So if we go and get the Flux pod logs, we can actually see the fact of exactly what it's gonna do in the background. Okay. Flux get pods. Nice. Log slash f. What's the timestamp I've So what you're seeing is the old if you see here on the your line there, you've got the old

1:08:05 Checking Flux Logs for Sync Progress

1:08:35 Git shard and the new Git shard. So it's now seeing a new a new Git shard there. So, yeah, where you are there. So you're gonna see the old one and then the new one. So so Flux is obviously gone. It's constantly reconciling Git. So the key thing to note is that, again, flux and well, kube Kubernetes in general is eventually consistent, but flux is also eventually consistent. So we're gonna have to go around this loop possibly a few times for us to be able to actually sync that namespace. Alright. Okay. So it's just not seen the

1:09:14 namespace yet. Is that? It's not seen any. Alright. Guess we'd see something here. Yeah. I'm seeing a lot of unchanged so far so we just have to make sure I pushed it properly and actually left. Right? Yeah. Okay. That's okay. See if I can see it on GitHub. Alright. So we added this and we got this. Cool. And this is definitely using our dev overlay. Can we confirm that? Yep. If you go to do a cube CTL describe deployment and then check the flux deployment. Oh, gotcha. Flux describe deploy flux. And what were we looking for here? We're

1:10:17 looking for the branch configuration. Scroll stay where you want. Yeah. So git git branch master. Ah, are you pushing to master or are you pushing to main? Oh, you must be pushing to master. Otherwise, it wouldn't have sunk. I pushed to master. Yeah. Okay. I would have expected to have kinda seen the names. Yeah. I would have expected to do it for now as well. So I got something wrong. Let's let's call it that. What did I get wrong then? Go back to the flux logs. Of course. So we can actually I guess we can

1:11:04 verify the commit shot here. Right? So Yeah. This is saying that it pulled 52CF which oh, it actually is my commit. Yeah. So what we also did as well is that we did the we did a test which essentially showed us what flux was going to reconcile. That's interesting. Why? Is this a valid namespace? Maybe I have to test that That looks valid to me. Where is this split? Let's just be sure. Rawkode. API version kind. Oh, that got capital n. Oh, yeah. That might do. Metadata name. Yeah. I wonder if that's it. I wonder

1:11:33 Successful GitOps Sync and Namespace Creation

1:11:57 if it's really just you think you would get an error message in the Flux logs. Yeah. That's that's not a resource. Fix. Don't be a dummy. There we go. I always do that. So annoying. Anyway, let's push. Let's jump back over here and I'm so confident that this one is gonna work. I would just give that sixty seconds. Yeah. Small end. That was still the other. Hopefully, was it. This is why if we were doing if we were doing a PR properly, alright, we'd we'd catch it with CI. So one of the things we probably do

1:12:40 GitOps CI Checks

1:12:42 wanna show is whilst we're waiting for this is if you go to the the circle CI directory, we can actually show some of the CI that is done or could be done to this to this repository. Okay. So using a there we go. Rawkode. There we go. Ta da. The Told you it was my fault. It's always my fault. So, obviously, what you did was a prime example of why we want to be using some mechanisms that we have available to us in gits to stop those kind of things from happening. Right? We don't want someone to accidentally

1:13:25 CI Validation Strategies for GitOps

1:13:27 merge something to master, and we end up breaking the whole entire cluster. That's not ideal. So in terms of tooling that I would recommend people leverage to stop this from these kind of things from happening, there are a number of tools. So the first one is is kubectl. And, actually, if we executed kubectl, kubectl would have told us that that was not a valid YAML file and would have stopped us from being able to merge it. Yep. So that's the first one that we have. So that's just doing strict validation against the upstream JSON schema.

1:14:00 Check for duplicate Helm releases. Again, that's what we talked about earlier about how your Helm release names, you don't want them to clash. So this is just checking for duplication. Deprecates check. So, obviously, as everybody knows, Kubernetes loves to deprecate APIs to click at one sixteen. So what this is doing is it runs in the background a set of Reggo policies. So Reggo policies think Open Policy Agent, uses conf test as a kind of command line tool to be able to run those, and we'll run those against the output of a customized build. So when they would run

1:14:40 make test dev, what he as actually running behind the scenes is customized build pointing at the dev directory. And it will run all the Reggo policies against all the outputs and then validate that you're not using any deprecated APIs. So these are kind of the top top level ones to stop us from, you know, making slightly silly mistakes. But the main one that we really wanna talk about is is HR Vowel. So if you noticed, we were deploying Helm releases. Right? So so doing KubeVowel and doing some contest against that, not really that useful in the in the grand scheme

1:15:18 of things because the Helm chart is actually what's what contains the raw YAML files. So what we really wanna be doing is validating the output of them or essentially what the Helm operator is going to be deploying. So what HR Vowel does is it takes a combination of obtaining the Helm chart at the at the version that you've specified, passing in a mixture of the default values and the values that you've specified in your Helm release, performing a Helm template. So Helm template is just gonna spit all of that out, all of the raw YAMLs for that Helm

1:15:55 release out into into the console and then runs kubectl against them because those are the actual things that we're deploying. The Helm release is just a wrapper. What we wanna be doing is actually validating the inside, the kind of the, you know, the the meat and potatoes of the Helm chart, not assuming that the Helm chart is gonna deploy. And then the the final one is is one that I've added because it catches me every single time, which is the customization specification is so specific that what you want is something that tells you that you've made a mistake before

1:16:32 you give it to Flux to tell you. So there's a customized edit fix, which essentially its job is to tell you that you've either misspelled something in your customization dot YAML or you're not conforming to the recommended way of doing it. So release the order of having releases and then patches, etcetera, etcetera. They are very they have very specific standards. So having that is gonna make your life more easier by the time Flux gets it and can't actually execute the customized build. And if you scroll down, I think that that's possibly it. Yep. Yep. And then we just put them

1:17:17 all together. So there, all we're doing again with CI is just getting more confidence that before Flux actually syncs the repository, it's gonna be able to deal with what we gave it. Yeah. But I was making up my wildcard syntax. I probably should have run this one. So that's really cool. I I like the way that works. I like, you know, the way we can compose those layers together with the bases and the dev and stuff like that and you know, there's something running in my cluster containers with pillow chase using apply them like, get up to Rawkode. Yeah.

1:17:38 Summary: IaC for Platform, GitOps for Workloads

1:17:55 So I think the the great thing about this, right, is let's let's assume that we didn't have to manually SSH into the cluster. We didn't have to run a couple of commands. And what we did is we had a really nice way of bootstrapping the cluster with flux, with a SSH key or a deploy key pointing at the right directory. We just run Pulumi up, and the whole cluster just bootstraps itself. Right. And then after x minutes, we come back and the whole cluster's there, all the all the workloads are there, all the services, all the ingress, everything. Right? But let's then

1:18:32 take it one step further and go, actually, do you know what? By accident, I run I don't know what the equivalent of Terraform destroyers. I'm assuming probably Pulumi down or something like that. We run the equivalent of Pulumi down because I did it by complete accident. I just run Pulumi up again, and I get back to exactly where I was the last time the cluster sunk because everything is stored in Git. Right? And if I want to take dev and create staging, I take my lovely Pulumi config. I change the word dev to stage and make the changes in the staging directory

1:19:06 and I just run Pulumi up and I've got a staging environment. Well, I I feel like you were calling me out here and saying that it would have been nice if the bit that I did actually worked and I I know that it's harsh but you're right and what I will do you know obviously we're not gonna have time to do this this afternoon I am gonna swap out the user data to do a KCS cluster which will be exposed to the internet I will dynamically get the cube config and I will install flux and I'll put I

1:19:34 would just push this as an example of a way to do it so that people can see it. So why don't we just is there sorry. Is there anything else you wanna cover or should we just jump into like a summary of of what we put together? No. That that's yeah. Let's jump into a summary. Make sure that we've answered answered everybody's questions that they have. Yeah. Please if you if you have questions drop them into the chat or into the discord, we'll tackle them just now before we finish up. But you know, the two pieces of the system that

1:19:58 we're I think we're kind of drawing a line in the sand here, right? What we're saying is that you need automation something like Terraform or Pulumi that does the platform provisioning aspect. And then we have the flux which is the workload provision aspect and if you can really get them working together, you have that ability in a single command to spin up an entire environment. But because we weren't using a kind of managed Kubernetes or like cluster API, a real way to spin up a bare metal cluster because I thought it would be quicker. We didn't have that programmatic access to a

1:20:29 cube config which is unfortunate because we could have deployed flux and a nice shiny fashion. That would have handed over to your flux pulling on the manifest using customize and the bases deploying the workloads and all of that would have been amazing demo of just pulling me up, boom, work. I I I think what we're trying to say right is that a lot of with with a specifically a GitOps paradigm, we want everything in Git. That's the kind of, you know, the golden standard here. So what we don't wanna be doing is pushing a lot of code or a lot of

1:20:51 Further GitOps Benefits (Self-Upgrading, Audit Trail)

1:21:04 YAML files or manifests into the Kubernetes cluster. We want the absolute bare minimum, and the bare minimum is gonna be flux pointing at our repository at a given directory. And then all of the config that you need gets sunk into the cluster. So it's not it's it's a very small push based model because you're gonna have to push some initial configuration, but then there is a massive pull based model as well. And I think the the one thing that you and I were actually going back on Twitter talking about was Flux can upgrade itself. So Flux can upgrade

1:21:38 Flux. So you can have a Flux in your Pulumi config, and you can have a corresponding Flux in your Flux repository. Yep. And you could start your cluster with, you know, Flux one dot five, but then in your repository, have one dot six, and it would constantly keep itself upgraded. So that initial push, you're not bothered about after Pulumi up has has completed because all of your life cycle now is done in that repository. And you're back to git, you're back to CI, and you're back to all of the PR process, etcetera, etcetera, etcetera. And you're driving

1:22:11 everything from from there. Yeah. Definitely. Hey, Marky. Good to see you. Hope you are. Alright. I'll give people just, I guess, thirty more seconds, please. If you have questions, let's let's get them in. Thank you for joining me today. It was really good to actually sit down with you and have a proper conversation and pair on something that we've not done that before. It's been very, it's it's always you and I going back and forth on Twitter and it's been really pretty good to sit down and I've learned loads. Right? I've never I've looked at Pulumi from the outside. I'm I'm

1:22:46 so kind of head in Terraform land because that's kind of what I'm what I'm familiar with. And actually seeing the the power of Pulumi and some of the things that actually Terraform doesn't do very nicely, and Pulumi's got that obstruction and how that can fit into the current workflow of, you know, just hooking in some some manifests and getting them to reconcile. Like, I think it's great. The the the bootstrapping mechanism is there whether you choose Pulumi, whether you choose Terraform, or whether you choose any other infrastructure as code tool. But the the pieces of the puzzle are there. Stop. Get the

1:23:19 cluster up, get a Flux instance, and away you go. You're you're ready to reconcile and start to deploy workloads. Yes. Definitely. I I just can't see past Pulumi anymore. I've just been using it too long, I understand you know it's got some weird things in it of course all software does but I know I've got over the hump there and I just when I go back to TerraForm or I try to we have Terraform modules for Equinix Medal and when I go to try and contribute to them I'm just like I feel like I've actually traveled back in

1:23:47 time like the Stone Age or something I find the tool is working against me and I used to love Terraform and it's still a great tool I just I don't know I struggle with it now and it's my thing and not not a bad thing about the tool. Guess just because we have a few minutes and you know you're a very knowledgeable person why don't we just talk about get ups then? Like Sure. What are the big changes with flux v two? Have you been keeping an eye on that? So I think with with flux v two,

1:24:07 Considerations When Choosing GitOps

1:24:15 if you if you think about flux or the flux agent, it was doing a lot of stuff. Right? It it it was looking for new images from its from the image registries. It was syncing repositories. And what Flux v two is doing is essentially splitting out and decoupling these things into into smaller agents or smaller controllers. So there's an image controller. There's a customized controller. There's a Helm operator that they've called Helm controller. So they're decoupling them, and they're they're moving much more to a model whereby you can declare absolutely everything with their configuration. So think about you've got a fleet of

1:24:56 clusters. In this scenario, we've just got a single cluster. Yeah. So how do you define a fleet when you do when you stand up one of those clusters within that fleet? What's it gonna be reconciling? And the overall observability of flux v two is is generally incredible compared to compared to flux v one, knowing we wouldn't be able to go to a dashboard and see that that sync wasn't being successful as an example. So a lot of the kind of griping pains that people had with flux v one is being fixed in flux v two,

1:25:28 and they're essentially decoupling it. So that is, unfortunately slightly more YAML files that you do have to get your head around, but you you have a tighter control over what each of those things does. And, yeah, I think that's I think that's a really powerful mechanism. Alright let me throw another question at you then like, is there ever a reason you shouldn't use GetOps? Because I think you've sold me on it, I see this incredible power and flexibility and continuous delivery, but as there as there anyone is watching, they should be like, oh, I shouldn't be doing this.

1:26:00 Is there any of those gotchas that people should be aware of? I I think it it comes down to firstly understanding the mech like Kubernetes as a as a as an orchestrator. Right? You there is a lot of power there just from what Kubernetes is doing itself. And if you have already got a load of pipelines that push your code or push your helm charts into Kubernetes, actually pulling that out and putting it into GitOps could be very, very painful. Right? There could there could be so much logic that's in those pipelines that developers or engineers may or may not

1:26:39 fully grasp. They just hope it works or, you know, they haven't they haven't looked at it for for a long time. Yeah. Maybe maybe that's not necessarily move the best way to move to to. But I think for me, it's not about the syncing. It's about two things. One is using the tools that every developer knows how to use, which is Git, understands PRs, understands CI. Then also the ability to to some degree, and I'm gonna caveat this with, obviously, of a number of caveats here. You can start to move to seeing your cluster as ephemeral

1:26:54 Key Takeaways on GitOps Benefits

1:27:18 because the cluster is just is just a way to deploy workloads, and it it always to some degree has. But if we can, like we were talking about earlier, remove or destroy the cluster, re spin it back up, and reconcile everything from a known state. That is extremely powerful. Obviously, you've got storage that you have to deal with, etcetera, etcetera, etcetera. So those there there comes the caveats, and Yep. I'm definitely not gonna talk about CloudMated storage. But also as well, I think the other useful thing is you have an audit trail of changes. I don't

1:27:52 have to troll the CI pipelines and work out how far things got through. I have an audit trail of all the changes. So if you're working in a financial or regulatory industry like I am with insurance, I can show the order to all the changes that happened at any point in time. I don't have to collate this lovely spreadsheet that they love to see. I just show them the commit history of our, you know, our x number of repositories that we have. Awesome. Alright. I could easily keep you here for another hour but I'm gonna let you get back

1:28:19 Conclusion and Thanks

1:28:23 to your evening. I'm sure it's late where we are so thank you very much for joining me. I had a lot of fun. I hope we do this again with more stuff. I hope that people watching enjoy themselves as well. Steve's Twitter handle is there under his head sway1987 give him a follow he is always tweeting on awesome stuff on get ups and I also see him answering so many questions from people as well like so if you need help I'm sure he'd be more than happy for you to reach out and he'll do his

1:28:52 best and of course you can reach out to me I'm just not as useful. So Steve have a great evening, thank you for joining me, will speak to you again soon. You too, it's a pleasure. Thanks. Thanks. Bye.