2:43 Introduction

2:43 Hello, and welcome back to the Rawkode Academy. Russell, save the throne and tell us that hello at least. Thank you. Alright. My name is David Flanagan, also known as Rawkode, and I'm your host for today's Rawkode live session. Today, we're taking a look at OpenUnison, a tool to bring identity management to Kubernetes. Now, guiding us today on our journey of OpenUnison, is my friend Mark. Hey, Mark. How's it going? Good. Hey, David. How are doing? I am doing very, very well. I get to sit and play with some Kubernetes software. So I'm That's always a good day. I'm I'm pretty

3:20 excited. Why don't you say hello and tell us a little bit about yourself, please? Sure. So my name is Mark Borschin. I am the CTO and cofounder of Trello Security and also the primary author of OpenUnison. A little bit of where you might have seen me around. I'm usually in, like, the SIGAuth or really anywhere in in the Kubernetes Slack answering questions wherever I can. I've made contributions to the Kate's documentation. I added some contributions to, like, the dashboard around impersonation. Have made some contributions. So I've been I've been around the community for quite some time.

3:25 Guest Introduction and Enterprise Kubernetes Challenges

4:03 Also, I'm gonna do some shameless self promotion here. Coauthor of Kubernetes, an enterprise guide second edition. Nice. So lot of discussion of advanced Kubernetes topics. So authentication, authorization, global load balancing. We get into Istio, and then we actually got rid of pod security policies and replaced it with open gatekeeper. So a a lot of very advanced at the end, we also get into building out a whole automated GitOps platform, which is a lot of fun. So definitely would encourage you to take a look at the book on Amazon. So yeah. So that's that's all about me.

5:00 Alright. Let let me throw the low hanging fruit question out then by default. Sure. I think Kubernetes enterprise ready at the box? Right. Yeah. You know, it it's actually a really great question because what the there there's I've always felt that there's a fundamental difference between enterprise IT and service provider IT. So if you if you're talking about folks like, you know, Airbnb and Netflix and all of these great businesses there, you know, Uber, you know, they they they scale these massive services, and everything is focused on that service. Right? Every every microservice that they build, if it's

5:46 Airbnb, ultimately, it's getting you to book the road. Yep. You know, if it's Uber, it's getting the ride or ordering the meal. If it's Netflix, it's to watch the video. Everything is focused on that. Whereas in the enterprise world, you have a thing that the IT supports, but the IT is not the end product. So if you're talking about a bank, for example, it's you know, you have to have IT. It's incredibly important, but at the end of the day, it's there to support the business trend, you know, the the the money transactions. You know, you talk about government agencies. We

6:23 do a lot of work in in different governments, and there are different departments that have different missions, and IT is there to support them. And so that ends up creating a very siloed organization, and then that siloed organization then translates down to IT. So where in some place like an Uber or another service provider, you have everybody's working towards that same goal. In in most enterprises, you have silos where you as a manager are responsible for a a thing. Right? Maybe it's maybe it's registration. Maybe it's marketing, whatever that is, and you have apps that support it.

7:03 But at the end of the day, your bonus, what you get paid for is based purely on what you're delivering. And so you wanna minimize the amount of outside interference, outside risk. Right? You wanna own as much of that as possible. And so most enterprises, you get a a couple of phenomena where you get a lot of replication. You know? So, like, I've been I've been preaching multi tenancy in Kubernetes since 01/13 was RBAC, I think, when RBAC. I think that was RBAC. You know, I I had a I had a booth. Tremolo had a booth at the

7:42 first KubeCon that was in Seattle when it still fit in a hotel ballroom, you know, in in the lobby. And I was talking to people about multitenant Kubernetes, and they were looking at me like, are you insane? Just have another cluster. You know? And and that works great when you have a lot of Kubernetes talent to manage it. You know, I don't know if you're aware of this, David, but it's very hard to debug Kubernetes clusters if they break. You might have heard that once you've I've got some experience there. Just a Yeah. Exactly. You know, setting them up is pretty easy

8:19 these days. But if they break, that's where things get really hard. And most enterprises just don't have the talent. You know, the people don't have the knowledge. So be like, oh, I spun up this Kubernetes cluster, and it works great. And then it breaks, and it's like, okay. Now what? You know, if it's managed, it's a little bit easier. But, you know so so the when you get into enterprise, you're dealing as much with silos as you are with technology. And in the Kubernetes world, that tends to manifest itself in a couple ways. One is multi tenancy.

8:58 You know, do I go with multiple clusters, or do I go with, know, many small clusters, which, you know, tends to be people's first instinct, or larger small a smaller number of larger clusters. And, you know, the the there's definitely been a drift towards that multitenancy world, especially in the last couple of years as as the technology has improved because you can provide that, you know, that that silo. And then the other thing is the this move from expecting Kubernetes to always be interacted with by a CICD pipeline. You know, in in in the services world,

9:42 it's always, well, I don't want people interacting with clusters. Okay. Well, that there is a lot of value to that, especially in the microservices world. But in the enterprise, you got a lot of monoliths. And the people who own those monoliths, if you tell them you can't log in to the pod, oh, you're get to explain that one to the CIO when there's a problem. So it it's developing the technology in a way to deal with those silos. And that's what we spend a lot of time with in the book is what are the technologies that you

10:18 need to know to properly handle these silos? Because what what I'll often see when I go and I talk to either a government agency or a company that's getting started with even DevOps in general, not even just Kubernetes, is they have this big grandiose idea of a PaaS that they wanna build. And they say, we're gonna build it, and it's going to be a product. And usually consultants who are saying, we're gonna build a product for you. And it's gonna be this product, and it's gonna be, you know, choose from column a, column b, column c, and that's

10:53 it. And, you know, we're gonna maintain and manage everything for you. And when you get into enterprise, you start running into all these little corner cases and edge cases, and all of a sudden, we the team that owns this becomes the people who are the roadblock. And so they're the ones answering the question, well, why hasn't this been done yet? If I just had a VPC, I could do this. Well, we do it this way to make your life easier, but you're not making my life easier. And so then it swings to the other way where, like, you know what? We're just

11:29 gonna give you an empty cluster and be done with it. And whatever happens, that's totally on you. So you get these pendulum swings back and forth. Whereas the beautiful thing about Kubernetes, and this one of the reasons I I I've always loved it. And and, you know, Kelsey Hightower nailed it right on the head as the way he usually does. It's a platform for building platforms. So you can go kind of that PaaS route if you really want to, or you can go the open route or you can go anywhere in between, and there's a lot of flexibility there.

12:02 And so that's that's the approach we take in the book is, you know, the authentication, the authorization. Yeah. When when we start talking about backups, you know, it's not the sexiest thing in the world, but you gotta do it. Right? And then even when we start getting into GitOps, we start talking about, okay. Well, what does multi tenancy look like in Kubernetes? Because it ain't just Kubernetes. You know, you've got something that's storing your Git, right, especially if you're talking about GitOps. You've got a GitOps controller, so you're talking usually Argo or Flux. So what does that

12:33 look like in a multi tenant situation? You know? And then what are the policies that you gotta think of, you know, like, you know, change control boards. How do you do cloud native change control boards? That's fun. Or, you know, dealing with privileged access accounts. And and so so yeah. So we we get a note to a lot of those very advanced enterprise topics that don't don't always come up when when you're reading through the blogs. Awesome. That was a lot of information there. And what I what I love is that, you know, I I haven't spent a lot

13:09 of time in the enterprise environment. I, you know, I haven't worked for a bank since, what, 2012 maybe. So just, you know, before my my Kubernetes days, before Kubernetes. And a lot of the things you were saying there kinda gave me little flashbacks, and I said, oh, shit. Yeah. That was the thing. And, like Is that why I saw that cold look of terror on your face? Well, yeah. But it's it's made me realize that I have a lot of bias in the discussions that I have these days because, like you said, right, I feel from the

13:37 conversations that I have and the people that I speak to at events is that the dogmatic advice these days is to have lots and lots of small clusters. But when you think about it from the enterprise perspective, like, that's a lot of authentication batteries, and that's a lot of moving targets. And that's the the I mean, even the process around handling that, I can see why they would push towards actually, no. I want large clusters because usually they have the money as well to throw at this high this kind of hardware build or data center or even just give it

14:09 all to AWS. But, yeah, they would want fewer clusters because it's less for them to build process and isolation around. So I never yeah. You're flipping around a lot of the bias that I had in my head just because I haven't been working with those kind of clients or those kind of people in such a long time, but it makes a lot of sense. And get up, small eternity, like, for fuck's sake. Like, that is such a beast as well. Like, I mean, I don't even think it was recent it was very recently, I think, ArgoCD even supported

14:37 deploying it within a single namespace. It still does. Like, it kinda does. One of the challenges with Argo and I love Argo. Argo like, if if I have a choice as to what I'm gonna go with and I'm actually we might even see a little demo with Argo today. I like Argo because coming from the enterprise world, I love my GUIs. Yeah. I really do. I know I'm in the minority in this land, but I I I love a good GUI. It makes my life easier. You know, we we don't just do product. We also do operations for some of our

15:14 customers. And I can tell you there have been many a time where I have pulled up the Kubernetes dashboard securely on this thing and done something important, and it works great. Yeah. You know? So I I do like Argo. The the issue with Argo with multitenancy that that people miss is Argo uses the a cluster admin service account for all of its synchronization. And so if you get if if you don't predefine in Argo's RBAC or in the project which objects can be synced, you could check-in, like, a resource quota update as an example. And Argo will sync it

16:08 because there's no RBAC there. Well, these get ops operators, like, the root and the cluster. I mean, the only difference between them and Teller, which we all kicked out really quickly, is that Teller has a whole API. Teller. People could run from their own machines, and GitOps doesn't. But still, I mean, it it's written in your cluster. Yeah. I mean, there there are so, like, Anthos actually does do it correctly. Anthos Centimeters, contact manager? Does mean? Manager. Thank you. Where they actually let you specify for each namespace a different service account to use. I think we've works or Flux does that

16:47 too. What the Argo folks originally had a like, they actually had a PR to do it, and then they gave up on it when application sets got merged because that actually solved what they thought was the problem. But yeah. I I have also not run into a lot of folks besides me who think that Argo should be a multi tenant system. I still tend to be a little ahead of the curve on that one. But yeah. It's you know? And and it's not just Argo. Right? I mean, you think about a cluster. Your cluster is more than just Kubernetes. Right?

17:28 You've got monitoring. You've got you know, and and that monitoring system has identities. Mhmm. You've got yeah. Even if you're just using Prometheus and Grafana, you want everybody seeing it. You've got, you know, you've got your GitOps controller. You might have GitLab running locally. Right? You know? And then if you're running Istio, KeyOwl, he's got a dashboard. Well, you need security around that. You know, traffic has its own dashboard. You probably wanna have some security around that. So a lot of things end up piling into this very, very quickly. Yep. And and the hardest part is really

18:08 just figuring out where the identities all fit together. You know? So, like, you know, I kinda like to talk about, you know, supply chain security is the the big buzzword now. Right? Like, it's s bombs. And that stuff's really important. Don't get me wrong. Yep. I've been beating that drum for a long time, but identity management is a core piece of supply chain security that a lot of people don't think about. I'm actually doing a workshop with John Osborne from ChainGuard at the B Sides Northern Virginia next Friday. So if anybody's watching in DC, would love to see you

18:51 about supply chain security. And, you know, if you think about GitOps, you've got GitHub repos. You've got your cluster. You've got a GitOps controller. Well, the GitOps controller has to talk to your GitHub repo. What identity are you gonna use? You have a GitHub action that creates a container and pushes that container into a registry. What identity are you gonna use? Are you gonna use a static key? I hope not. You know, you've got and and then building these things. How are you gonna automate that? Because you don't want to build that stuff manually. And so the that that's all built in

19:35 to you know, identity management's a big part of that that I think a lot of people are missing when when they talk about supply chain security. And that that's really where where, you know, I'm hoping to show you a little bit of that today where where really our sweet spot hits. Well, you have no set. Supplies in security five times, and we've had two VCs phone in and offer each other. Yeah. Alright. That's okay. They won't give me any money. No. Identity is obviously a big thing. Multi currency is is definitely you know, we have a

20:09 SEC dedicated to it now. We're trying to improve this in the Kubernetes. I mean Yep. Is there software out there that can help us with this chaotic problem within a Kubernetes system? It's funny that you asked that, David. Yes. There is. So what does OpenUnison do? So I haven't really talked a lot about what OpenUnison does. So, you know, the open source identity management, which can mean a bazillion different things. Our largest implementations are actually have nothing to do with Kubernetes. We started off Trello Security started off nice Metallica shirt, by the way. Kubernetes start or Trello Security started off as

20:23 Introducing OpenUnison: What it Does

20:50 a idea to simplify authentication using reverse proxies. So that should sound a little familiar. And the original idea was actually to use Kerberos to eliminate agents. And yeah. Yeah. It failed miserably. It was great. And we ended up growing up from there from doing authentication to doing just in time provisioning. So a user logs in, and we automatically update things, applications. And then that grew into a larger workflow engine, then we got into the user provisioning game. And so we we grew up from a simplistic authentication engine to really powerful identity system. And so, for instance, I'm gonna be talking

21:40 in a couple weeks at kube con or Kubernetes Community Days DC and then also at kube con in Detroit about a public safety system we manage here in the DC region, Washington DC in The US for for all of our international friends, that is built on top of Kubernetes. But the system itself is actually has nothing to do with DevOps or Kubernetes. It's all about managing public safety systems. So our our largest implementations are in kind of the more generic enterprise identity world. I just found this niche in DevOps and Kubernetes and and said, you know, nobody else

22:19 is really doing a lot here. So it's probably a place where I can create a niche. And so in 02/2016, we came out with a namespace as a service utility that allowed you to integrate any kind of authentication into Kubernetes and allowed users to just log in, request namespaces, and and admins could approve it. And this was light years ahead of what people wanted at the time. And so we took that, and we slimmed it down. We said, okay. Let's just create a simple authentication portal, something that's quick and easy to deploy. Most of our users work with,

23:02 you know, an external identity provider. So something like Azure AD and Okta or an active directory. We have a few GitHub users. Most of our open source users or or most of the GitHub users are in the open source world. They're they're they're not so much in the enterprise world. And we we we always strive to make it as simple as possible to deploy with requiring as few external dependencies as possible. So we we went through kind of an interesting journey where and and we're kinda going full circuit is we started with a container that you would launch that would do

23:41 all the work and deploy it because this was still Helm two territory. Then we created an operator when that's what all the cool kids were doing. Yep. And that that helped, but it was still kind of bit of a bear. And then as we're getting more deployments, we found that most of the problems people were having had absolutely nothing to do with OpenUnison and with certificates, load balancers, host names, IPs. And so we we kinda took a step back, and we created Helm charts to simplify it. Yes. But we ended up having, like, three Helm charts because

24:19 anybody who tells you there's eventual consistency in Kubernetes is lying to you. I know it's another controversial one that a lot of people don't agree with me with, but if you've ever had to deal with a custom admission controller, you'll know what I mean. And so we ended up saying, okay. Let's let's simplify that even further. So we we keep created the OU control command to really automate as much as we could. And so that that was one side of it. We said, okay. Let's create this portal that's really simple to deploy. The other half of it that we wanted

24:54 to do was we wanted to make it so that you didn't have to distribute anything for authentication into your cluster. So we didn't want you to have to download a kubectl configuration file. We didn't want you to have to prepublish configuration files. That's one of the harder things with a lot of OpenID Connect implementations is, you know, they'll they'll give you a kube config file or you can use the the great OIDC kube control plug in, but you have to preconfigure it with some stuff. We didn't want you to have to do any of that.

25:27 We want everything to be be prepopulated for you. So our first version, there's just a little button you click that says copy my kube control command, and we set up your entire kube config via kube control, including all your certificates. So everything is trusted off the bat. A lot of developers didn't love that. It worked well if you had, like, a a, you know, jump box, god forbid. But, you know, developers wanted to run kube control locally. They didn't really wanna have to do this copy and paste thing. And so then we developed the OU login

26:00 plug in, which you just give it the host name of your OpenUnison, and it automatically logs you in. You don't have to have a preconfigured Kube control configuration. The other priority we had was short lived tokens. We really wanted to make it so that those tokens didn't live more than a minute and to make it transparent to users because, you know, I mean, that the great thing about tokens is is that they they scale really well. The bad things is if you lose it, bad things are gonna happen. That that's that's a real risk when it

26:34 comes to bearer tokens. So and by the way, we here. Woah. Yeah. And we we we get in the book. We get into an obscene amount of detail as to how that works and why. And it's actually a free chapter. So if you check out the the GitHub repo, so no registration, nobody knows who you are. You just download the PDF for that chapter. But yeah. So, you know, we we we wanted to make it so that you could very easily use short lived tokens transparently, and we want to make GUIs easier. Like I said, I love the dashboard, so we

27:13 wanted to make it easy to securely use the dashboard. I'm I'm a big proponent. And in fact, again, in the book, I spent an entire chapter on the security of the dashboard because it's really important to understand how that security works when people say, oh, the dashboard's insecure. No. It's not insecure. It's insecure how people deploy it. You know, you're using Kube control port forward or you're using Kube control proxy. Yeah. It's insecure. You know, you give it an RBAC, you know, a privileged RBAC credential. Yeah. It's insecure. But, you know, you use it properly with

27:48 the reverse proxy. No less secure than anything else, especially you had multifactor authentication to it. You know, you're in good shape. So yeah. So, you know, the the the real benefit of using OpenUnison is just being able to quickly get into clusters, being able to integrate your enterprise authorization because most enterprises have some kind of authorization system. You know you know a product does its job real well when a massive public company that does identity management uses our little open source project for integration with their Kubernetes clusters. So, you know, that that told me that

28:31 we were probably on the right track for what how to do it. So I know I've spent a lot of time talking. You wanna get in some tech? Yeah. I'm excited. I I wanna I do So yeah. So you've got a few clusters ready to roll. Right? I do. Let's cover the what have I prepared in advance? The answer, as always, is pretty much nothing. Yep. So today, we have the OpenUnison documentation. This is available at openunison.github.io. The GitHub repository is available at github.com/tromolosecurity/openunison. And we have a couple of clusters today, kindly provided by our friends at Civo. So I have three

28:47 Preparing the Demo Environment

29:15 Civo clusters. OpenUnison one, OpenUnison two, OpenUnison three. We have a Docker desktop in Manikip, but they're not as fun. So what's on these clusters? The answer is nada. All I have done is use simplified app management to display engine x ingress with a load balancer. So these are fresh, clean, happy to be OpenUnison's clusters. Sound good? Sounds good. Alright. So in the documentation here, we have some nice bullet points telling us that OpenUnison simplifies access and increases security to things that all Kubernetes clusters need. And let's get do we just go jump straight into deploying the syndication portal? Is that step

29:50 Deploying OpenUnison Control Plane

30:01 one, Mark? Yeah. Let's rock. Alright. Okay. Prerequisites, ingress controller. Check. Site specific configuration. This is something we'll need to do at some point, I guess. And well, Danny, to install all your controllers, I'll look at it. As you go through, I I kinda wanted to give people a a heads up as to what you're gonna do and then go through all the the individual steps. And you're saying we can do all this in thirty five ish minutes? Yeah. Quicker? Usually, I hope quicker than that. Yeah. Alright. Okay. So I'm gonna skip the ingress controller. I deployed NGINX. Good.

30:46 We're gonna deploy the Kubernetes dashboard. Nice. Let's do it. I'm just trusting. Yeah. That looks relatively safe. Yeah. Yeah. We'll go with that. So site specific configuration. So this for our helm chart. I see there's some values that yaml mentioned here that we can customize. So let's click on that, do a bit of a copy. And I'll just leave this here for as we need it. And then we can deploy the portal. Alright. Let's Oh, you skipped a step. If you if you scroll up or nope. Go back to I'm just gonna add that. Oh,

31:10 Configuring the OpenUnison Control Plane

31:36 okay. Gotcha. Alright. So did I miss the step? Yeah. You did. The the three boxes there, how to customize your default values. Alright. Okay. So okay. I should read the docs, shouldn't I? Sometimes I just get I can't read the way in this. So okay. So the first thing says, do we want to enable impersonation? You wanna give us the do you wanna give us the TLDR in these options? Yeah. Sure. So OpenUnison will work whether you're running a on prem cluster where you can change the the API server configuration flags for OpenID Connect. If you're if you're running

32:22 an on excuse me. If you're running an on prem cluster, that's always what I recommend as as, you know, bypass the the impersonation proxy. But if you're running in a managed cluster where those that's not a possibility, like Civo, AKS, you know, GKE, any of these other ones, enable the impersonation proxy. And what that does is it launches a Kubo IDC proxy that acts as a impersonating proxy. So kubectl actually interacts with that proxy, and then that proxy interacts with your API server on the user's behalf using Kubernetes impersonation. And then, like, if you look in the

33:05 audit logs, it'll actually say that, oh, this this action was performed, you know, by David via impersonation. Okay. So you're keeping all that information in there. The network, this is where people tend to get hung up the most, mostly because it's the first thing you tend to deploy inside of a cluster is your authentication system a lot of times. And so they they haven't quite wrapped their heads around how Kubernetes and networking works. And so here's you know, it's just three settings. Yeah. What do you want your host names to be? It's just the it always has to point

33:44 to the load balancer for your cluster or more specifically for your for your ingress controller. Mhmm. And then finally, authentication. Inside of the values dot YAML, you're gonna find commented blocks of all of the different authentication mechanisms. You uncomment one of them. You go through the steps in the documentation, and you go ahead and configure it. You know, we try to we try to make the documentation as single path as possible. So even though you diverge a little bit. So, like, in the network session section, you've got the OpenUnison host, dashboard host, and API server host.

34:29 You're gonna want those to point to your load balancer. So as an example, I'm a big fan of nip.i0 for development. So if you just did, like, katesi0dot, you know, ip.nip.i0 or whatever you wanna call it. Okay. You need to put the IP address in You have to put the IP address in there. Yeah. It doesn't have to be dashes, but you could do that. And then the next one's gonna be your dashboard host. And then the last one is the host name for your impersonation proxy. So does it work with dots? I thought it had to be dashes.

35:15 No. It works fine with dots. I actually prefer to do dashes only because it looks more real, I guess. But, yeah, it'll work perfectly fine either way. Alright. I'm gonna make sure that I have not messed this up, and we'll make sure this resolves to the IP address that we expect. Yep. So perfect. Right. Now most enterprises will usually have, like, a wildcard cert that they're using. So you can configure this to do that. We actually include a mini cert manager that will generate all self signed certs for you. So if you're just getting started, you

35:54 just wanna get the thing up and running, you don't have to have, like, cert manager deployed ahead of time. We're gonna we're gonna generate those certificates all for you. Yeah. I guess it's important for people to know that when you use a service like that or net is that Let's Encrypt will not give you a certificate for those domains. So you have to do something else. Exactly. Okay. So let's the network embed looks fine to me now. Right? It's good to me. Yep. Okay. The OU, I'm I'm assuming we don't have to tweak this. This is just for

36:22 the x five. Yeah. I mean, if you really wanted to, you could. Alright. I'll tell you what. Well, at least the I'll go with the academy. There we go. Yeah. And they're all self signed certs. And Tremolo. Okay. We don't need to configure the image, cluster name, impersonation, we I would actually change the cluster name because we're gonna talk about multicluster in a little bit. So, like, I usually call the first one, like, control plane or something like that so that you know it's your it's your primary. Okay. I just call OpenUnison one to match the

36:54 cluster name from Excel. We're using JetStack's OIDC proxy. Although, looks like you've got your own build of it. Is that tweaked in some way? Yeah. So we took over maintenance of the OIDC proxy about a year ago, I think. Yeah. JetStack, you know, it's a JetStack created a great project. We found that there were a couple of things that were missing that that we found would be really helpful. And then we also wanted to just get it to the point where it's being maintained and actively tested and updated. All the libraries are updated. So one thing

37:35 that we take great pride in at Tremolo is making sure that whenever we release software and you look at those known vulnerabilities, it's as close to goose eggs as possible. And in fact, we have continuous scanning of our containers. Everything's built on Ubuntu. So whenever Canonical releases patches to CVEs, that automatically triggers rebuilds. So we're we're constantly pushing out fresh versions. Alright. Okay. Let's run through the rest of these values fail then and get this thing deployed. Nothing here in the dashboard looks like looks like it needs changing. The the only time you really need to

38:14 change that is if you have a custom deployment to the dashboard or the if you use the helm chart for the dashboard, you gotta adjust a couple of those values. Okay. I'm assuming we leave this false to your cert manager. True. We just use the Kubernetes basic cert stuff, I guess. Yeah. It originally, we thought it would be cool to use Kubernetes on built in certificates so that everything was automatically trusted except we found, a, it wasn't automatically trusted. And, b, on x on, you know, managed clusters, you often couldn't even get access to it.

38:51 Like, that private key wasn't there, like EKS and AKS and whatnot. So we just always disable it now. And then I think that you yep. So our on the enterprise side, active directory and OIDC are definitely our largest implementation base. Mhmm. I always thought SAML would be more popular. It never was. People have just skipped SAML and seem to go straight to OpenID Connect. Yeah. SAML was the thing for, like, two minutes, and then it just everything's like A little longer than two minutes. You know, it what really kinda changed things was the pandemic, to be honest,

39:37 because that's when organizations really shifted from using their own on prem active directory federation services, which is only SAML, to Azure AD, which supports both SAML and OpenID Connect. And because you know, SAML's really great in in really segmented networks because you don't have that back channel communication that you have with OpenID Connect. But in when your identity provider is on the Internet and it's Azure AD or a doctor, who cares? Okay. That's fair. That's fair. I'll I'll I'll buy that. But, yeah, it it it it definitely has never been as popular as I thought

40:23 it was gonna be. Yeah. Alright. So we're we're gonna go with GitHub. Right? Yep. Mhmm. K. So that is the GitHub application? Yep. So you're gonna go into your either an organization or you can do it on your personal settings and create a a auth app. Alright. I'm not sure if I can I can't remember it? Flashes secret and stuff, so I was gonna do it. Is it okay to do on on screen? I mean, I would just delete it afterwards. Alright. Okay. You know, people in chat or who are watching live, please be good neighbors.

40:59 Yeah. One day, I'm gonna learn to stop flashing tokens on a stream. Alright. Let's go. Waff apps. New. Open. Unison. Rawkode. And is this gonna be my net IP? So that's gonna be the if you go to your network section, it's gonna be OpenUnison h h t t p s, your OpenUnison host value slash auth slash GitHub. Auth slash GitHub. And we're I'm I'm only telling you what's already in the docs, so you're you're not getting anything secret here. Alright. So you got that client ID. That can go into the values dot YAML. And I'm assuming it's like a secret and a

41:50 secret. No? Maybe? You're actually just gonna put the secret in a file, and then OU control will create your secret for you. Alright. And you got the client ID in there? Okay. Cool. It's there. Other policies, I'm assuming we can skip that. You can skip that for now. I mean, we give you a lot of customization points. And then, you know, if we have time to get into namespace as a service, that's where you get into that. But you don't need a data. The the thing with OpenUnison is we say, only implement infrastructure you need for what you're gonna

42:24 do. So because the authentication proxy doesn't have any asynchronous workflows or self-service in it. We don't make you to deploy a database. We don't make you deploy anything for notification. So we wanna keep it as simple as possible. Only deploy what you need for what you're gonna do. Got it. So that looks good? It does. And then go ahead and create a file with your GitHub secret in it. Might as well display Russell's comment just now. Alright. Safe. Nobody got it. Cool. Alright. And so you already deployed that. So you gotta download almost. You gotta download the OU control command for

42:50 Using ou-control for Deployment

43:37 whichever platform you'd like. Is this m one ready? Yes. It will I'm assuming you have Rosetta installed. I do. Yeah. Okay. Dash o o u c t l. No length. Oh, it must be a redirect. So dash f f. That's weird. What if you just do a w get on the ah, where'd you go? Was just making sure I could actually write oh, no. Wait. I'm just being Oh, you're in the can are you in a container? No. I've just got my parameters the wrong way around. Failed to receive data to disk. Alright. I'll do it in my home directory.

44:26 Maybe that temp just has or is it small or for rate to fail? It may just be Honestly, I just use w get. Oh, yeah. It's small or Yeah. I like w get because it gives you a little progress bar. Alright. I'm old school like that. Oh, I can't type. Alright. Okay. User local. Is that hate typing my password on screen. At least that one, we can't see. I know. But I was doing it once on a stream with Alex Ellis, and sudo crashed as I was typing. And then my password was just all over the

45:08 Of course, I mean, I I don't I don't duplicate any passwords, and it's only my local machine password. So it's not a big deal. But still Now I haven't done it yet, but do you have the Touch ID keyboard? So I I type on a good keyboard. But Oh, you've got the mechanical. Okay. But I keep this little thing to the side purely for thumb, for my fingerprint. Gotcha. I I haven't done it yet, which is weird because I live by Touch ID. But you can set up sudo to use Touch ID on on Mac.

45:42 Nice. Need to look into that in one day. You can also I I know I can do it on my watch, but then I always find it weird while we're tapping it as well, which is anyway, we're taking our rest. Let's let's get back on track. So we've got all your control. Progress? So, yeah, so you've got all your control. You're ready to rock and roll. Alright. So we're going to install, point it to our secret file, and then the YAML file. Yep. So the YAML file is our values. Right? And the secret file is my please don't copy.

46:15 Exactly. Please don't copy me dot text. I think I called it. Yes. No. Don't getting there. Well, no. That's Don't copy me, please. I should've called it Yeah. I seem to remember there was a please in there. Alright. So is this going to create a Kubernetes secret, web our secret our client secret for the GitHub app and to help deploy into our cluster? Exactly. Oh, no database section. Check out the values. I have a oh, I know why. Shoot. Okay. See where it says enable provisioning is true? I gotta fix this. I was messing with

46:59 it. Kids, don't mess with the website right before the the demo. It's in the OpenUnison section. Yep. Yeah. So change that to false. And then on the next line, change that to true. Go ahead and run. I gotta fix that right after this. Here we go. Yeah. So I decided not to tell people to make the secret because what people were doing was they would do echo secret into base six you know, pipe base 64. And so everybody would be telling me, it's not working. It's not working. It's not working. I keep getting access to nine. Well, yeah,

47:36 because echo adds that that slash m at the end. And so I was like, you know what? I'm just gonna take that out and and and I didn't wanna have it on the command line because I hate having secrets as command line parameters. So so yeah. So we we actually embed Helm into this command. So it's just running the Helm charts, but the nice thing is is that it's paying attention to the cluster. Like, it's waiting for things to finish. It gives a little pauses. I I I love Sivo to death for especially for my testing clusters. I do find that when

48:13 you're very heavy on watches and you're very heavy on CRDs the way we are, because everything is configured through a custom resource. It I've I've found Cboe to be a little more persnickety than most other clusters I've worked with. Okay. So you said, just while we're in, everything is configured by some sort of custom resource. So you can get up Pretty much. Entire OpenUnison deployment then as well? Oh, yeah. Yeah. We've got we actually have we haven't formally released it yet, but we have a Argo version that uses waves. So you create the secret manually

48:55 and you point Argo to the special repo that's just a, you know, just a sync of all the correct repos, but with the wave not label. What's the other thing besides the label I can put on object? Thank you. Annotation. I know Kubernetes. I promise. With the correct annotations with the right waves, and so that way, you don't use the OU control command, at least for that instance. So this is running. Yeah. I'm gonna do some investigation. So It's a it's a slightly larger pod, and it's just pulling I think it's coding images. Yeah. Yeah.

49:51 Yeah. That that that's strange because I mean, that CUBO IDC proxy is only 45 megs. The Orchestra one's pretty big. So, hopefully, think it I think it gives up after a thousand tries. Alright. Structured that that top is, which is weird because oh, you're you're you're probably running your cluster cluster in in Europe. You know what? I don't even think I paid attention to where I stuck this thing. Not that it should matter because it's all coming from Docker Hub. Well, it doesn't tell you a region by default. Where where did I put it? I'm looking in your bottom left.

50:43 Oh, yeah. London. Yeah. So I don't know. Maybe maybe they just don't have good networking over to Docker Hub from London. Yeah. Take a look at the events. Is there a Let me Is there something that's not mounting? Oh, we're missing Oh, there we go. Secret volume, orchestra not found. Do a k get nodes or k get secrets? Yeah. Okay. Take a look at your there's a pod that's called OpenUnison operator. Take a look at that. Did we miss something? Or I'm sorry. Look at the logs for that pod. Yeah. Finally. Just gave up. Oh. Alright.

51:47 Where oh, interesting. It had a problem talking to see see, this is why I mean where it'll it'll sometimes have weird issues with civo's API server. Okay. Let's do this. Just go ahead and rerun the command again. So one thing about the o u control command is it's designed to be rerunnable. So if something does fail, you can always rerun it with the exact same parameters, and it will check to see if thing so, like, the secret already exists. So let's say so let's see here. There's Orchestra secret source. So right now, it's redeploying that.

52:38 Oh, there we go. Yeah. There we go. Okay. Cool. So, like, one nice thing about OU control is, let's say you're just doing an update to your configuration, but you're not changing the secret. You don't need to have that secret file. It'll say, oh, the secret's already there. I'm not even gonna you know, unless you specify a new secret to rotate it, I'm just gonna leave it alone. Like, it it builds some of the intelligence that an operator would wanna have a human operator would wanna have during deployment. There we go. Alright. I'm assuming that's gonna be much quicker

53:15 now since it's Yeah. I mean, you're already running. So it's we're just waiting for it to get to the point where it's ready one one. There we go. Now the old one. And you can see, like, it's saying, oh, I'm gonna wait until there's just one pod because that's the number of images I want. I don't wanna release control until we're at that point. So we're gonna wait for things to to terminate. It's waiting for the eventual consistent to be consistent. Right? Exactly. You know, which is something that, like, Helm doesn't do. Right? Helm doesn't

53:57 you know, if it fails because of a webhook, it just fails, which is where my whole eventual consistency is a lie comes from. Yeah. I have that all that problem all the time. I mean, admission admission controllers are definitely a problem, but finalizers are the ones that really get on my my nerves. Although, I had fun this morning with a customer with pod disruption budgets. That that that really caused some heartburn. Combination of pod disruption budget and having to rebuild a cluster that had lost its authentication token. Oops. I'm gonna try and speed this up a

54:38 little bit. There we go. When in doubt, force. Yep. And so the next thing it's doing so what you've done right now is you've deployed the Orchestra pod. So that's got a very simple baseline configuration. Okay. And then yeah. And here we go. So joys of webhooks. Let's go ahead and run it again. It should be much quicker this time because I was just waiting for the certificate. We probably had that too quickly. But I forgot what I was gonna say. What were you talking about before the oh, admission controllers. Yes. And and finalizers and pod disruption budgets.

55:39 Yeah. I mean, I love admission controls. I mean, we use admission controllers because we've got very complex CRs that just can't be can't be they just they can't be validated just through schema. Like, even the new whatever the the new schema validation language in '25. You know, there's that new scripting language that you can embed into your schema for CRs in one dot 25. It's not Turing complete. Yeah. I I can't, yeah, I can't remember the name of it, but the idea is is that I think it's a beta in 25. It's either an alpha or beta in 25. It's

56:32 not GA. But it it lets you it gives you a little more capabilities around validating schema rather than just open API. I'll try one more search before I give up on it, but I'm gonna have to look into the Combiner. That's what it is. Or no. Combiner is the name of the release. There is in there I cannot remember the name of it for the life of me. Then again, I couldn't remember when annotation was, so I don't know why anybody would listen to me. I thought it went to '25 as a beta. I don't know. We'll find out later. And

57:14 probably that's have to be honest, I've just not paid attention to the 01/25 release really. I I because I've not upgraded the cluster to it yet, but that is my next plan. Gotcha. So alright. Let's get back over to our dogs and here. It's shutting down. I'm still trying to delete that pod. I'm assuming it doesn't respect the sec term or it's just not got a program. It does. It's just the it it's cleaning up itself. I do find that it takes a while to to shut itself down. Although this does seem to be taking longer

58:02 than usual, usually within about thirty seconds. Maybe I should've sprung for some bigger nodes. What what size did you go with? Four CPUs, eight gig around. Nice and bad. That's what I've been using. Yeah. We've got a probe failure for check alive. I mean, it's unhealthy. It should be dying. Why won't you die? Oh, there we go. Alright. Now I just just needed to restart before it finally Yeah. It looks like something was going on there. Yeah. I've gotta spend a little more time. I I figure Civo's probably exposing an underlying issue that I need to to

58:59 look at when it comes to the way that we're oh, but it's there. It's waiting a few seconds. So it waits about ten seconds. Alright. Alright. So Looks like we're in business. So if you take a look at your pods actually, you should be able to just log in. So, obviously, there's no it's a self signed cert. Yep. So you gotta let us see see all your unmentionables. I'm pretty sure I used that org. Yeah. You used Rawkode Academy as your authorizing organization. Cool. So you're in. So now you click on that dashboard, and you'll be in. You won't be able to

59:17 Accessing the Cluster via OpenUnison Portal

59:56 see anything because you still have to enable some RBAC policies. So, yeah, if you look at the bell in that upper right hand corner, you'll you'll see bunch of RBAC errors. And if you look at your little human on the right hand side because you're using impersonation, it'll actually have your GitHub username. I wrote that. So, yeah, if you grant now the nice thing is is you can grant RBAC based on your GitHub works. So you're not granting RBAC directly to a user. So if you go back to the documentation, I actually have a example in there.

1:00:40 So scroll you actually, on the right hand side, see where it says GitHub RBAC bindings? So you wanna copy that and just change the group from Tremblay security owners to, you know, whatever you want, really. Alright. So we're saying all owners here get cluster admin and the cluster. Yep. I call this RBAC. So now you go back to your dashboard, and you can see all the errors are gone. Man. Maybe not. I may have to log in back in? No. Click on the that little bell thing again. Is forbidden. Interesting. Go back to the main

1:01:28 OpenUnison screen. And then in the upper left hand corner, you'll see your name Rawkode. Click on that. And I actually can't read it. It's really fuzzy. But do you see Rawkode I don't see anybody called Rawkode owners. That's probably the problem. So if if you wanna just be Also that You wanna just do Rawkode slash or my you know, any one of those. Would that work? Just Rawkode? No. It has to be slash. So slash nothing means you wanna let the entire organization in. Alright. Let's just do the cluster team because I I trust me.

1:02:14 Is that is that a smart move? No. So okay. Now it's all gone. So now you are, in fact, a cluster administrator, and you can do everything to your heart's content that the dashboard lets you do. Stuff like the terminal works beautifully. You wanna terminal into things. So and then we've got all sorts of little controls in the back, like, you know, if you log out of the main portal and they'll they'll break you out of this. So you can't leave the the main portal. You know, you can't leave the main portal but still get into the dashboard.

1:02:59 You know, most enterprises have policies around things like idle time out, stuff like that. And then if you wanna access your cluster directly using kubectl, you go back to that main screen. You click on home, see that Kubernetes token. Wow. I think my Internet fast. Struggling. Well, I do stream on a fry a five g connection, which is usually pretty solid. Really? Yeah. That's impressive. The fiber company for this building wanted an extortionate amount of money, and I said, no. I'll go with five g. And I actually get, like, 500, six hundred meg down and,

1:03:08 Accessing the Cluster via kubectl

1:03:46 like, 300 meg up. Well, from what I real I think y'all Europeans are much harder on your telcos for marketing nonsense than we are here in The States. I think here as long as it's not, you know, one g accounts as five g. There we go. There it is. Okay. So it took a second. So now you've got a kube control command and a Windows kube control command. And if you actually look, there's a little set of boxes next to the word kube control command. It makes it a lot easier. And so you take that. And what I

1:04:18 would do is don't use the same kubectl configuration so that way you don't lose your you know, potentially overwrite your existing context. It's alright. I'm using Kubi. So all of my Oh, okay. They're all different files. So Gotcha. Well so yeah. So what's gonna happen is you paste that in, and it's going to generate a kube control configuration file using the kube control command. So it's gonna go to whatever. So now you can see if you do a kubectl get nodes. Notice. Yeah. There you go. You now have access to everything. And so the left the phone down here.

1:05:05 The yeah. You're right now going through that that Kubo IDC proxy, and and the reason we use that is because Kubernetes again? There is the Who am I making is that a plugin? That is a plugin. I I think the API was included in '25, but they haven't added anything to kubectl yet. I would say do something you're not allowed to do, but you're a cluster admin. I know. I've got everything. So So you've got everything. But the nice thing is is that you didn't set up any certificates. You didn't set up any host names. It

1:05:48 all just worked. Now really cool thing is, why don't you set up a watch on, like, kubectl get pods or whatever? Well, could you help me understand what actually happened here? Yeah. So Is this set up as service? You drew two owl two circles, and then you had Kubernetes. So what happened was if you look at your if you go back in your command history to the actual command, it's gonna be this big giant thing. I'll just scroll up. Yeah. There you go. So what we're doing is we're actually using all existing control configuration commands. Like, the you're doing the you didn't

1:05:58 How kubectl Access Works (Proxy & Tokens)

1:06:40 download anything to actually run this. And so the idea here is that we wanted it so that you didn't have to deploy anything to access clusters other than kubectl. So what do we do? We set the server up with the correct host name. Now in this instance, that host name will be our API our API proxy. So you're not talking directly to the Cboe API server. You're talking to the Kube OIDC proxy. We set up the certificate for it. We set up the OIDC configuration, so where the issuer is, the certificate for the issuer. We and then we went ahead and also

1:07:22 set up your initial ID token and refresh token. So now you can just go to your heart's content, and kubectl is gonna continuously refresh that token until one or two things happen. Either you idle time out or if you log out of OpenUnison, it will end your session. So the security folks in enterprises love that, that there's a way to actually kill a session in within about a minute. And that's just by revoking the refresh token. Right? Yep. And then, like, let's say, you know, David's gotta be walked out for whatever reason. Know? He's gotta be fired again. Yeah.

1:08:07 It it you know, it's Tuesday. Right? You know, you go in and the session is stored as CRs, so you just delete all of David's sessions, and within a minute, those tokens are gonna stop working. So I think it's called I think if you do, like, a kubectl get o I d c dash sessions and OpenUnison, you'll see them. O I d c dash sessions. Yep. So you've got a bunch of sessions open. You delete those, and then within about a minute, your your token will become useless. So as an operator, you wanna you wanna

1:08:55 kill somebody's sessions. Alright. Need to look at what it is. Sure. Now I am the first person to say you should never store anything sensitive in a CR. I break that rule here because this is all encrypted, and RBAC protected. So if somebody has this, unless they also have the decryption key, they don't have access to anything. Yeah. I think it is binary encrypted and then base base encoded. No. I don't know. I see the equals, and I think maybe base Yeah. The that one's just base 64 encoded, I think. But that has to match up with stuff in

1:09:46 the encrypted ID token and the access token. Nice. I you know what? There's there's so many benefits from storing stuff and custom resources because I get to hook into all of the tooling that I already know. Yeah. And I I just think this is a really, really sleek way of doing it. This is very cool. It works really well. We used to do it in a database. And back when we first started it and once CRs became a thing I remember when I first said, you know, how would I store custom information in Kubernetes? This is this is before CRs were even

1:10:25 available. And I was like, you know, would I just talk to etcd directly? And they're like, no. And I go, why? He goes, you know, you know, are you some kind of masochist? You wanna talk to etcd directly? You you enjoy pain? And so the the CRs, I think, are are really going to become a big part of what makes Kubernetes your your data center operating system rather than just a pod control management system, a container control management system. I really hope that the API machinery team add some sort of sensitive or encrypted label for custom resource fields so that even

1:11:04 that can be handled by the API server. But I mean, I've I've got a whole rant on secrets management in Kubernetes. But now the cool thing is a lot of people don't like what we did with copying and pasting that token. Mhmm. And so if you go back to our documentation, there is a or I don't know. Do you have a crew installed? The crew plugin installed? No. I don't. Oh, okay. So we have a a a crew plugin where that actually let a kube control plugin that lets you bypass that whole thing. So you'll see a picture of it here where,

1:11:16 Introducing the ou-login kubectl Plugin

1:11:52 you know, you run kubectl get nodes. Okay. It's not authenticated. This was the first time I ever made a GIF out of a video. I think this took me about four hours. But you see that pops open the the browser window you log in and and similar to, you know, if if you're working with any of the cloud CLIs that pop open Nice. Or the the OU login. But the the cool thing about that is is that there's no preset configuration options at all. There's nothing to distribute because that that's one of the hardest things to do in an

1:12:30 enterprise is distribute things, keep it updated. I've got a bank that's probably my largest Kubernetes customer, and they've held off on just this, the OU login plug in for almost two years now just because it's a pain for them to go through the submitting something to be distributed across the bank. Mhmm. You know, it's gotta get scanned. It's gotta get you know, there has to be a deployment plan. You know, there's a central repository. You know, you can't just download stuff and run it. You've gotta go and get the official thing. So we wanted to give you the option

1:13:15 to do both. Now we you know, it's it's quarter to two. So I I have all the time in the world. Do you wanna try adding a cluster for multi cluster? Yeah. Let's do it. Okay. Cool. So grab your your yep. Go right there. You got it. So what we did with multicluster. So there are two ways you can do multicluster with OpenUnison depending on your level of comfort with Kubernetes. You can do it where every cluster gets its this happens regardless. Every cluster always has its own OpenUnison. And the reason why we do that is

1:13:17 Multi-Cluster Setup: Introduction and Architecture

1:14:03 let's go back to we were talking about silos. Right? If I have my own cluster and I want to add Argo CD to it and do SSO into Argo CD, I don't wanna have to go to you as the cluster manager, as the Kubernetes team, and say, please integrate my cluster. I wanna integrate into my cluster's OpenUnison so that I can do SSO. And so everybody gets their own OpenUnison. Now you can either set up each OpenUnison to go back to your main identity provider. So, like, this would be basically repeating the process we just did for every cluster,

1:14:47 which is a little painful. But if your main Kubernetes goes to or your main cluster, your control plane cluster goes down, it doesn't take out everybody else. So it limits your your blast radius. The major downside to that is if you don't own the connection up to your identity provider, that can be painful. And and, again, silos and enterprise, the people who own authentication don't generally aren't the same people who own your Kubernetes clusters. They're different people. They care about different things, and they have a backlog of people who want onboarding, you know, from here to the other ocean.

1:15:26 So, you know, it doesn't matter which way you go around. So it it's a trade off of, you know, of blast radius versus flexibility. I've seen a tendency to move towards the flexibility side in enterprise more and more because people are getting more comfortable with with low downtime Kubernetes, maybe not zero downtime because somebody's selling you zero downtime. They're probably lying. But, you know, low downtime Kubernetes, you know, planned downtime Kubernetes. And so this now means that I can now grant authentication to new clusters. I can automate onboarding new clusters whenever I want. And so that makes it

1:16:14 easier to automate. You know, we talked about small clusters versus big clusters. Well, if you do wanna go that small cluster route or virtual clusters, you can now have every cluster has its own authentication into that control plane, and you can control it yourself. So it gives you the best of both worlds. So in order to do that, you treat your control plane OpenUnison as an identity provider. So that means, you you know, that work where we did where there's a client secret and you've gotta exchange that trust, that's not rocket science, but it's manual task

1:16:52 that no one likes to do. Well, everything's API enabled, so why don't we just automate that? So that's what we did was we want to make it so that you enter as little information as possible. So the the first thing you do is you do this piece where you go back to your main values YAML. So we're still on the control plane. And if you go down to the OpenUnison section, see where it says well, you wanna get rid of it. But Oh, alright. There's already a show portal org. It's false. So you wanna change that to

1:17:27 true. Alright. And then go ahead and save that, redeploy OpenUnison using the yep. And notice you're not putting any of your secrets. I am a professional. Oh, yeah. I know how to copy and paste your documents. I don't. So, you know, I assume no one else does either. So this is update in our the OpenUnison deploy that we have in our control plane cluster. It's gonna add something to this this thing. Right? So see where you now see there's that little bump where it's got that little tree there? So now it's breaking up the badges

1:18:07 so that's a little more organized. Because once you start adding clusters you know, you got a dozen clusters. You don't wanna have 24 badges all over the screen. It gets difficult to organize. Plus the other nice thing is you're probably not gonna give access to everybody to every cluster. You know, usually certain clusters, only certain people will have access to. This makes it a little bit easier to organize that. So while that's updating, we can actually get started on the next piece, which is there's a new OpenUnison values YAML. And what's important about this particular file is

1:18:40 Configuring the Satellite Cluster

1:18:47 that it does not have it it does not have an authentication section because we figured that out for you. We already have all the information from the control plane. So why ask you to put in information that we already know, you know, on our side. Mhmm. So if you go ahead and you start editing that Alright. So I need the IP address for cluster two. Yep. Okay. I can do this in the command line. Alright. So 2 and then Yep. Beautiful. I can do that. Hold on. Oh, you you got better Vim skills than I do.

1:20:06 Dev.io. Boom. Alright. Damn. If I knew that, would have passed my c k a d a lot more easier. I'm assuming we don't need need to modify anything else. Right? This is just Yep. Everything else is good. Do we need to point it to cluster one? So so this is where the beauty of of Kubernetes comes in. Right? So you're gonna set up the you're gonna so you've downloaded let me think about this. So the way I typically do this is I don't use Kubey myself. So I create a single kube control file with contexts to both. So what I'll do

1:20:52 is I'll take the Bootstrap contact Bootstrap file, and then I'll make that my main kube config file. And then I'll log in to my control plane, run the run the onboarding command, you know, that that the kube control creation command, and that gives me a unified kube config file. So our control plane is gonna reach out to the other clusters to to So you you you wanna have your your your control plane file and your or your your bootstrap for your satellite cluster. Okay. And do the And then overlay 51 9 need to match? Like, the No. No.

1:21:34 No. We do all the certificate exchange and everything. Because, again, we know what those certs are, so why bother asking you? Okay. So what do I need to do now? I need to So you wanna be pointing to your satellite and install the dashboard. That I can do. Yep. That part's easy. And so then the last part is creating the COOP config and then running the OU control satellite install. Alright. Okay. So OU control install satellite. This is now cluster two dot YAML. So let me Yep. Did the other deployment finish? It Almost. Did it actually finish or did it fail?

1:22:36 It failed. Oh, there it goes. Yeah. Oh, okay. That's fine. I think it was just the webhook again. Yeah. Okay. So this is going to be my So that's gonna be your control plane configuration, and then the second one is gonna be your satellite configuration. Yeah. That's gonna be And we're using the Go SDK, so I don't know if that's gonna work. Because we only I don't know. Because doesn't Kubey just wrap? All my configs are in different files, so we're gonna have to merge. Yeah. What I what you can do is so this is how I do it. I

1:23:22 set kubeconfig to the bootstrap for the satellite. Mhmm. And then I log in to my control plane, get a token, and in the same kubeconfig, run that so that way I get a merged configuration. Okay. So let's jump back. Sorry. I've made this really difficult now by using Kube. Well, it's it's it's it's an interesting problem that I need to take a note on because one of the things that we're working on is simplifying this because we always wanna make it easier. So, you know, it's a good point. I I tend not to use Koobie.

1:24:09 So, you know, is there a way to make it easier when somebody is using it? Okay. So we're now using this. Yeah. It could be config. So now if you go to log in to your control plane and pick up a token Oh, I just did that. Okay. And then paste it in. Alright. Shit. Okay. Cool. Onto cluster two. On my control plane, and I wanna grab Run the kubectl command. Uh-huh. So now if you run k c t x, you should have more code control context. So yeah. So you've got now two. Perfect. So now your

1:25:01 your OpenUnison one context is your control plane. OpenUnison two is Gotcha. Your satellite. So that's what you're gonna use. Too many steps ahead of me. That's what it was. Yeah. Alright. So we wanna do open Rawkode or OpenUnison one. Nope. Just OpenUnison dash one because you want just the context. And then open OpenUnison dash two. So now if you look at your values dot YAML, we actually updated it for you. So what we've done is we've created a client secret inside of the control plane, big, nasty, random monster, so you don't have to think of it

1:25:15 Deploying OpenUnison on the Satellite Cluster

1:25:54 yourself. And now we're deploying OpenUnison as a satellite directly into your satellite cluster. Yeah. I'll be able to look at it in a minute. I also make things really difficult for us on this session by always using temp directories for everything that I don't know the path to. Yeah. That makes sense. But Because I have a directory history, so I never like to be in a real directory when I do demos because then I have all the commands in my in that. Right. Yeah. No. That makes that makes sense. So yeah. So now what we're just doing

1:26:31 is we're installing OpenUnison the same way. It's just if when you look at the values dot YAML that you create that you created, you're gonna see that it is it now has an OIDC section. And that OIDC section points to your control plane as your identity provider. So can I see these satellite clusters as custom resources on the control plane cluster? Not to the easiest way is to actually do a helm list in the OpenUnison namespace. And you'll see there it is, satellite dash Kubernetes satellite three. So that is your satellite cluster and all the resources associated with that

1:27:25 chart. So it created a identity provider. It created a like, if you do kubectl get applications in the OpenUnison namespace on your control plane, you'll see that there is a cluster IDP Kubernetes satellite. Yep. That's where we onboard all of your satellites onto. Do a kubectl get trusts? No. I don't think it's there yet. Oh, no. It's just Okay. Now it might be. Maybe not. Not yet. Yeah. There are too many objects to keep track of. But if you now go back to OpenUnison, See where you have that Kubernetes satellite? Now if you click on it

1:28:08 Accessing the Satellite Cluster Dashboard

1:28:14 and you go to the dashboard, it's gonna make you click through a bunch of stuff because we're using all self signed certs. And boom. Now you still have the same problem with a little bell because of RBAC, but the important thing is is you now have a satellite OpenUnison deployed, and you didn't have to do anything with OpenID Connect or GitHub or anything else. But I can just swap between these different different clusters like so. Exactly. And then, you know, we haven't really talked about other applications, but, you know, like Argo or Grafana or anything else. You can now integrate that

1:28:56 here. You know, you can you can integrate it into your control plane or into your into your satellite. Now the other cool thing is the OU control. If you click on, like, click on Kube control's satellite tokens so this actually runs its own OpenUnison. So you can go directly to this host too if you wanna bypass it. So you've got that option as well. So, like, the OU control command that I was talking about, you know which hosts you have access to. You just type in OU control dash dash host equals my OpenUnison, pops up the browser, you log in, you

1:29:43 close. So you don't have to go through this rigmarole. And then, you know, of course, if you've got, you know, a CA that you wanna use, you can use that instead of having to do the self signed certificates. You know, we we wanted to just make it as easy as possible to get started. This is really cool. I like this. Thank you. So what we haven't really talked about is so we we we've we've done the authentication portal. We haven't really talked about namespace as a service. Do you want to deploy namespace as a service?

1:30:11 Namespace as a Service (NsaaS) Introduction

1:30:22 Well, I I don't know what namespace as a service is. Do you wanna get me the the pitch? Yeah. So namespace as a service, the idea is you can log in to a self-service portal like this, and there's a little button that'll say request or there'll be a button here that says new namespace. Click on new namespace. I enter some information, like what I want the namespace name to be, what I can add additional attributes, you know, for things like chargeback and labeling, and then I hit submit. And then the cluster owners say, yay or nay.

1:31:00 And then OpenUnison will actually provision that namespace. It'll do a few things. It'll provision the namespace. It will provision the RBAC bindings. If you want to use external groups so, like, let's say you wanted to use your GitHub teams to manage access, it'll map those for you. And so at that point, you now have a namespace, and you can also customize the workflow to add things like network policies, resource quotas, etcetera. So now as a business owner, I've been able to get a namespace without running kubectl. And as a cluster manager, I've been able to create a namespace without

1:31:44 running kubectl. So that that's the first part of it. The second part of it is who has access to it. So we allow you to do either internal or external or both. And by that, I mean, you can let OpenUnison have a set of groups for you and manage it that way. You can rely on an external identity provider like GitHub or Active Directory and groups there, or you can do both. And we find folks that use this functionality, they always start with, oh, I just wanna use the enterprise groups. And then they realize, oh, it's a lot

1:32:18 harder than we need when there's an emergency to onboard someone, and so we wanna use the internal groups too. And so what that then gives you is the ability to do self-service to get access to those namespaces all without having to do what you want? All without having to do any kube control commands. And then we keep an audit log of everything that's happening. So, like, at KubeCon twenty twenty Europe yeah. It was twenty twenty Europe. That was the one that got delayed twice. Right? Yep. It was all virtual. I did a session on learning RBAC called

1:33:00 I can RBAC and so can you, where I used OpenUnison with Fairwind's RBAC manager to let people create namespaces without actually having to go through any approval process, where you created a team which would provision the RBAC. Are you familiar with what Fairwinds RBAC manager does? Yeah. Okay. So for anybody who isn't, it lets you do access management via labels inside of namespaces. Really great tool. So you would request instead of namespace, you would request a I forget the name of the object, but it's basically a Fairwinds RBAC object that would define what labels namespaces would have, and you would have access

1:33:44 to those namespaces. And then at that point, once that got approved and provisioned, you could create all the namespaces you wanted associated with your Fairwinds object. So as a business owner, I could create all the namespaces I wanted. And as long as I had access via that Fairwinds object, it would create the namespace and set the appropriate labels. So it's a really powerful tool. My favorite thing that I'm working on right now is virtual cluster as a service, which is really cool. So if if you're not familiar with virtual clusters, it's it's essentially a k threes that runs inside

1:34:11 Virtual Cluster as a Service (VCaaS) Concept

1:34:25 of your cluster, and it synchronizes objects and gives you a smaller world view. So you think you have your own cluster, but in reality, you're still running in a large multi tenant cluster. So Loft, the folks that make vCluster, recently released the cluster API for provisioning of this. So what you can actual what we have is this way where where this portal I can show it to you where when you request a namespace, there's a little option. Do you want a namespace or do you want a v cluster? You pick v cluster. We provision the v cluster.

1:35:03 We provision OpenUnison into it, and we link it via SSO. And now you have a v cluster ready to go with all your RBAC bindings, and that's it. Alright. Well, I would love to get home from my kids' bedtime. So I'm gonna share your screen. Okay. Cool. Give us the d cluster demo and we can take a look at it because it sounds pretty awesome. And then Yeah. We'll wrap this up because I reckon we could talk about this stuff for for a couple of Alright. Can you see my screen? Yes. We can see your screen. Thank you. So

1:35:23 Virtual Cluster as a Service (VCaaS) Demo

1:35:44 I'm gonna log in. Don't save. So I've I've deployed a couple of these, but I think I deleted them, to be honest. So this this link didn't exist. This request access link doesn't exist in your OpenUnison. Yep. And this badge, this new Kubernetes space badge doesn't exist. So first thing I'm gonna do is I'm gonna open up the dashboard, make it a little easier to follow what's going on because there's a lot of moving pieces here. So we got our dashboard up and running, and I'm gonna go ahead. I'm going to say create a new namespace.

1:36:18 So this is our namespace as a service. It's multicluster. So this has just a single cluster for management. And I'm gonna do test Rawkode because I'm really original. And I'm gonna choose v cluster and test. So that's amid the request. Now I've gotten an email that says, hey, Mark. Somebody's requested. Oh, cool. And so this gives you the current roles. We're we're in the midst of making this a little bit better because there's just more information you're gonna wanna have. Now some of the fun things you can do here with custom approvals is you can do escalations.

1:37:01 So I'm not doing my job. It's been a week. You know, escalate it to my manager or escalate to someone else or automatically close it. We're also doing things around capacity management where if you request the namespace and there's a certain amount of open capacity, just let it through. Don't bother requesting it because we're tied into the cluster. We can do all that fun stuff. I'm just gonna go ahead and approve the request. So while that's going, I'm gonna go ahead and show you the reporting. Right? So we're keeping track of all this stuff. And when it comes time for the auditors

1:37:39 to come by, you know, you can give them access. You can generate the Excel spreadsheet. All that stuff's being tracked. So let's come to the dashboard. And if we go to the OpenUnison namespace, let me go check out pods. See here. Did it launch? Didn't. Alright. So first question is, did I make the appropriate did I make the appropriate sacrifices to the gods of live demos? That's okay. Sometimes this takes a little while. Oh, wait. Actually, first, let me see here. Test. Oh, this test raw code. And okay. So it's actually deploying our virtual cluster. That's right. First, it deploys the virtual

1:38:35 cluster, then it launches a a helm chart that launches a job that integrates it. So you can see it's actually building out the dashboard for us. And this is actually our OpenUnison. So what's actually happening now, you're seeing it in real time, is we've created the virtual cluster. Now we're integrating that virtual cluster into OpenUnison. And so if we look at these pods, we're actually looking at our our whatchamacallit? Our virtual cluster being stood up. And so this is a little bit of a slow moment, but what's happening is behind the scenes is we wrote a custom hook into our

1:39:22 onboarding workflow that uses the cluster API that Loft built for v cluster to create the v cluster. Once and then we just wait for that v cluster to to be done. Once that v cluster is done, we then return execution to go ahead and kick off that job. So now if I look at this, I'm pretty sure that's running. So if I go to the OpenUnison namespace, I look at pods. So this is still running. So this should look familiar. Right? So this is that automated. We we we we took a we took a pod, and we put kubectl,

1:40:14 v cluster, and Helm in it. So that way, we could automate all this stuff. So all that pain you went through of setting a context, we did that automatically with the vCluster. Right. So now see. Do I have access to it? I don't. So I'm gonna go ahead and log out and log back in. So now I've got a new v cluster dashboard. Now the dashboard actually doesn't really work too well because v cluster by default doesn't deploy metrics. So we're still working out the kinks here, but you can see we we've got it automated so you even see which dashboard you're

1:41:02 in. And then when you get a token I didn't really think this through, did I? Let's see if I can set this up so that you can see what's going on here. I'm gonna make this the same size. And then if I switch my sharing woah. That's cool. How do I get out of this? There we go. So I'm going to stop sharing, and I'm gonna share that. That makes it a little bit easier. Can you see that now okay? Can you drag the height down a tiny bit of your window? Oh, this? Like that?

1:41:51 You're sharing just a single window. Right? Yeah. Yeah. So if you just make it less tall, that that's it. Yeah. Go for it. Yeah. So I'm right now logged in to the v cluster as the logged in user. So the the the really nice thing about this is I'm interacting as me. Now if I went back and I I don't wanna give everybody a headache by switching Windows again. But if I go back and I look at the Kubernetes audit logs, it shows I'm logged in as Moseley to the virtual cluster all for free. Nice.

1:42:32 It's all automated. So I'm actually working with a customer. The end result of this is something I call an ephemeral cluster where the idea is going to be to create a completely disposable cluster that you never have to debug. So that automation, in addition to the OpenUnison, will deploy an Argo CD. It'll deploy a GitHub instance, link everything up for you securely. And that way, if you get to a point where the cluster is irreparably broken, you can time box and say we're gonna spend two hours trying to fix this. Okay. I give up. Nuke it, but it rebuild itself from source.

1:43:18 That is very cool. I mean, that's not just namespace as a service. That's an entire cluster as a service. I really like what Loft are doing with v cluster and being able to hook that in with this, like, resource request process through OpenUnison to get people access to stuff. It's just it's really awesome. Really, really cool work. And in fact, Rawkode on the chat is doing the same. Icing on the cake for a great tool. And he's right. Like, everything we've seen today. It's one of those things that, you know I I spend up a lot of clusters.

1:43:43 Conclusion

1:43:49 I work with a lot of clusters, security is the thing that I just always hide and stray from because it's hard. So having tools like this where we can make them the first thing we deploy and get that identity and access problem just done and dusted out is awesome. So I I mean Awesome. Thank you. I appreciate that. That means a lot. Sweet. So any last words? Anything you wanna share before we finish up today? I think we're gonna have to schedule a part two and do something else together. I I think we can come up with pretty Pretty cool.

1:44:24 I I've got some interesting ideas or something we can build together. Oh, yeah. Definitely, man. I wanna hear them. My I I I I I think everybody's ideas are far better than mine. So I really wanna hear what you know, I I I love being able to to take other people's ideas and make them reality. So yeah. Absolutely. Hey. I'm really easy to get a hold of at MLBIM on Twitter. Same thing in the Kubernetes Slack. I'm always there. Feel free to hit me up with a question. I love answering questions. I love helping people out.

1:44:58 Again, I'm gonna do the the shameless self promotion. Amazon. Amazon dot co dot u k. Where is it? There we go. Amazon.co.uk. Yeah. It's it's up there. You know, I I will say that I have been absolutely intimidated by the idea of going on cluster date. It's I see what people pull off, and I'm like, yeah. That makes perfect sense. It never would have dawned on me to do that. You know? And I just I I would love to, but then people might realize that I'm I'm I'm I'm a sham. I'm I'm more worried that we could deploy

1:45:37 an open to the cluster, deploy an approach to cluster and save the cluster and set up the local KubeConflict to point to that. Like, that's what I'm scared of. But that that would be a lot of fun. What I gotta do is I gotta see if I can drag my partner in crime on the book, Scott Sarovich, on with me because he's like, when it comes to to living with these massive headaches of debugging clusters, he knows it even better than I do, much better than I do. But yeah. But I've had a blast. Thank you so much for

1:46:16 doing this and having me on. I'm having a lot of fun, and I look forward to having fun doing more stuff. Well, I really appreciate you taking the time today to kinda walk us through OpenUnison. It's a super powerful tool. It solves a whole lot bunch of problems, and I hope that a lot of people find this very useful. So thank you very much, Mark, for joining me. I'll reach out to you afterwards. Like I said, I've got ideas. I'm gonna talk to you. But until then Definitely. Have wonderful weekend, and I'll catch you soon.

1:46:42 You too, David. Adios. Adios.