0:00 Introductions

0:00 If you would all be so kind, and we'll start with you, Eli, and we'll just kinda work around in clockwise order. If you could tell me your name, who you work for, and anything else that you wish to share with the kind people listening to this podcast. Hi. Yes. I'm Eli. My pronouns are they and them, and I'm a developer advocate at Scaleway, which is a small European cloud provider. We've got data centers all over Europe, all your all your dev tooling, all that good stuff. And before that, I was developer advocate at a small tooling company, developer tooling company.

0:32 And then before that, I've done bunch of stuff, data engineering, started out writing c straight out of university. So spent some some time in in the trenches of sort of real real programming. Yeah. I like knitting and Tolkien and hanging out with my cats and generally being paid to be a professional nerd. So that's that's me. Hi, everyone. My name is Abdel. That's my nickname. Actually, if you look me up on LinkedIn, you will find probably my long Arabic name, which a lot of people have difficulties with, so I just go by Abdel. It's easier for everyone.

1:08 I am a cloud developer advocate at Google. I've been at Google for almost ten years now. So I'm a cloud developer advocate focused on Kubernetes and service mesh and pretty much anything containers. I cohost the Kubernetes podcast. Maybe some people here are are listeners. If you have been listening since October, I am one of the annoying voices because we are two hosts of it, actually. The other host is my colleague, Kathleen Fields, who lives in Seattle. So, yeah, I do pretty much everything around containers and Kubernetes. My upbringing is kind of weird because I did software engineering as a master's degree, but

1:41 I never really worked as software engineer. I worked in data centers, which is kind of a place where not that many people actually work, I guess. And my first job actually was in data center back home in Morocco, and then I moved to Google to join data center in Belgium. And then as I like to say, I've been moving up the stack in terms of structural layers. So all the way from hardcore servers to virtual machines to containers to whatever AI is gonna bring in the next ten years. So so, yeah, that's me. Awesome. Thank you. And

2:10 Neil? I'm Neil. I'm CEO and cofounder of Fortainer. I like to describe myself as an engineer because to think otherwise would horrify me. I have been twenty five years building, breaking, and fixing things. VMware engineer, IBM guy. I dabbled a little bit with development. I I am embarrassed to say it's Visual Basic, though. So I don't know if it's if that's actually development or not. But started out with basic, so that that that also shows my age. And I like to make things easy. I hate complexity. I think complexity is the root of all evil, and I like to make things easier

2:51 for people. That's just my natural born hobby. Actual hobby is is fishing and hiking for those that know me. So I love being outdoors. Alright. Thank you very much. So this is the first episode of the Cloud Native Compass, and we're gonna keep today's topics broad as you're all aware. We're gonna kind of take a look at a cross section of the Cloud Native landscape and that being clouds and containers and Kubernetes. But I prepared, three or four questions at each of those categories. But, of course, Adrian, Anurag, and anyone else who wants to join, if you have

3:22 any questions, please feel free to pop them into the chat. And if you really want to, feel free to raise your hand, and we will promote you and invite you on to ask your question live. So we're gonna start with the first question, which I know everyone's gonna love because it's a topic that a lot of people find frustrating but important, and that is the cost of cloud adoption. So based on my own experience of going to events over the last kind of twelve months, maybe broader, I see that the the cost of their cloud instances

3:25 Cost of Cloud

3:53 or their hard their software services that they're using on these cloud providers seems to be growing and growing and becoming more and more of a challenge, especially in the macroeconomic climate right now where costs are a big problem. So I'm curious if this is something that you're seeing with your own experience, your own customers, your own partners, and if you have any advice or best practices to help people navigate this tricky situation. I can have a comment. We have a lot of infrastructure in the cloud. It's actually our entire QA environment. We obviously have to spin up

4:27 Docker standalone environments, Docker swarm environments, no man environments, Kubernetes clusters of multiple different versions. So we we we support three versions back. And so every single time we release a new version of Plotainer, we've gotta spin up environments and test something against it. So we've got we've got environments that come and go frequently throughout the day. So we we might spin up a cluster as live for for an hour while while some QA testing is, and then it's pulled down again. And we do this across almost every cloud provider you can possibly imagine. We currently support six cloud providers that we

4:58 test in QA again. So we've got we've got instances everywhere. That gives me a pretty good insight into how much money each provider costs are relatively the same thing, you know, because it's it's always the same, environment that's set up across the providers. And it's eye watering how much money we spend. And one of the things that I've kind of figured out is if you have a relatively consistent workload, like the CPU and RAM allocation is consistent and you can right size your environments, it's probably pretty good in the cloud. If you've got environments that are very, very bursty,

5:32 and you would be able to benefit from from oversubscribing physical VMware hosts or Proximox hosts if you wanna do things a bit cheaper. If you can benefit from oversubscribing and making sure that you can time slice, it's it's probably substantially cheaper. We're actually gonna be moving a lot of our workloads back from the cloud into a cloud bare metal as a service provider running virtualization, and we think it's it's gonna save us quite a bit of money in that in that landscape. So, you know, it definitely, the cost differential between the top three and everyone else, I think, is still large. There's

6:10 there's a large gap. You you pay a premium for the top three versus everyone else. Yet I think the capability gap is narrowed. You know, previously, if you if you if you wanted a really reliable service, you went to one of the top three. And if you wanted something a bit more niche and more cost effective, you went to went to anyone else. I think that's definitely narrowed out a lot now. You get some very, very reliable services from DigitalOcean, LeanOut, Civo, you name them. You get some very, very reliable services super cost effectively now. So that's my my my views

6:41 on it. Anyone else like to add anything there? My my flippant answer was gonna be if you're on Amazon, then you can always hire the duckbill group. Get Corey Quinn in to come and jot down your bill. Right? It does make sense to me, though, that making sure that you are buying exactly what you need is is sort of is an answer. Might even be the best answer. Like, that that feels right. I I think early bit me to it, but I I was about to say there are people who built businesses on top of this idea of cost optimization,

7:11 and is probably the best example of that. Right? They help people optimize costs. And I think there are few factors that that determine, like, how much you're gonna spend in the cloud. And I'm I'm I'm I am in my line of work seeing more and more people kind of, like, caring about the cost. There is if you have been to KubeCon in Valencia last year, there is, an entire trend around FinOps that have been starting around, and, know, there are a bunch of booths of companies doing basically financial optimization for companies. So there are a few factors.

7:46 One factor is is and this is something I've seen I've seen in my in my experience is that that people just turn things on and forget them. Right? So as they go around to a cloud console, they turn a feature on, they never look at it. There is also what I call hidden costs. Sometimes you would turn a feature on. I I don't know. Azure virtual machines. But there is an underlying login that you are charged for, but you don't see it. You have to go dig into the documentation to understand how those logs are

8:12 built. I have had this before with Google Cloud, specifically customers coming to us, and they go, well, our login bill is much higher than everything else we are spending money on. Right? And that's because they're just writing logs, and logs cost money. And you can you can basically, anything that is a hidden feature that gets turned on automatically without you knowing, that could be a problem. As Neil said, like, the the the the margins between a row compute across the three premium cloud providers is getting very narrow, and the the other nonpremium cloud providers, if you want to

8:47 call them so, are getting also very close, like DigitalOcean or VH. They are getting they're they're starting to beat AWS and Google Cloud on pricing. Right? And so what what what I'm noticing on my end is lot of what a lot of cloud providers are doing, they are moving toward kinda more added value services. Alright. Let's instead of just selling you raw compute, we're gonna give you, like, containers of service so you don't have to worry about all the underlying complexity of the infrastructure. There is one comment I want to to to add, and I'll just stop here.

9:17 I think when we talk about costs, a lot of times people like to compare apples to apples where you say how much CPU memory cost me on prem versus how much it cost me in a in a cloud provider, and they tend to forget the cost of engineering. Right? When you are paying Google Cloud to run your virtual machines, you're not only paying for that virtual machine. You're paying for all the engineering work that goes behind it that makes the service reliable. And of that engineering work, you would have to do it yourself. It would if you

9:40 would hold the VM yourself. Right? So if you start thinking about your engineers as an investment center rather than a cost center, then then it's entirely different discussion around cost cost optimization. Right? Yeah. I think there's a couple of things there. One, you mentioned that there are products popping up now, and I think that's that's very true. We're seeing Terraform integrations that try and tell you on a pull request what your plan or the changes to your your infrastructure's code is actually gonna cost you on a rolling monthly basis, which is interesting. There's things like KubeCost, which try to protect what a

10:11 Helm chart can cost you when you deploy it in your cluster, which again is quite interesting. And I think something else that was interesting there that you mentioned is, like, the the understanding that you've got the operational cost versus the capital. What is it? Capital expenditure and operational expenditure. So as a company, you have to make the decision. Do you want to own hardware and be an operator of hardware and provide a service, or do you want to be a little bit of a premium and just use one of these cloud providers? And I think these are a lot of

10:40 challenges that people need to make. Oh, I was just gonna say that, you know, owning your own hardware does come with its own operating cost as well because who who's wearing that who's wearing the pager? That's one of the things you get with a cloud provider. Right? It's it's not not you. Definitely in a a production environment. I know when I started writing dev or being a dev, like, we had a cupboard that had a rack in it. It had four servers on it, and that was dev and staging and QA and preprode. And to us, like, it doesn't really require

11:11 much maintenance. If anything went wrong, you pull the plug on it and turn it back on and then and then, well, was barely else. Forty minutes later, maybe you had a working machine again. Who knows? But we're now in a situation that we're probably engineered for ourselves, and the cloud native predominantly means microservices. Right? And running microservices and a production lake environment, even if it is in production, usually requires more than one machine. With our model, if you could spin up a bare metal server and a cupboard or even just some not a Raspberry Pi, but, know, you can

11:40 stick some novelty hardware into a cupboard and deploy a binary to it. But with microservices, that's not really an option anymore. It's like we've engineered ourselves into this cost parity, which I'm I'm sure there are pros and cons too. But what what's people's thoughts there? Like, if we made this mess ourselves? Well, it's also just be careful. You don't you don't do binary alternates. Right? So, yeah, like I said, we we're moving to bare metal as a service and a cloud provider. So we we we we get we get an entire server in a managed data center from a provider

12:11 with high levels of SLA and Internet connectivity and everything else. So we we just get an x a console access to it. We can install VMware on it or Proxmox, whatever on it and and start putting our environment. So, yes, we have to manage the hypervisor layer as a cost saving, but we're not worried about the data center or the server or any of the connectivity requirements of it. It's highly available. So that that there's actually a nice halfway house as well in between fully managed by by a cloud, you know, hardly managed by a cloud or

12:37 self hosted. And, you know, I I don't think I would ever even consider reverting back to fully self hosted now. It's just it's just too too hard now, you know, and there's there's no need. But definitely, being able to to have your own dedicated hardware, being able to overcommit that hardware at a ratio that that you're comfortable with, that makes sense for your business, then then that'd that that's definitely a viable option to to save money, lots of money. Yeah. I mean, there's quite a a public migration right now happening with Hey, the email company, and Basecamp,

13:13 where they're they've they've given up on the cloud and said it's too expensive. And they were saying cost, I think, was it, like, 3 plus million per year to run their email service? And they've went all in and buy their own hardware and sticking it into a data stream or in managing that themselves. And you're right. I I I could never imagine going back to that level of of operational complexity. I I I like the simplicity of play. I like the fact that we can burst up and burst down based on what's happening. I mean,

13:41 the companies have predictable load to be able to do that. Like, it just doesn't seem like that's the case anymore. Well, I have I have to say something about this whole base camp and hay situation. I mean, all all respect to the person who wrote those articles, there is one thing I am not able to understand is at that scale at $3,000,000 per year, they have a very powerful negotiation power. Right? They're not paying premium prices. They're not paying catalog prices, or they shouldn't be paying catalog prices at at least. Right? If you are a customer of a cloud provider

14:12 who whose bill is $50,000, if you bring AWS or Google Cloud, they will tell you that. They just go away. We don't care. Right? But you're paying $3,000,000 per year, then you can sit down at the table. And I've seen customers negotiating up to 60% discounts on their own computes by spending couple millions a year. Right? So I I I read I read those articles. I could not understand why the cost was substantial and why they were complaining about it. And, also, I if I was in their shoes, I would not consider going back to

14:39 the run my own server situation. And Neil said something interesting. Like, bare metal server is a thing, and it's it's kinda halfway between, you know, fully managed and somehow half managed, I would say. So so I I don't like, the articles that you're referring to, they were doing kind of an apple to apple conversion. That's what I always advocate against. Right? Like, just look at the catalog prices of AWS and Google Cloud and and Azure and say, like, oh, see, this is expensive. I can run the VM for half that price. That's not the full story.

15:10 Yeah. I do wonder if at Scaleway, we will see more people coming back to we have elastic metal service because historically, that's been just steady. There is steady consistent demand for those. So it'll be interesting to see if more people adopt, you know, your your approach, Neil, of mixing and matching and finding the right tools for the actual sort of more more well suited tools for each job. Alright. Well, based on where this conversation is going and based on what the next two questions are, I'm gonna probably gonna ask them both and just throw them out there and let

15:42 you tackle whichever one you want to. You know? But the next question is, again, from going to conferences and speaking to people at the conference, I've seen a lot more people adopt cloud providers outside of the big three. Like, you know, we have Azure, we have AWS, and we have Google. But, you know, I think what a lot of people are agreeing with is that costs are cheaper when they go to the smaller cloud providers. They're also more targeted and seems to be able to innovate in a certain space faster than the major players. So that could be

15:45 Clouds Outside the Big 3

16:10 a driving factor there, and I wanna kinda touch on that. But you also both or all three of you mentioned kind of hybrid architectures. And if I think back to early COVID and even before COVID, there was a lot of buzz and talks at KubeCon and other events where people were talking about hybrid cloud and multi cloud. And I'm not seeing that anymore either. So I'm curious if you say that. Yeah. So maybe you can share some thoughts on are the smaller clouds an advantage, and are we gonna see more adoption there? And is there a hybrid play, a multi cloud

16:42 play that is disappearing, or maybe I'm just not seeing it anymore? Well, I'll do a tiny product picture, which is that Scaleway does run a multi cloud Kubernetes managed service. So we have your control plane and everything, lives on Scaleway hardware and architecture, and then you can put your nodes wherever you like. So we are definitely we're hoping that the multi cloud isn't dying as a concept. I think it's something that you know, maybe it will never be mainstream, but there will be use cases for it. And I think it should be easier for people to do.

17:14 Yeah. Hybrid cloud, I think, is or or multi cloud, whatever you wanna describe it there, I think it's actually pretty important because it does let you have the right cost for the right application. Like, why would you wanna run dev environments on the top three and pay premium if you don't need the high levels of service availability and everything else that comes with the top three that you just get and take for granted? If you don't need that for dev test QA staging, what do call it, what you like, then why would you run them there? You

17:43 might wanna run them somewhere that that's that's substantially cheaper. That's one of the great things about Kubernetes. Right? It's this this unifying layer. As long as you can deploy in Kubernetes, you can deploy anywhere, and you're gonna get relatively the same result. Now I say I say relatively because, you know, the the cloud providers like to make you somewhat sticky, and so they're trying to encourage you to use their authentication and their logging and all their other stuff. And if if you if if you buy into that and go all in on Amazon's authentication, you know, I am everything else, then it

18:13 makes it very difficult to then go and use any other cloud provider. So as as long as you have some way of, you know, normalizing or or neutralizing how how you authenticate users in the clusters, how you handle RBAC, how you handle all of the policies, everything else. How you do all that and don't don't use it from just one one provider, then you can you can balance your loads and have have the right the right provider for the right workload. And I think that's that's key. Otherwise, it makes it very difficult. That does require a pretty decent working knowledge

18:43 of which provider would be the best for each workload. Now if you're if you're conceiving a project from scratch and you go, okay. I have this I have this very high GPU usage that workload that needs to happen. You know, there's there's if you don't already know off the top of your head where do you want to run that, that's sort of that's research time, I suppose. But then that's you're saving money as a result. Result. So Well, just just just have two. High cost, high SLA, low cost, lower SLA, and just just balance that way. Prod, nonprod. Just even start even

19:12 start something simple as that. Right? I'm I'm gonna have to go a little bit against the flow of what's have been said. I I understand all these points. They're all very valid points, but I think one of the problems I have with hybrid cloud and multi cloud discussion is it tends actually to always focus on a single layer in a stack. Like, what Ilya and Nain mentioned, it they they just talked, oh, the Kubernetes, you can do Kubernetes across cloud because it's unified API. But in reality, no application lives in a vacuum. You need the database. You need storage.

19:40 You need messaging queues. You need a bunch of extra things. You're not gonna stick your app on AWS and put your database on Google Cloud. That doesn't make any sense. Right? You're not gonna put your your application on a container on DigitalOcean and have your Kafka running on AWS. That doesn't make any sense. So where where where I see this this again, this is my own opinion. Where I see multi cloud valuable are few of the following cases. Some of the following cases, this is from experience. First is location or availability of the service. There

20:10 are certain industries where you are required by law to be in a certain physical location. And if your cloud provider doesn't have presence in that physical location, then you will have to choose another one. That's one valid use case for multi cloud. Right? Or maybe either physical location because regulations or because you want to be close to the user from a latency perspective. Right? So that's one. The second one is you you you touched on this a little bit is added value services. Like, if you want analytics tooling, probably Google Cloud is one of the best

20:40 places where you can have, like, a, like, a, a data warehouse with BigQuery. So you go to the cloud provider because they have a service that they are the only one offering, and they are doing it very well. Right? If you want cheap compute, you probably go to AWS because they have been doing that for a while. And they optimize even their costs to be able to offer VMs for cheaper than the other two. If you want developer tooling, you go to to Azure. They have a pretty good DevOps stack, which a lot of developers like, especially if

21:07 you're in the dot net space. Right? So that's another use case of multi cloud in my opinion. This, I would call it, world where people want to run a single abstraction layer on top of six cloud providers and have a single API to talk to them and then have the app automatically move from one cloud provider to another because, you know, it's not that that data center is down. That's just very hard in practice to actually implement, I would say. So I'm not dismissing that multi cloud is valid. I just think that people think about it from a single

21:40 layer perspective in the entire stack, and that's probably not the right way to do it, but that's just my opinion. I think you're all in more agreement than you probably realize because I think, Abdul, what you came out of from there is, like, the production use case where, you know, I don't wanna manage my own stateful databases and production. So I am gonna use, like, Cloud Spanner or Cloud SQL, etcetera. But then I think Eli and Neil were more of the let's slice our environments across clouds. Like, have dev and staging live somewhere else and

22:09 not really speak like, they're never gonna use I'm not gonna pay for for Cloud Spanner for my dev server or my staging server because the cost it's just not cost effective. But I probably would stick it on Scaleway or Civo or Linode or any of these other providers and maybe rely on Google Cloud for my production workloads because I do wanna who can to more of those value added services. And I think that's actually probably a pretty good way for people to start leveraging cost savings by moving those lower priority environments or more ephemeral environments than

22:41 other providers. Plus it gets them more familiar with the tooling as well. Maybe the infrastructure's code is gonna improve because there's no retit in a modular fashion where it can be augmented with a different cloud provider at one time. Who knows? Like, everybody's gonna do something a little bit different. But I do think these were all in more or less agreement in some way there. We agree? There are also open source, you know, alternates for almost all of the cloud provider services. So for non prod, you can you can actually stand up an entire Kube manifest with,

23:14 you know, your your database layers, your your your your messaging layers, whatever else, and actually make it make it stateful. So you can do that. Would you do that for production? Yeah. Probably not. You'd probably want that that with a really high SLA, you know, specialist, you know, sort of run databases amazingly well. So but yeah. Yeah. We're seeing some really interesting stuff like CockroachDB with their serverless offering. It's it's really interesting. We've got Neon dot tech, which is doing, like, this managed serverless Postgres as well. There's PlanetScale who do the test MySQL at a cloud environment. So, like, if people are

23:47 already stored in their state and these other services and just using cloud providers for compute, maybe they have more flexibility to really understand and mitigate any cost explosion there too. So here is one interesting fact for you guys. I I don't know if you've seen the you you're all aware of the last the the the registry migration, the abstract registry migration for Kubernetes that's happened over the last few months. Right? I've been so we we we did, an episode on the podcast about that, and we interviewed one of the software engineers who led that effort.

24:18 And in researching that topic, I came across a document that was written by some people in the CNCF in which they analyzed like, they they they just, you know, highlighted what's the problem statement and how they are proposing to solve it. And in that problem statement phase, they said there was something around they they analyzed the traffic that was going toward the Google Container Registry, so khs.gsr.io, the old one. And they realized that about 70% of the traffic going there was coming from AWS IP addresses. So so there's actually a lot of people on AWS that are stand so these are

24:50 not EKS. It's not it's not the AWS managed Kubernetes pulling from our container registry or from the the k x s one. It's people standing their own Kubernetes cluster on top of AWS using tools like KubeADM or KubeSpray because KubeDM and KubeSpray uses the old registry to pull images from. Right? And that was a very interesting, like, fact to learn. It's like, okay. There are bunch of people doing their own Kubernetes cluster on top of AWS, which is Why would you? Why would you then? It's so weird. You already have the expertise. You want to

25:24 be lower in the abstraction sort of offerings. I I don't know. There must there must be significant use cases for it if they're seeing that much Did I run my own? And so, actually, the migration toward the new registry was driven by this because the the the project the Kubernetes project used Google Cloud to host the artifacts and host the CICD pipelines, etcetera, etcetera. But they realized that out of the $35,000,000 that Google Cloud was giving them every year to pay for everything, they were paying 50% of it on egress costs. That's all the image pools going toward the different cloud provider.

25:50 Do we trust Docker?

25:58 Right? So the new registry, if you look at the details of the implementation, it's just a very simple proxy that forwards you toward either AWS or to Google Cloud based on where where you're coming from. So they're actually hosting images in both cloud providers so that they can cut cut on the egress cost. That's so clever. That's really funny. I had good idea. Yeah. It's it's it's it was a very interesting, like like, realization. Like, 70% egress cost going towards AWS, which was like, okay. Sure. But I'm not sure people doing that. Well, I think this gives us a nice

26:29 segue into our next topic because registries are expensive, and people that operate them may wanna register costs. And a very notable company made some decisions lately that kind of angered the cloud native community. And, of course, I'm talking about Docker who kinda made themselves a villain when they announced that they were deprecating, removing, deleting, whatever they were doing, the free plan, which meant that a lot of people who host their container images on the Docker Hub were told that within was it thirty days or ninety days? I don't remember exactly. Their images were gonna be deleted. Now they've

27:04 resented it. They said, okay. We're we're gonna migrate people to open source programs, and we're gonna give I think there is some sort of free plan now, but do we forgive them, or have they broken their trust one too many times now? I think there's definitely gonna be fear because if you can just change a service with only thirty days notice, what else can you change? It it is the price for other services gonna be 10 times this next week because they choose that they need to change it. So there's there's definitely gonna be fear

27:32 as a result. You know, history often repeats. So if that is if that is becoming a behavioral trend to do these kind of things, then I think there'll be fear for what will happen again in the future. Most most providers have EULAs that say, we actually reserve the right to change us at any time without without any warning, and that's part of their their standard agreement. So I think there's gonna be fear. I don't know whether there's much more impacts, but if you can think back when they changed the licensing for Docker desktop, and it

28:06 was a huge uproar on that. But Yeah. They make they they're making millions from Docker desktop licensing. So did it have that much of a negative effect on their business? The inverse, it had a massive positive effect on their business by making that change. So I don't it it's hard there. At the end of the day, if you if you're an open source entity, you have to be commercially viable to to continue. They have to find ways to monetize and, you know, could could could could have been, you know, come to better? Absolutely. I think this is where I struggle. Right?

28:37 You know, I think Docker changed the game for everybody in this community. Right? We all have a lot of respect for Docker and defend the team and what they've done. We want Docker to succeed. But like you said, it's not the first time they've violated their trust. Like and and they've done it so catastrophically through very poor communication, not really bringing people along with them with the changes in, hey. This is why we're doing this thing and this is what we need to to keep our business alive. They've just came out with a big delete

29:05 hammer, the big sledgehammer. Went, sorry. It's going away now. You know, in the Docker desktop one was another one. We've seen so many alternatives pop up now from Rancher desktop to Podman desktop to Colima and all this other tooling trying to get people alternatives. Even recently, Orbstack released the product as well to kinda compete with this. I I don't know where I am on the trust radar here. I I still want Docker to succeed, but I'm not pushing any more images to Docker Hub. Well, we we we just we just pay the money and a a Docker verified publisher because, you know,

29:38 we get immense value from their technology, and it just feels like it's a moral obligation to to help them as a company succeed and grow and keep keep creating the technology that's that's changing the game. You know, everybody relies on the Mobi project that's almost entirely funded, you know, by by Docker, and that and that's the effort. It just it just felt like the right thing to do. It's interesting. I don't know if you saw Morentis when they started monetizing Lens. They they they went through the same kind of thing that Docker did with desktop, and they also semi surprised

30:14 the industry and said, hey. By the way, you know how how it's open source? Yeah. It's not really. The the the version we publish is actually our closed source version, so the license changed. And there's there's also equally, you know, strong backlash, and that's why now this OpenLens came out of came out of nowhere. So it it's kind of you gotta be super careful monetizing open source and do it, you know, very, very gracefully, gently, and transparently. I mean, is OpenCourse still a feasible business model for OpenCourse? That's what we are. I think there's different

30:43 there's ways and ways of doing OpenCourse. Right? I mean, when if you OpenCourse part of your your business logic sort of thing, do you do you accept pull requests from people outside of your organization, or is it publishing sort of broadcasting rather than accepting incoming contributions? So for us for for us, we actually accept PRs into our our Portainer c version. We we we got two two code bases, Community Edition and Business Edition. They're they're they're linked behind the scenes, but we accept PRs against the c edition, and then we will we we will, ourselves, backport

31:19 it to business edition if we so want. So, yeah, we absolutely do. Right. So and then if you so want, so there would be PRs in the community edition that are not present potentially in the business edition. Do you then see those two things diverging? And at what point does what's open no longer match what's in the core as it were? So I use so so if if we so want, they're probably a little bit too openly. Everything is currently and CE is in business edition. In the future I ask is because my previous company ran on the exact

31:48 same model, but we didn't really find ourselves accepting PRs. I would I would accept PRs all day every day. It it it makes the product better, and and why why would I not? If if someone raises a PR against our open source product that competes against the business product, I'd still I'll I'll still consider and accept it. All I'd say is then, obviously, there's not enough value add for that particular feature in the business edition, and we we need we need to be more creative and to come up with with with a new value add.

32:18 I think that's what makes a very healthy open source variant is that you don't make it a walled garden and close, and you control it. It has to be open. And if it's not open, then it's it's it is an open source. I I I would agree. Yeah. I think that this whole Docker mishandling communication in the last one, and it has happened a couple of times, and it happened to other companies. Redisyn was one of them when they changed the licensing a few years back. It happened to a bunch of other companies. It's always reminds me of these

32:46 quotes of, like, this is why we cannot have nice things. It just that's basically as a community, we are entitled to free stuff. We just think that whatever we use is free and should remain free forever. Right? And when a business goes like, oh, no. No. No. No, folks. This is not gonna be free because we have to make money on this. We are paying people and offices and, you know, engineers and everything like that. Then everybody goes goes crazy. And forget the fact that, well, you're probably using stuff for free, and you're probably not

33:18 even contributing to them. Right? So it's a very hard thing to balance. Like like like like, balancing open source with commercial is very difficult. I think that the last one the last Docker announcement was probably more a case of mishandling communication than anything else. Right? One of the things I was thinking about as that whole situation was kinda unfolding is as a community, if we think that there are a bench of stuff, containers, for example, that are important. Like, one of the person who was very vocal about the Docker change was Alexi, the OpenFast founder. He wrote, like,

33:52 a bunch of articles about it, etcetera. And for me, I was thinking, like, why don't we, as a community, use the same model of the CNCF in terms of open governance and decide, okay. All these open source projects that are important enough for everybody will be hosted by an independent entity and paid for by everybody. Right? So why don't we have a CNCF equivalent or CNCF themselves that manage and runs a version of Docker Hub that is free to use for everybody who wants to use it, but somehow everybody have to contribute to it either financially or through through

34:25 your contributions. Right? And this is probably a farfetched idea. It would take time to unfold, and I'm I'm not sure even it will happen. But I just we tend to forget that relying on a single provider is never a good idea anyway. Right? I can give you an example actually of a project I worked on, which was even funnier than this. I'm gonna So there is a database called Arrow Aerospace something that's like an in memory database, very popular, that one of my customers was using. And they were using a version of it which has a commercial license, but the free edition

34:57 allows you to run up to, like, 32 nodes cluster. Right? And you combine them in a single, you know, cluster. And the company behind came up with the new version, and they said, okay. In the new version, the open source edition, you can only run four nodes clusters. And if you want to run more than four nodes, then you have to pay for it. The that company, that customer I was working with, they're already running at max. They were already running 32 nodes. And what's even worse is that that company k said, okay. We're releasing version four,

35:24 and three months from now, we're deleting the binaries for version three from our repositories. So my customer ended up dumping the binaries for version three and putting them in a Dropbox. So they can see they cannot they they have the binaries going forward. Right? So it's I don't know. It's very difficult. I I also don't know how to feel about this whole thing. I just feel you are breaching the trust of your community when you do a sudden change and you don't communicate it properly, and and you don't communicate the intent behind it, which is very important. Right?

35:52 Yeah. I think when you're having these difficult discussions, you always have to start with why the decision has been made, and I don't think Tucker did that. Like, say, Tucker says, we will not be in business in twelve, twenty four, thirty six months because here's the costs of running Docker's hubs free tier, and it's not feasible. People will go, oh, alright. Okay. We didn't owe it. That's that much. And maybe they'll have a bit more empathy for the company, but the communication didn't come across that way. And, also, we're a very entitled bunch. Right? Developers.

36:24 Oh, you gave me this thing for free? Oh, cool. I'll I'll use it. I'll use it. I'll it. Oh, you're taking it away. I hate you so much. I mean, like and I've seen lots of people move to GHCR. Like, GitHub's container registry seems to just become in the new default because it's where all our code lives. But how long are they gonna run that for free? Exactly. Do we have a guarantee that Microsoft ten years from now, they're gonna go like, okay. GitHub is not free anymore. Right? And we're back to, you know, have a

36:50 server in your basement and surf surf all your own containers images yourself, I guess. Pretty much. Alright. Let's let's see something positive about Docker. I'm I'm just really excited by their Wasm announcements recently. I love that they're opening up hybrid architectures with WebAssembly and containers side by side. And the fact that I can do this with kind of Docker image pool and OCI artifacts, but not only that, the enhanced support for Docker Compose where I can run WebAssembly binaries as services speaking to a Postgres database running in a container and everything works together cohesively. And I think this is really exciting for

37:00 WebAssembly Adoption

37:26 WebAssembly and for Docker. So I'm curious if anyone has any any thoughts of the clip of it. Do they like it? Do they not like it? Feel free to share. So for edge use cases, I think it's I think it's quite amazing. The there's still a real problem at the true edge, and I'm talking around the industrial edge where you've got micro micro devices, which are you know, they they are they are computers. They have an arm 32 CPU. They've got a gig of RAM. And, really, your only choice today for running containers at that type of edge device is

38:07 Docker. Everyone everyone talks a great story about Kubernetes at the edge. Not at that edge. No no nowhere near. And so if your only option is to run full fat containers on that device, still, a slow ARM 32 CPU and one gig of RAM only goes so far. And so the more efficiency that you can you can get out of it, the the faster, I think, we're going to see the adoption of things like industry four because, yeah, industry invest in technology, and they they expect their technology last ten plus years. They're not gonna go refresh their entire,

38:46 you know, hardware estate overnight because this they they they wanna do containers. They wanna find a way to repurpose what's there, and it's old and slow. So I think WASM at that edge is gonna be amazing because it get it actually gets so much more efficiency or it's so much more efficient way of delivering applications at that edge. In the in the data center, I'm I'm not convinced. But That's that's an interesting take. I think I I see I see, actually, there is quite a lot of movements across the major cloud providers to start supporting

39:16 Wasm for serverless stuff. So Wasm mass functions or Wasm as, you know, for container service type platforms, which you can do today, I guess, like, with the with the Docker support for Wasm. You could basically write a Wasm module and run it inside Docker container. And if your cloud provider allows you to run Docker containers on top of the platform, it would just work. Right? I think for functions, I see a value, especially that basically WASM programs or libraries are very fast to start. And for specific specifically functions of service, that have always been a problem, the startup

39:50 time or the cold start time or whatever you want to call it. Right? So if wasn't can solve that by giving you a fast to start binary, that's probably a good place for it to be. Right? But but, yes, I I think I have to agree with you in the sense that I don't see it as something where five five years from now, everybody will say, okay. Right. We're just gonna rewrite all our. That's probably not gonna happen. So the other one probably is this. The sandboxing aspect of Wasm is a very competing feature to a lot of companies that allow

40:23 you to run third party code on their platform, like cloud providers with functions and stuff like that. Right? Like, as a cloud provider, you should inherently not trust people shipping code and running it on top of your infrastructure. And the more you can sandbox that code away from other customers running on the same host, the better it is. And Wasm has a very good sandboxing environment. Right? So, yeah, that's that's my take on it. I I think it's gonna we'll have to wait. This this is gonna be this is gonna unfold over the next five

40:51 to ten years. Yeah. I I wanna quantify some of the things you mentioned there. So, Cloudflare announced, their workers' project, which is their, functions as a serverless serverless kind of workload thing, supports WebAssembly workloads. They announced that, like, two weeks ago, which is cool. And they have 256 Edge locations. I know Neil's gonna be like, that's not edge, but, you know, they they have these locations out there for people to run these workloads. And as far as quantifying it, WebAssembly run times binaries could be started in microseconds compared to, like, a container, which we're talking,

41:24 what, five hundred milliseconds probably to start a container set the sandbox up, especially in a Kubernetes environment. It's a little bit slower. And I think that's quite interesting. But but there's also the data aspect. It's like with container runtimes, have to pull the image, and container images are not small. Right? Yes. We have container images on the megs, but we've all seen them tens of megs, hundreds of megs, and even gigs of container images. So even the pool time on that can be slow. Whereas WebAssembly binaries, again, are very, very small. Typically, four to eight meg depending on the complexity of

41:57 the application. Of course, your mileage may vary. I'm not saying all of them are. But I think at those numbers, that is a really compelling story for developers who wanna do things that way. And then the sandbox is just the kind of cherry on the top. Right? It's like you get all of those benefits without having to deal with Linux c groups, without having to worry about the architecture of the application. Yeah. Go cross compiles, but am I cross compiling for arms? Am I cross compiling for x a h six? Like, these are all concerns that I hope we don't need to

42:26 worry about. And I I do actually agree. I don't think we're all gonna be writing WebAssembly in five years, but I do think we'll be writing a mix of WebAssembly complemented with container based workloads. Another another space where I am seeing WebAssembly also growing very fast is I don't know how much you are familiar with Invoice, the the proxy. Mhmm. Invoice supports WebAssembly as as a filter. So you can Invoice is a very extendable proxy, and you can, like, add filters to it, and those filters can be written in Lua or in WebAssembly. Right? And with that, you can actually implement

42:59 a lot of complex logic at the filter level so you can do authentication, authorization, and batch of extra things, rate limiting, for example, stuff like that. And Envoy, although it started with Lyft, it is being adopted. We are changing all our load balances to be Envoy based at Google. I I know at least one company, Spotify. I know that they they are running Envoy as their edge proxy. They're not doing with assembly today, but they're thinking about it. So and I think probably Neil would now agree with with this. I I still think this is an edge case where you're running a

43:31 proxy at the edge of your network and pushing a bunch of that logic that you're doing in the back end to the proxy layer. You know, SSL termination is the something we've been doing for a while, but, like, authentication authorization, blah blah, closer to the user is is is of value. Right? And I think the reason why Envoy and WASM are becoming interesting is because all the other proxies that exist for a while have not been innovating that much. Like, NGINX did not change much in the last ten, fifteen years. HAProxy did not change much in the last ten,

44:02 fifteen years. Right? They probably support HTTP too, but, like, who cares? Right? So so so, yeah, I I I I see that as another kind of area that would be interesting to look at or to keep an eye on. Yeah. I think WebAssembly is an extensibility model. It's definitely on the rise too. Like you said, with with Envoy, Red Panda has, like, a built in WebAssembly runtime for the stream transformations on events going to the system. And there's a really cool framework that I don't remember the name, and I'm not gonna remember it. But it's like a desktop extension

44:33 model. So if you write desktop apps, you can now write WebAssembly modules that get hooked into it, and it makes it run on Mac, Linux, and Windows without having to worry about anything else. I think that's quite a interesting use case too. Alright. We are very quickly running out of time. So unless everybody would like to share more on WebAssembly, I'm gonna suggest that we move on to our final topic for today, which is Kubernetes. Any last words on WebAssembly? Kubernetes, it is. Kubernetes. Awesome. Well, I'll throw this one over to you, Adele, since you've already mentioned it. But very recently,

45:08 register change as part of Kubernetes is a a big deal. People need to know well, I guess it's not as important anymore because of the automatic redirect, but there's still things people need to know. So maybe you could share a little bit on what's happening and what's important for people to take away. Yeah. So a little bit of context. Kubernetes was up to 1.26, which was released December. They were pushing all their images to k8s.gcr.io. So GCR stands for Google Container Container Registry, which is a Google managed version of Docker Hub, essentially, or Docker registry.

45:42 And k8s.gcr.i0 was a subdomain that Google created specifically for Kubernetes. Right? It was kind of like its own instance. It did not share infrastructure with the rest of the container registry space. So the creates, like, a subdomain, and they just give it to the community. And, you know, and then, you know, I talked about this earlier. They realized they were running out of money because, you know, spending way too much money on egress. So they decided to do the migration. And the migration today or the migration essentially was we're gonna move from kxs.register.i0 to word registry dot Kubernetes or dot dot

46:16 k x s dot I o, which is essentially if you care care about the implementation detail, it's a proxy running on top of Cloud Run, which does the redirection. It's a very low low low weight Go binary. The code is open source. You can read it. And, essentially, what it does, it receives the request. It looks at the IP address, and it uses a Go package, which which which can give you the list of known IP addresses that AWS use. So if your traffic is coming if request is coming from AWS, it forwards toward the container registry of AWS.

46:48 Right? So the images are still technically mirrored across both Google Cloud and AWS, and maybe they are planning to add more cloud cloud providers later. What's happened basically is AWS came to the Kubernetes community, and they said, we're gonna give you money to be able to host stuff on AWS. Right? And maybe later, more more cloud providers will chime in, and they would be able to do the same. But, essentially, now when when Kubernetes is built, they will push all the artifacts toward old cloud providers. They're mirroring them on Google Cloud and AWS and maybe Azure in the future. And then

47:20 this proxy, this little Go binary, just forwards depends on which the IP where the IP is coming from. Right? And that's gonna help a lot with pull time. So if you're on AWS and you're pulling for the new Container Registry, you will see a significant improvement in terms of the pull pull speed. Right? And, yeah, you just reduce the cost on a single cloud provider and kinda distribute across multiple cloud providers. So that's the context. Do you know what that is? That's multi cloud. That's multi cloud. Yeah. That's actually very helpful. It is. I was waiting

47:50 was waiting for somebody to make a comment. And and, actually, you know what it is, to be honest with you? It's basically what we used to do with Linux packages. It's mirrored Linux packages across multiple servers. Right? It used to be a long time. I I when I was living in Morocco, I did establish a mirror for APT. In Morocco, I bought a bunch of servers, and we established we we we we we up set up a mirror for APT packages. That's essentially what this is, except it's done by the community, not by a bunch of

48:18 volunteers. Right? How is it not just a CDN? It is a CDN. A problem. Is a CDN. I'm I'm surprised that that Google just just didn't host all images on a CDN. Well, I think that the reason is because containers require a very specific implementation of the registry itself. Right? The registry API is very specific. It's not like it's not like an a picture or like a CSS file that you can just put on an HTTP server. That's due to the client implementation of the kubelet on Kubernetes because the kubelet is not just doing an HTTP pull request. It's using

48:49 the the Docker registry client's library to pull the image. Right? But I would agree with you. If if it was if images could be pulled through a standard HTTP request, then what's what why are we bothering with this? We could just implement it on top of legacy. Yeah. Right? Alright. Well, thank you for sharing all that. So we got a couple of questions left. So, Adrian, everyone else in the chat, if you wanna ask any questions before we finish, drop them into the comments, and we'll move on to developer portals. So a trend that I've seen through reviewing

49:00 Internal Developer Portals (IDPs)?

49:17 CFPs for a couple of conferences and, again, speaking to people at conferences is that Backstage has really kicked off this new I I don't wanna use the word movement, but, you know, this initiative where teams feel like they need to have developer portals that get them a gateway into their infrastructure, their services, their projects. Like and I'm curious. Is this something that you guys are seeing again with your own customers and partners? Do you use yourself? And what is the value proposition that people are looking for here? Is it purely developer experience, or is it something else that I'm missing?

49:47 Well, I'm still unclear on the difference between a developer portal and, like, a console view for your what you're doing with your cloud provider. Is this for the things that you've got on on-site or what sort of agnostic to where the things are actually running? Or Yeah. So the back speech, just an example, is kind of used to give you, like, a service catalog. So as a developer team deploying microservices, it'll actually show you the all the services, how they interact with each other, the message formats, and visual overviews. And we're seeing extensions now where it can

50:18 hook into Kubernetes and give you some limited capabilities there. It can hook in to your cloud provider and show you I'm models and our back rails and a whole bunch of other stuff. So Gotcha. We're we're seeing this kind of DX based portal experience trying to make developer teams more efficient. I'm not sure if that's the value add, but, yeah, Neil, you were gonna say something there too. Yeah. I was gonna say, as as Kubernetes or containers get more and more and more mainstream, you're getting into the realm of web devs, front end developers. In all honesty, those people who who know

50:50 how to use those developers who know how to how to deploy and manage applications on Kubernetes are probably full stack, and now now they're extra full stack in order to be able to do that as well. But there is a huge proportion of the developer community who probably see themselves as front end devs or web devs, and they they may or may not have the skills to be able to do infrastructure tasks. And so there is definitely a need for an abstraction where you say, I just wanna take my code, and and I wanna deploy my code. Off you

51:18 go. And that's where I think these kind of dev platforms will shine is is being able to engage with the dev in a language the dev speaks, which is which is code, and being able to to to get them and and or have a bunch of free can templates available. They can just click to deploy, and they get their server up. They can then go and and push their code into. I I think it's critical as as we get more mainstream, we get into more emerging markets where where the level where where there might be more more of

51:47 a skills delta. I think it's I think I think it's just a just a a sign of maturity of Kubernetes. Alright. Can we try an experiment? Are you happy to try an experiment? You smile. I'll take it. Alright. There was a report recently that said GitHub just crossed a hundred million monthly active users. So let's pretend that this is science and there's a hundred million developers out there in the world. So in the chat box and don't hit return until I ask ourselves to hit return. Type how many developers you think have used a kube control command. And the audience could feel

52:20 free to do this too. But don't hit return. Checkbox. Yeah. But just type a number, and I'm I'm gonna count to three. And on three, we can all hit return. I can't even find the checkbox. At the bottom, there should be, like, there's a timer saying we've been live for fifty six fifty eight minutes, and then there's a little square for chat. So in the chat box, type a number of how many developers that is a hundred million have used a cube control command, and we'll all hit return at the same time. Does everyone got a number?

52:50 I do. Alright. So on three. One, two, three. Wow. We all went very different. So I said 500,000, Eli's five thousand, Neil went in at 10 mil, and Abdul said every developer has run a cube control command. Yeah. I mean, we'll never know, but I I just you know, KubeCon has tens as a sellout event at 10,000 people going to to Amsterdam. As a general rule of thumb, when I talk about developers that go to user groups and conferences, I always say if you go to these things, then you're in, like, a five to 10% bubble of people that really

53:33 wanna push their knowledge and their careers and other things. But I've always made that number up, so I I really have no idea. Right. So I mean, it's extremely low because I think, well, I don't know. We're practitioners of a thing always think the thing is more widespread than it actually is. How many people on GitHub are just using it to host, like, little personal website that they've made or But now you're claiming 50% of the attendees at KubeCon have never run a Kube control command. Yeah. But if we're doing prices right, we'll know. Might be right. Yeah. I I mean, probably.

54:04 I don't know. We could sample it when I'm there. Maybe I'll just run around with a pen and paper and say, have you run a Kube control command and see how many people I can get to answer it? I guess I guess that's probably you are right in the sense that there are people there are probably people using Kubernetes without running the kubectl command. Right? Select the dashboard. If you deploy the Kubernetes dashboard, you don't need the command line. Can just deploy things from the dashboard. So there are probably people who are using Kubernetes, but they're not using the

54:26 command line. They're just using some sort of high level of structure there. But is Kubernetes still not I mean, it's growing. We can see that through the CNCF. Can see that sort of projects, open source contributions, so people at conferences. But it's it's still niche. Right? Like, arbitrary number. 90% of people that write code are probably deploying to something that's not Kubernetes still. Is that right? Is that wrong? Kubernetes, maybe. Containers, I would say, is more mainstream. I just just this last weekend, I was walking down a random street in Berlin, and there was, you know, those big digital

54:58 signboards. And so we're walking down the street, and there's there's 20 digital sign boards in a row each selling. And as I walk past one, I'm like, oh my god. That's that's Docker desktop. And and, basically, the the sign board had crashed, and in the screen was Docker desktop. And with with the the the stack for the software, it's crashed. And so and I'm like, okay. Cool. You just never know where where you're gonna where you're actually gonna gonna come across containers. So I think I think I think containers have far more adoption than Kubernetes,

55:30 but Kubernetes adoption is definitely is definitely rising. Yeah. I'd be curious to pay attention, you know, GitHub and Stack Overflow do annual reports where they ask a whole bunch of questions. And I haven't checked, so I don't know. But maybe containers and Kubernetes adoption is in there, and there are numbers. So I'll see if I can find them and and share them with people afterwards. But fun experiment to see. This is complete for wildly different guesses, but I'll take I'd be very interested to see the results if you do a straw poll of running around

55:57 KubeCon and asking people, have you actually run KubeControl? I mean, of course, I'm gonna do it. I'm gonna film it at the same time. So, yeah, it's gonna happen. To to be honest, asking it in an echo chamber, you you you might get a very different result than across some more broad spectrum sample. So if you go go to an actual dev forum. Yeah. I was about to say that I think it would be more more interesting to to go to a conference which is not cloud native focused. So one of the conferences I want to go to next

56:23 year is called Scale, South California Linux Expo. It takes place in Sacramento, I think, every year. And my cohost was there and interviewed some people for the podcast. And the from the interviews I listened to, there was, like, a widely different spectrum of what people are interested in. Right? It's like a totally different population. They are not in containers. Like, some of them, say, oh, we heard about this. They called Kubernetes, and we're thinking about looking at it, but they're still in, you know, hardcore Linux, low level kernel stuff. So and there is there is actually an

56:53 article that I've read a long time ago. I have to find dig it up and find it. I think it was called Shadow Developers, and it I don't even remember who wrote it. It was this this person who said, like like, we people who are on the Internet, we're only exposed to a certain population of developers. There are a bunch of like, your developer sitting in a, like, a, like, a town hall of a city of, I don't know, country in The UK writing visual basic code on top of, you know, of, like, the Microsoft Access databases. That's a

57:24 developer as well. Right? Are they gonna fill in a Stack Overflow survey? Probably not. Are they gonna go a conference? Probably not. It's Yeah. I mean, it's it's something we come up against in DevRel. Right? It's like you can only reach the people who are who you know about, and then there's all this sort of yeah. Like like, dark matter. Where are all the people who aren't on the Internet, who aren't terminally online? Alright. We have gone a little bit over that. I just wanna check. Does anyone have a hard stop, or are you okay to

57:48 answer one more question? And we'll finish up for today. I'm I'm good to go. I'm good to keep going, brother. Well, I think keep going as well. Yeah. I mean, I think, given everything that we see on Twitter these days with open API, open AI, and chat GPT that we should talk about the way that that may influence our little niche Kubernetes and Docker and container bubble and the way that we work. And specifically calling out Alex Jones from Canonical's project of KHJPT, which hooks into, like, the Kubernetes event log and auto log to tell you when things

58:23 are wrong with your cluster and tell you how to remediate them. Is this something we're gonna see more of, and is it gonna take all our jobs? Which is hyperbole. Awesome. Gonna ask awesome. It's it's it's very, very cool. Yeah. Triaging is one of the one of the things that and and let's be honest, 90% of us see an error, copy paste it into Stack Overflow, and and look for the answer anyway. So that that this is just just automating that loop. And I like that we've now got to the stage where we've written software so

58:51 complex that we need to write more software to understand it. You know? We we we are you know, I don't think that our jobs are at risk because you ask chat GPT anything about, you know, real facts and it spews you complete nonsense. But it's certain tasks like that, I think it could be really, really useful. So I also say that whilst it still has quite a lag in its training, if you ask it questions about Kubernetes one dot 26, do you think it's gonna better help you? It maybe not. Yeah. Because it's it's it's got that cut off. So it

59:24 it as long as it's a bug that existed back when it was trained, it'll help you. I'm just I'm not I'm not sure. I mean, I'm I'm not that smart enough to understand, you know, if if it can determine things that that that's newer, bugs that are newer than what it was trained on. I don't know how it handles. So you you have to give it all the context. That's what I I've seen is that people are are feeding the one twenty six manifest as examples and then asking the questions based on it. And it

59:49 does seem to have a weird ability to understand what's happening and answer questions. Like, it it could be retrained with modern data if you're willing to give it the modern data. Do you think this is something you would hook into retainer? I am all about making things easier for our users. So if this was a way to make things easier, we actually had a discussion. We we used to include Compose with a k in Protainer. So you you could take a Docker Compose file, and we we would auto deploy that against Kubernetes. We we had to pull it out because

1:00:19 it was full of CVEs. If this was a way to bring back that type of capability where someone could could put a put you know, give container Compose file, we we asked chat GBT to convert it, and then they gave us back a manifest so we could then go direct deploy. I'll take that. Cool. Yeah. That'll be cool. So so, absolutely, if if if we can do if we we can use it to improve the service that we deliver to our users, I'm all over it. Yeah. I also wonder, like, from the Google and Skillway point

1:00:45 of view, can we, like, have it have an understanding of all the resources we have and then give us cost optimizations or even just remind you that you've had an instance running that you're not using anymore? Like, hey. The traffic on this virtual machine hasn't really moved a needle in ten days. Do you wanna shut it down? Like, are we gonna see more stuff like this? Our our most of our recommendation tips that we have in the Google Cloud Console are actually ML based. They're not bot based. They're not based on. Like, right now, if you log in to the Google Cloud

1:01:13 Console, there will be some recommendation in your virtual machine interface to say, hey. You haven't used this virtual machine for a while. Oh, wow. So that's actually ML based. And another way another place where we are using a machine learning based model is we have a software called Cloud Armor, which is like a web application firewall. It's a WAF. You can attach it to a cloud load balancer, and then it can do a bunch of things like XSS, like a cross site scripting detection, SQL injection, etcetera. But one of the things it does is something called predictive traffic analysis.

1:01:43 So, basically, when you turn that on, it creates a base model for your traffic on your load balancer. And if it sees something weird, it's gonna actually notify you and say, hey. We're seeing these IP addresses are potentially gonna be harmful. And you can turn it on to either create a firewall rule to block those IP addresses automatically or just give you a recommendation and tell you, please apply this policy to block those IP addresses. And last year, that feature was used by one of the customers, and it helped them block the largest DDoS attack known to human in history. It was,

1:02:14 like, 70 47,000,000 requests or something like that. So they didn't they didn't have the automatic feature. They just had the recommendation feature, but somebody caught that early and managed to apply the policy to, you know, to prevent issues. So in these kind of cases, I do see value for using machine learning based, whether they are or BART or something else, but something that could that is clever enough to see what your base usage is and can detect deviations. Right? But this is not new to ChargeGPT. This is like deviation detection machine learning is a very old problem. Right? So ChargeGPT

1:02:50 as is today being useful for for us with what we do today, I'm very skeptical, but that's just me. Yeah. I sort of think of chat with GPT as it's almost like a toy or like a demo of, like, the under the technology, the ML technology in targeted ways. Like, exactly as you say, like, very specific targeted ways that's trained for this one purpose, I think have an enormous potential. I'm thinking about, like, observability, like, price prediction, all of these things, but it may not be, like, a chat GPT in particular. It may be something more specifically

1:03:20 Shameless Plugs

1:03:22 purpose built. Alright. Awesome. Well, thank you all for joining. It's been a pleasure talking with you all. I'll give you all a a minute. If you just wanna plug anything, share anything anything that that may be, feel free to do that now. And then we'll wrap this up. I don't wanna plug Fortainer at all. That just seem that that that that seems seems shallow. But so no. I I I think I think the more people that get exposed to to containers and Kubernetes and realize that not everyone needs to be an expert, the better. I think I think there

1:03:58 are varying levels of skill required, and I think we need to be completely comfortable that those in out there in the ecosystem have varying level varying levels of skill and embrace them and and have tools and capabilities that that help check know, the the the the chatty bitty conversation just seen as a prime example. It it helps those, you know, underskilled to do things that their more skilled peers might be able to do. So in anything anything that helps people, you know, increase the skill level is is a good thing. I'm gonna do a shameless plug

1:04:29 and ask people to go listen to the Kubernetes podcast. We have David on it. That was the last episode. It was pretty cool to talk to to to to you. But we have a bunch of stuff coming up, very interesting things. We're gonna be covering platform engineering and Backstage. It's one of the topics. We're gonna be covering Docker and Wasm as well, and we're gonna be also covering KubeCon Amsterdam. So we're gonna be asking people just harassing people in the show, asking them, like, how it's going on there. So it's Kubernetespodcast.com or you can follow us on Twitter. And,

1:04:57 yeah, thank you for having me. Was fantastic. Suppose my plug would be my my team at Scaleway, the developer relations team, we do we do some livestreams almost every Friday. Not this not this one coming, but almost every Friday at 3PM UK time. We go live and chitchat about the cloud. Sometimes we have guests. We just did a bunch of Kubernetes focus streams during March. So, you know, if this if that's your jam, come and come and hang out with us. You can find us on Twitter at scale way underscore devs, and we always tweet

1:05:25 when we're going live. It'll be really good to have folks come and harass us on stream. Cool. Alright. Awesome.