2:40 Introduction

2:40 Hello, and welcome back to the Rawkode Academy. My name is David Flanagan, I am your host for today on this episode of Rawkode Live. Rawkode Live is a show where we take a look at all the best open source tools and technologies solving real world problems in the cloud native and Kubernetes ecosystem. And today, we are taking a look at something I'm very excited to play with, Namespace. Joining us today to introduce us to Namespace is Hugo. Hey, Hugo. How's it going? Hey, David. It's going great. Apart from the fact that they've been a little bit under the weather,

3:15 we had a you know, kids were sick, then I'm sick. It just goes around. But it's actually a great time right now and just getting into past fall and into winter. And every day is actually super exciting about seeing started to talk a little bit about what we're doing and and getting people engaging with the product. So super excited about that. Awesome. Well, I'm very excited to sit down and spend some time with you today, learning more, kicking tires and having some fun. And I will also just segue and confirm what you said. Yeah. When

3:50 my kids get sick, I am sick the following two days afterwards. And then it seems to be on this wild loop. Someone in their school gets sick. They get sick. I get sick. My wife gets sick and I'm look, look, look. I feel that pain. Alright, awesome. So before we talk about namespace, do you want to share a little bit more information about you who you are, what you're up to? Just give us the who who is Hugo. Yeah. You know, I and it's actually thank you for the opportunity. It's really exciting to to spend some time with you and and your

4:05 Guest Background & Journey

4:24 audience. I've been living in a cave for many years and and I'm living that cave now and it's actually very exciting. But but the story starts early. I I'm a software engineer at heart and I've been I've been thinking about software and and working on software for as long as I remember. When I was 15, I sold my first piece of software that I wrote for BOS. And Wow. Trying really hard to make that my main operating system after a stint. Like, my first system was Windows and spent some time on Linux. Red Hat 5.2 was my first distribution.

5:00 Then then tried to do kinda run macOS eight fully emulated. It's it's because I really wanted to run a Mac and then, moved over to POS, which was the kind of closest thing. And I that was kind of part of my journey just exploring, figuring out things by by myself. Then later on college, spent a lot of time on open source, very focused on networking and networking infrastructure and authentication and these types of challenges, which I still have a a passion for, although I don't spend as much time there nowadays. And college was was done then kind of did

5:42 a regular job as as many of us do and but then the I started getting a hitch of of of wanting to start something by myself and and move over to Finland. I'm originally from Portugal. I had moved to Germany to to move to The to kind of for my first job, and then I moved to Finland to start the company. This was 02/2010 doing, well, kind of edge computing. Before edge computing was really a thing, we were streaming smartphone like experiences to feature phones. And it was a great time, not not because of the point in time.

6:19 02/2010 was really after a recession, so probably one of the worst times to start the company, but, like, putting together a team and and just working on the technology was was really exciting. But we couldn't make the business work. So a couple years later, we kind of the company got transformed, and it was bought by Facebook. But just before that, I Google had been asking me, hey. Do you want a guy do you wanna come and and spend some time with us? I I had a couple friends there, and I said, why not? So I I

6:51 moved to Switzerland where I've been since now it's been almost ten years. And at Google, I I worked on I I've always worked on software infrastructure. Actually, I tried to stint at consumer, but almost next day, I was back at, hey, guys. Have you seen this infrastructure problem that we have? And and, yeah, at Google, I had kind of two major journeys. One was building or helping to build Google's internal development platform or application platform called Bock, which, unfortunately, we never really spent a lot of time talking about it externally. We we'll we'll get back with, I guess,

7:28 a little bit later. And, and then as part of kind of helping the company move over to Bock, I I got to work with search, and I got really passionate about search. So I joined the the search team and, in particular, the assistant team that works on all the kinda smart devices, like speakers and displays and other things. So I helped build kind of the infrastructure and the platform for for that organization as well. And after those two, I I really was craving kind of going back to to my kind of startup roots, if you

7:58 can call them that. And and I I I left the company last year, and I it's was a tremendous experience. But I I wanted to start something from scratch, and I was going back to actually some of those kinda early infrastructure ideas, and I was playing around with some soft software defined storage. That's another passion of mine. And as I was building that, I I got to experience a lot of the challenges that then I got to learn that a lot of folks, do do experience as well. Like, okay. I need I have the GoControl

8:33 plane. What's what's the best way for me to do kind of incremental builds? Or I have multiple services, and they need to authenticate, with each other what's kind of the right way to do that, and what's kind of the right deployment strategy. So this whole life cycle of going from code into having something meaningful running in production, I was finding myself answering those questions like, kinda on a on a case by case basis. And I thought there must be a better way. And and, actually, I got to learn about a lot of good technology that is

9:07 out there, but we we feel like that there's more that we can do to really help, ourselves, actually, first and foremost, but then a broader set of of developer teams, to to just go well, go faster, and and have the confidence that you're building kind of on a world class platform or infrastructure, and you can focus on your application. So that gets us to Namespace we're at today. Awesome. Thank you for sharing. I'll go back to the first thing you said, which is you're a software developer at heart. That's why I describe myself these days because I never

9:33 The Problem Namespace Solves

9:41 get to actually write any software anymore. So it says, you know, you've got a lot of experience having worked on search, the assistant and for such a problems at Google, signing a company like that is a lot of things over many years. And I'm hoping that what we're gonna look at today with Namespace as you condensing these many years of pains, failures, experience and challenging things into something that's really gonna resonate with our audience today. So when real world problems that a lot of people face during this, I don't know, epidemic of Kubernetes adoption. I

10:20 don't know. Like, is it a pandemic? Like we see a lot of traction towards Kubernetes. This right. It's now almost the ubiquitous way to build and run applications at scale. Even for people that aren't at scale, which I see even more of now than people at proper scale. However, Kubernetes as that API that keeps us cloud agnostic to a certain degree, but it comes with many, many, many challenges. And one of them, it's what we're gonna talk about today. It's like, how do we get that joyful developer experience and deployment pipeline back. So I wanna I wanna learn. I wanna

10:53 take all this experience out of your head and I wanna see a really cool product today. That that's our mission. Sounds great. And and for it's worth, we we we're actually very inspired by by not just Kubernetes, the technology, but by the the kind of the ecosystem around it. A tremendous amount of respect for for what, like, so many people have worked and and put together. But, yeah, like, approaching it from from I just want to get my application out there, and I want to use this infrastructure for my benefit. Benefit. Like, I want to benefit from this ecosystem,

11:29 from this Kubernetes ecosystem, but not having to make that my full time job of, actually, how do I make this work for me? That's that's really part of what we're what we're about. Like and and we'll learn that, obviously, you need to make certain trade offs on the way, and we'll talk a little bit, hopefully about those. But, but we care deeply about this application centric, philosophy that builds on great infrastructure. Awesome. It looks like we have one of your colleagues in the chat. Happy to help with questions as well. Carol. Hello, Carol. Nice to meet you.

12:02 Overview of Namespace

12:02 Alright. Let's give everybody a little bit of an overview on our namespace today. What are we going to be taking a look at? What problems does it wanna solve? And how is it different from other tools in the space? I'll let you answer them in whatever order you want. Go nuts. Yeah. Let me preface by we're we're early. Like, we we know we're we're at the beginning of a journey. Like, we have we have an idea of where we're going based on both our our experience as software engineers ourselves and and kind of building applications,

12:03 Namespace Overview & Philosophy

12:33 but but also what we've been learning from from folks in different companies. But you you'll hear a snapshot that makes sense for us today, but I it is likely that as we go over time, it that that story will also shift. But where we are today is and and what folks can already try, is that we have a alpha version of an application platform. We call it an end to end application platform that builds on Kubernetes. It's end to end because it focuses on your whole development experience, not just kind of building, not just testing, not just

13:09 deployment, but but how do these three things kinda work together so that you can just write code and kinda compose infrastructure and ship it to production without having to kind of on the way go and figure out, kind of individual parts of the story. So Namespace is it kind of facilitates you. It kinda helps you build your application. It helps you build an application that is composed of multiple services, but it's really up to you of how many services those those are. It helps you whether you're in a single repository or multiple repositories. So it takes

13:43 all of these kind of real world problems that folks are are facing and and tries to package it in a developer experience centric platform that brings you build. So you can you you can you work with code, and we build images for you. It brings you testing so you can both do you know, more constrained testing, but it also allows you to do, like, whole what we call old system testing. So actually replicating production like stack for for testing purposes. And then you can also ship that code that you've been writing and that we built

14:19 to production. And it does that exclusively over Kubernetes today. And the experience we're aiming for we're aiming for an experience that is much more like Docker Compose, which like, tremendous amount of respect for Docker has been kinda instrumental in the industry, but but it's not confined to development. It actually goes all the way to fairly complex scenarios as well. And, yeah, that's that's a little bit of of of what we're about. Awesome. Alright. Naveen asking in the comments. Are we going to just talk? Are we going to show a demo? Of course, we're gonna do

14:56 a demo, Naveen. Come on. This is the hands on show. We don't we don't just talk. So let's just cover a couple more questions and then we will jump over to the demo and we'll take a look at it in action. Why why do we need a new tool for this? Like, I don't that's not a disparaging comment. It's kind of a passing question. I remember speaking to Brian Grant at Google, about the saturation of this marketplace, And not that saturated with I think he said there were over 400 or 500 tools in the spreadsheet that he's

15:38 got for different tools all deployed to Kubernetes. And I think that is kind of an affirming position because clearly deploying to Kubernetes is a bit of a challenge. We don't have 500 different tools for the sake of having 500 different tools. I think we have 500 different tools because there's different opinions on what that pipeline, what that process should look like. Yeah. I'm curious what Why Namespace? Why now? And why is it different? Yeah. That's a great question. And I I won't pretend to to to to say I won't I won't say that there's like an exact answer to

16:16 it. I think part of the answer is, we do have first of all, well, we look at the the the technology of these already out there. It's great. And and I think folks have been doing a great job at delivering individual tools for individual problems that really nail those problems really well. And that's great. Like, if you want to do authentication, you have a few different options out there. They're really solid, and that's great. But what what do we see kind of over and over is that many teams, and and if you're an engineer in in

16:51 a in whether it's a startup or midsize company or or whatever size company, you're still it's still up to you to kind of go and figure out how you put those things together. And there are a few approaches to it out there in the industry. We believe that something that kinda brings development closer to production is is kind of instrumental on not just from a technology perspective, but also from how we work together. Like, I think one of the interesting and and things that I'm actually most proud of from from my journey back at Google was kind of

17:27 to help kind of close a little bit the gap between the development teams and the SRE teams. Like, kind of each one lived in their own world. They have this shared kind of set of concerns, but because of the technology space was very separate, it also meant that they were often talking different languages, like human languages, like programming languages, and and just just really con being concerned about different things. And I think technology can be an enabler to kinda bring groups of people together. And, at the fur but but the most important thing is that we care about the mission

18:00 more so than the technology. Like, the technology that we're building is it we feel like it fills a gap, that even when we look at some of the other, software out there, we feel like there's a unique gap, and, hopefully, we'll we'll get to see some of that during the hands on today. But but first and foremost, we care about the mission, which is helping developer teams, helping the DevOps teams, helping SREs just kind of do better and and being able to ship applications robustly. And if on the way we figure out there's a better kind of piece of open

18:31 source that does can do a better job than what we do, then we'll just go and use that. And that's already the case today. We build on the shoulders of giants. Kubernetes is built in. BuildKit is built in. Queue is built in. Like, we we already do that, and it is very much part of of how we approach software. Awesome. Look, two of those technologies you mentioned in the last three seconds are are two of the most exciting technologies to me that we have in this space right now. I think BuildKit is completely underutilized as a tool Mhmm.

18:49 Getting Started: Installation & Workspace

18:59 Especially for CICD pipelines and as a local development tool. And q, which I think I've been a very loud advocate for for a number of years now is just a language and I use for a pretty much everything these days. In fact, I even built a database with that as the query language. So so I'm really excited to sit down and I think this would be a great time for me to share my screen. We'll take a look at the homepage and we'll get started with the hands on component. Sound good? That's good. Awesome. Alright, where's my screen share? There we go.

19:32 Okay, so here we have the Namespace website, you can find us at Namespace. So, it says is that all in one development platform unified experience from development to production. So, that's a pretty that's a pretty good teaser right off the bat like I'm excited about that. We've got a little code window there that says we can set up a cluster, develop test and deploy. So this is our full life cycle experience situation. Pretty strong opening, gotta say. Well, we you know, because I mean, we're very transparent and I hopefully, that's something that folks will kind of

20:09 learn about us. Like, part of part of the challenge that we have is also because the the industry is so focused on individual tools, so so something that does authentication really well or something that does deployment really well or something that does building really well, We we're still figuring out what's the best way to talk about how we think about the problem and and how we feel like we can help teams. And this all in one has been kinda a a monitor that we've leaned on for that, but I I think that there's still there's still more work for us to

20:44 do there to kinda better explain what we're about. Awesome. Alright. And just to confirm as always, I think we said at the start, everything is open source. Right? Everything we're looking at today is is available. It's on GitHub. People can can contribute and open issues and all of that good stuff that we expect from open source. Right? That's that's right. So it's the everything that we're looking at is Apache two licensed. We actually had a couple of contributions already that it's really kind of fills my heart that to see people engaging and, yeah, we're happy

21:17 to have people come over, test it out and and contribute as well. So that's that's definitely part of the journey. Alright. Awesome. Well, I'm gonna click on start from scratch. And I'm curious if this is gonna tell me how to nope. Doesn't tell me how to install it. So let's find the install instructions first. We'll we'll take notes as we go as well, David. There's no install instructions. The well, if you press quick start, it will give you install instructions. Alright. Let's go definitely to bash, but I I will go for it. Yeah. I mean, the docs say starting from scratch.

21:57 I mean, it's almost scratch, but as long as you've got the NS command. Right? Let's see. Yeah. And as many other tools in this space, we have our our CLI. We we're we're thinking about, like, some of those trade offs. Like, one of the things that we actually care a lot about is not having to manage dependencies. We we care deeply about knowing that we're using the right dependencies, but you just saw some of that. Like, you installed NS from Brew, but it will actually the first time that you run, we went and fetched the latest version that

22:02 Managing dependencies

22:35 we just released an hour an hour ago. I did notice. I was curious why NS dash dash help was fetching something from some sort of repository somewhere. Right. Okay. So just updating itself. Okay. Yep. Alright. So and I'll just start from scratch. We need something called a namespace workspace, which is a directory or some sort of NS workspace dot q file. So I'm just gonna initialize this. When we talk about modules, should I think of them in the same light as I would with with Go modules? So typically, github.com slash some sort of repository. Yeah. That's that's right. And we

23:19 we you'll see some of some inspiration that we've had comes comes from Go. That that's perhaps, like it's it's it's probably worth kinda spending just a minute on this. Like, Namespace includes a module system very much like Go. And and the reason we do that is we're kind of tired of having to manage dependencies manually. And, and you'll get you we'll see that a little bit more during the during the hands on, but you'll you with Namespace, you can refer to servers or components, we call them resources, in any repository in the same way that you do

24:00 with Go. Like, you just import the the that repository, and we'll fetch it as as needed. And we'll always do that in a way that doesn't really interfere with anything else in your machine. We're we also like it to be very tidy and and not kinda random software, kinda doing random things. But we we try to tackle as much of this kind of figuring out what are the dependencies, unloading those dependencies, making sure that you're up to date for you. Okay. I don't wanna jump the gun on it too early because it sounds like a conversation

24:32 we may have further on into the session. But when you talk about having a referencing another repository and other packages, is this, like, platform teams that provide building blocks for people to deploy MongoDB or Postgres or Redis and all these other infrastructure components? Yeah. Like, we think of it as a building block, and it can be applied in multiple ways. Like, one way is what you just described. You're you're inside of an organization, and you have a team that is focusing more on infrastructure, and they put together a series of resources and resource providers as kind

25:04 of Namespace calls them. And you can import them, from if they're if they live in different repository. They can also live in the same repository. Like, we support kind of arbitrarily sized mono repos. But I think from a open source community and ecosystem, what is really interesting is we won't have the chance of kind of going through that today. But imagine that you have an application that is, well, what's a monitoring stack. And what do you do today? You go to, probably the Prometheus, community managed Helm page, and then it tells you, hey. Run these commands, and now you have Prometheus.

25:43 And we we turn that into a dependency set. You go to your application, and you say, hey. I depend on Prometheus, and this is kind of a one line, import. And then when you deploy your application, it will also automatically kind of deploy the right Prometheus to go, with it. We'll see a little bit of that later on, in the hands on, or a part of the getting started, But the that's how we think about, like, kinda package management. It's it's both something that could you can use, like, build your own, but also other people in the in the ecosystem can

26:18 and the community can build their own, and then you it's very easy for you to use them. Like, you can just make depend on them and and you're ready to go. Alright. Awesome. Well, running our edit command, got us a namespace workspace dot q, which defines our module, some environment, and then dependency. I'm assuming if I were to curl this, it would probably still too take a hub repository somewhere, and this is likely a GET share. I mean, I mean Yeah. That's right. Okay. Cool. And foundation is just it it was for a long time, like,

26:53 maybe maybe a little bit of insider baseball as as our our North American friends would would say. It was for a long time our our internal code name, and I think we have we have a soft spot for for that name. So even though the product like, we you'll often, hear us say, namespace, and that's kind of the platform, a lot of the dependencies come from foundation because that's that's where where the core orchestration is, where the core platform and a lot of the dependencies live in. Okay. Is that after the book, or have I missed something there?

27:02 Namespace

27:28 You know, you can I'll I'll I'll I won't try to rewrite story rewrite history. But I I I think it's a name that there's a history to it also for for us. The the team and that kind of the Namespace Labs team is we've been working together for a while, and and Namespace has sorry. Foundation has been a name that showed up in our past as well, so it's so there's a kind of special meaning to it. Right. Okay. No worries. Got it. Yeah. So this takes us to some repository. I'm sure we'll come back to this at

28:05 some point later. Let's jump back to the kinesarctic gate. Oh, no. No. There was a link in the docs. I'm sorry. See it? It's there. Would have taken you back, so that's Alright. So now we need to do some sort of prepare step. So we run this, Namespace is going to do it's downloading k three s. You wanna fill us in on what's happening to site. Yeah. So and and this is one of those things that we would love to get feedback on, also the the names of some of these commands. We've actually gone back and forth on

28:08 Preparing the Local Environment (K3s, Registry)

28:39 prepare. So what prepare is doing is it's setting up the an environment. And often when you don't see an environment reference with with Namespace, the default is dev. But even though there's a list of, like, dev prod and and staging in the in the file above, actually, Namespace doesn't care. They're apart from dev being the default, there's there's no other special meaning to them. You can have as many different environments as you want. You can have, like, 10 different development environments, cloud versions, local versions, etcetera. Doesn't really matter. And what prepare then does, it makes sure

29:16 that you have a Kubernetes cluster set up to be usable by namespace in that environment as well as long with other dependencies that we that we use. So for example, we also set up an image registry so that you don't have to go and figure out how how what which registry should I be using. So when you say prepare local, we do run k three s inside of Docker. We use k three d for that purpose, which has been fantastic to actually have that and be able to use it. We're also big friends of k three s. I don't want to kind

29:47 of stir any any any any and I I know that not everyone like, I think I know that a lot of folks like KIND, and we also really like KIND, but we started early with K3S, and and and it's been we really appreciate how flexible it's been for us through kind of development and production. And and it does a couple other things. It also makes sure that you have an ingress controller inside of that cluster, which K3s by default does, obviously, with traffic, but we we install NGINX. And and and then we also we have a

30:20 component that helps orchestrate some of the things that Namespace does that gets installed into the into the cluster. And that's another of one another one of those pieces that we've kind of gone a little bit back and forth on the design. Like, previously, we didn't. Now we do. And and we may still kind of change that. But there's our orchestrator also gets deployed into into that cluster. And after it's done, we update a local file dev host, we call it, which kind of says, well, for this environment, use this cluster that we've just set up.

30:54 Alright. Thank you. We have a question in the chat from Diego, who's asking which image registry is used, and are emojis encrypted and private? Are they readable by Namespace Labs? Yeah. It's a great question. We're so when we when you do prepare local, we set up an image registry inside Docker running on your machine. So we have no access to it. We it's something that we we care a lot about so that's something as well that we're kind of been working on and we keep working on. So we really wanted it to be as local as possible.

31:28 We even have ideas of having an image registry that much more closely map maps what the containerd image store is like so that you don't even have, like, an intermediate step with images are are are stored, and we can kind of seed containerd directly, but we haven't done that yet. So but but in essence, like, local things are all complete in your machine. We don't have access to them. You're just using the the software. There's other arguments to to prepare. So you can prepare EKS, for example. And when you prepare EKS, we set up an environment to use

32:06 Amazon's image registry, so ECR. And then if you use if you use prepare existing, then you tell us which registry you want to use, and that's really up to you to decide how how that's set up. And we this this set of options will increase over time. One interesting thing is that not in the current implementation, but kind of part of our design ethos, if you will, is is extensibility. We want we we care a lot about, like, this core experience, but we know that there's a lot of kind of good cloud providers out there. Not everyone is in EKS. Like, we

32:45 don't want to even have to have, like, a strong opinion on which cloud provider you should be using. So over time, we'll actually change this interface so that anyone can build a module that does the same type of setup, and and it will work in a similar way. You'll just tell us well, set up a environment with a module that lives in this repository, and we'll make sure that we build it. We we, kind of run it for you so that as a user, it's also something that just works out of the But at the moment,

33:16 we have these three options that are built into into NS. Okay. A question on the top of my head is the orchestrator component. Is this something I would run-in my production environment if I were using Namespace? So today, if you go and deploy to production, you do see it there. In our final design, no. We we our our goal is that production becomes as hair tie air airtight as possible. And the big the the the reason why we haven't done that yet is because we care very deeply about, having the minimum kind of access necessary in order to deploy to production, and

34:00 we haven't done that in a way that the orchestrator will live somewhere else and deploy to your production cluster. We we have done that in other parts of Namespace. So for example, we won't go through that today, but if you use an s three bucket out of AWS and you kind of instantiate that with Namespace, we out of the box set up an IAM policy that is kind of the minimum policy for you to be able to use that that bucket in your application. And those are the types of things that we would want to do

34:30 as well when we're setting up production, but we haven't gotten to it yet. Okay. Two questions. Does the lack of an orchestrator in the future in production mean that Namespace won't adopt or support get ops patterns, or is that something you just see being done in a different way? Yeah. So the orchestrator does need to live somewhere else. It just it wouldn't live inside of your cluster. So it will live somewhere else where your GitOps stack, if you will, would be running. So anything that is then consuming these events, we we we're strong believers in GitOps.

35:07 We it's something that it was our bread and butter for many years and and got this idea that configuration lives in the repository and kinda major changes go through a review process, and then they're deployed from from from the actual repository. We care a lot about provenance. Like, do we understand where those changes come from? And also being able to ensure that we have automated testing that goes into into those changes as they're getting rolled out. So we got big thumbs up for GitOps, and it's it's you'll see it kind of first kinda at at the force center in in in

35:42 Namespace. But but, yeah, we don't see that as kind of preventing GitOps. We would just have the orchestra just living somewhere else. Okay. Last question, and I'll let you plead the fifth if you don't wanna answer it. But as an ex Googler, I see EKS, and I don't see GCR or GKE. Was that an intentional decision? Is that something you plan to support longer term? Like, I'm just curious. Yeah. Well, we we care about I I think the most important thing for us is to go where users are, like, help people where they are. And it's no question

36:19 that a lot of folks are in AWS, So that's that's where we started. We have friends at Google. I I really like Google's products. GCP, I think, in the Kubernetes space is probably one of the best kind of managed Kubernetes solutions. And it may still show up in this mode that is kind of built into NS, but what I would really like to do is just get our extension system working for also these these part of the of the system, which at the moment, it doesn't. It only works for other parts so that whether it's ourselves

36:55 or someone else in the community can go and build that integration. Alright. Cool. So business decision, not a technical decision. I like Yeah. Okay. So I think we're prepared. Let's see what's next. You've already covered the dev host file. Now we want to develop our first namespace application. So we need to create a server dot queue and I'm just gonna copy and paste all of this. Yeah. I will do my best to speculate about what this is gonna do, but feel free to correct me if you wish. Sure. Yes. I trust myself. Yes. Good. Thank you.

37:05 Defining & Running the First Server

37:33 Just so well, I'll leave it up to you. There was something in the instructions. It asked you to put this file under a server directory. You can see what happens if you don't, if you want to discover a little bit. But because later on, we're going to add a second server so we can already be set up for for later. Okay. So you're right. The instructions that say create a package directory first called server. I'm assuming this is for the queue module system. So I will just do this. But if I'm wrong, then maybe we should dive into that

38:07 a little bit. It's it's so queue does play a role like we queue also follows, as as you probably know, well, like, the a similar model as Go that a directory establishes a package and the package being, like, a a unit of of well, in this case, configuration. We have the same concepts. We were it's lucky. I guess we're lucky that we all kind of follow the same semantics. And and similarly, so server will be a package, so it's something that you can refer to. And that when we're done with these first steps, it will have both the server

38:44 definition, which is what we're seeing in front of us, but also the code of the server. And and that's maybe something that is a little bit different from some of other tools out there. I know that Acorn also collocates application definition and and code, but but we we do and and for for a few reasons that I'm happy to kinda go then more into. Well, yeah. I mean, I'm 100% in agreement with you and Acorn. Like, I have been working with customers and clients for for years. I always say you're deploying code to your developer code should all live right next to

39:20 the code that is orchestrating and working with. And it's amazing how many times you go to companies or speak to people at conferences and they're like, oh yeah, we have a repository that keeps all of that. And you're like, why is it your code? Yeah. You're creating a boundary that I think it's just it just impedes your ability to ship faster. So That's that's right. And and it's part of those it's it's a human pro it's a human challenge. Right? Like, it's probably someone didn't feel comfortable having those configurations there, but it's we care deeply about kinda helping teams kinda work

39:53 better together. So it's kinda part of our design that these that these live next to each other. Alright. Well, I I will help you spread that message as well as I can. Appreciate it. With regards to the server definition, I understand the name. Everything should have a name even if sometimes we struggle to name the name. I understand the service. This looks to me like a Kubernetes service of a core account and ingress that got it. I don't know what integration goal is providing here. So maybe you can tell us in on on this. Yeah.

40:26 I think the interesting it's it's useful to kinda and since this is kinda more I I know that your audience also of dives deep in into a lot of these projects. I think it's useful to think of our system as has a kind of simple to use control surface. Like, you write configuration files like these that are very simple, and and that's part of the goal. But behind the scenes, it's a programmable system. So, actually, what happens when you say integration go, you kick off, kind of a decorator, something that is going to add more definitions

41:04 to your server. And one of those definitions is a binary build rule. So we'll we'll, if you use the short form as as you're using here, as as the example suggests you to do, we assume that the Go code base lives alongside the server definition, and then we're going to instantiate a a build rule for for building that Go server for the code that was living next to it. The code will come now, as part of the example, but that's what the integration Go, in this case, does. It adds a few other things. So these decorators can actually, for

41:39 example, add services. So we have integration web, which kind of the production build builds static files for for serving, And you, as a user, don't need to say, well, here's here's where these files are being served. We actually, it is the integration itself that is able to emit that that definition for you. Okay. Let's work through this and see if we can get something running then. So I'm gonna copy main dot go. And I wasn't reading the instructions properly, but I'm gonna assume it goes in the top level. That may have been a naive assumption.

42:02 Go Mod

42:25 It goes in into the same directory as server, and then and maybe that's something we should clarify in our in our instructions. Okay. Next, are going to do and our workspace route. Okay. That's here. Which gives me a GoMod and a GoSum here. So main.Go goes in server, but the GoMod and GoSum go here. Is that correct? Yeah. In this example, we did it like this. You can have multiple GoMods if you prefer. We we sometimes we see different teams do different things where where we work with both with both systems. Okay. Or with both models.

43:09 Yeah. Okay. Sure. I use Go Workspaces quite a lot these days. Is that does Namespace just work that with that as a box that it doesn't really care that much? Like, especially in a mono repository environment with that. Would that be fine? I'll be honest with you. I don't know whether we would actually work out of the box with Go Workspaces. We haven't tried it. But we so the the way that our Go integration works, it it it looks up in the tree looking for the closest Go mod, very much like Go does, and then it uses that as a

43:44 root for building. Because whenever you want to have a referencing to other files, other packages from a versioning perspective, that kind of your Go mod defines that. So we kind of do some of the same things that Go Workspaces tries to do out of the box, but I I don't know if what what would happen if you actually put a Go Workspaces file in there. Okay. Well, that's not something for us to do in a livestream, but I'll I'll I'll play with that later on. I'll let you know. Okay. So now that we've done this, we're going

44:09 Developer Workflow: Live Updates & Tools

44:11 to run NS dev server. Now before I run this, we have a workspace definition, which doesn't tell me much. We have a main dot goal, which is just an issue to peer. We have some server dot queue that tells it to build a goal. So when I run NS dev, it's gonna compel the go application, build a container image, deploy it to a KCS cluster and expose it with a service on the ingress endpoint. Yep. That's kinda neat. Okay. And that's There's one more thing, which is it also like other tools that are this is more common in the JavaScript

45:04 or TypeScript ecosystems. But the difference between dev and other commands that we have is that dev listens. So dev is stateful. It as you're doing changes, it will redeploy those changes as well. Okay. I'm curious about the server. I'm assuming that's the directory, that's the package that we're deploying. If I do NSDev, does it do everything or nothing? You can write with everything. Okay. But but right now, we'll stick to to server. Well, that runs. We do have another question from Diego in the chat. Do we still need to write Kubernetes manifest for persistent volumes for databases like Redis or

45:43 is it automated? If not, how do we manage scaling? You you actually, I you you hit one of those bugs, but I think it had to do with, yeah, the the the module name that you use for for the workspace. You didn't use what we have with with queue before. That's that's my fault. So a Namespace bug. Is that right? Yeah. It should that should that should do it. So the question yeah. You can there's kind of two two sides to this question. We know that there's a lot of folks out there that already have

46:27 huge investments in in how they run their applications and how they provision them, how they build them, and we try really hard to work well with those. So for example, if you already have something that is provisioned outside of Namespace, Namespace can refer you can refer to it through Namespace. We won't automate it, but it it will just work. Now when you kinda bring when you come into our our ecosystem, we do make certain things easier. So for example, what you the the question was referring to volumes. You could can define volumes as part of

47:00 your application, and we'll manage those for you. We'll, set up, like, the the volume sources and kind of the amounts, appropriately into into the into the application container as well. We'll manage sizes. And then something that you'll be able to do is apply modifiers. This is not implemented yet. Actually, it was an idea that you, David, had some time ago that is still in the in the queue. You there's two ways that you can do configuration, and one of them is you can parametize within queue. So you can go to the queue file and say, hey. If this environment is for

47:37 development, do these settings. If this environment is for production or is named Fubar, do these other settings. So that's kind of one way of doing parametization, and we'll have a second one that is based on the target environment. So if you say that I'm deploying to production, and in production, I want this application to have one terabyte of of data or storage available in in its volume, you'll be able to provide configuration for that as well. But that's not implemented yet. Okay. Awesome. We also had a comment from your colleague just saying that persistent volumes are modeled and

48:07 Persistent Volumes

48:11 namespace configuration. So maybe we'll, I don't know what's in the demo, but perhaps we'll get to something that shows how that works. Our NSDev server now worked. We have a URL, which says hello world, which is our web application. So I'm gonna go off script for just a moment because you said something that is it monitors your changes. So I'm gonna make a change and I'm gonna go to the web browser. I'm not gonna do anything else. Hey. Look at that. We knew that you're going to do that change, and we've Yes. You intercepted the CCB traffic on the

48:56 the redirect. Yeah. In your local network. That that's neat. That's one of those common frustrations. Right? With with especially with compiled deployment and that developer experience is that typically you have to stop the world to the recall file, spread it back out. But that was almost magic to the point where I made that change and it probably took about four or five seconds. And I could see it on my little macro stage manager thing there. It was doing stuff in the background. I don't really know. I'm assuming it just rebuild the image and swap them out. Yeah. Okay. Yep. For go,

49:31 we do that. If you would use the Node. Js integration, we would copy the files that are changed. And and also we try always to find the right trade off with giving controls whenever whenever it makes sense. So for example, for when we build a Go image, for folks that are familiar with Co, we actually use a very similar method to Co. We use a local Go toolchain to build your binary, and it is Namespace that then builds an image fairly quickly and kinda pushes that into the repository. And and we're we we do a lot

50:08 of these things where we try to kind of really optimize for for the developer experience. What we also didn't see is that you never probably in your machine, you have Go, but you never had to install Go. We went and fetched Go the first time that you did NSDEV, and we placed it in our little cache. And whenever you're you're you're kinda building that that version is being used, if you change your application and you update your the Go version that your application is using, then we'll fetch a different version of Go. And and, again, like, you don't need to

50:17 Language Support & Build Process

50:44 manage that yourself. And I guess you're leading me on that question there. Like, what languages are supported then? Like, it sounds like go is it's got pretty solid support. I'm assuming namespace is right and go your go developers. I can assume assume assumptions all the way here. But you mentioned Node. Js as well. So like what Yeah. What are those first class languages where what does it look like for languages aren't supported? So we the integrations that we have at the moment are go, Node. Js, and we call it Dockerfile. There there's one more, which is Shellscript, but

50:47 Supported Languages

51:20 I'll get to that in a moment. So with Dockerfile, you can you you can write the Dockerfile to support whatever that you want. And and remember, when you say integration Go, we're actually what it's doing, it's writing some of those definitions for you, but you can always write them from scratch. You can have a server definition that has all of the services specced out, all of the volumes specced out, like, manually. So you can go and use, I don't know, build a Rust server, and and it will work. You just have to write a Dockerfile.

51:50 But we do believe that we can do better for users with these integrations, and there's a couple more that are planned out for Java and Python in particular. So those are probably the ones that will come after. But those will also be extensible. So folks will also be able to write their own and and contribute builders, contribute kind of deployment configurations that make sense for those. One of the things that this getting started doesn't cover, but our testing facilities, we capture information out of the servers after a test is run, And that's something that the integration can tell us about. Like,

52:31 oh, this is a Go server. It can you know, you can run pprof against this endpoint and go and fetch some some information out of that. So an integration will be able to do things like that as well. Okay. So when you said that you download, go to a local store and you use that, that's really just for cool. Right? You there's no need for you to download Rust and do stuff or any other language on time because that all happens in the container. Yeah. So we we in this right now, we haven't used BuildKit

52:42 Containers

53:00 so far. There there will be cases where we're using BuildKit, we're building things in a container. But for Go, the development model doesn't use a container for building. If you run it in CI, then it will automatically use BuildKit, so it will kind of run-in a container. So, yeah, it depends a little bit on on the thing that we're that we're building. And this system that we have, we we we kinda we thought about it, but we actually have since learned that a few other companies are using similar system. Cash App has something that over open source called Hermit,

53:40 which which is an ability it gives you the ability to basically manage kind of pre already released software in a way that it doesn't pollute your your your workstation, and and you have a lot of control over those versions. And that's similar thing that you're seeing here. If you run maybe just to show it once more, you you can can leave this one, and you can run n s t for tools or n s tools and kubectl, and then just use get pod if if you want. I I don't know how to kubectl. So you see here, we're actually doing exactly the

54:20 same thing. We are getting kubectl, for you, and, then we're we're running it. And by default, when you run when you use kubectl through NS, it will be configured with the right namespace and and and the right context and and all of that so that you don't have to kind of think about it. But there's always this idea that you don't need to go and figure out what to download. We'll help you download, and we can manage those downloads for you those dependencies for you. Okay. We have a question on the chat related to the it's gonna CI side of

54:47 Base Image

54:52 things here, but George is asking, will there be a base image of namespace that could be used at a build server such as GitLab? Would that be an interesting question. So right now, the only distribution mechanism that we have for Namespace itself is the binary. We don't yet have base images, but it would make sense that we would have a base image. So I think the answer is yes. We should have one, but we don't have it yet. Alright. Awesome. Hope that helps, Sean. Okay. Let's go back to our our guide. So next, it wants us to okay. We've seen

55:23 Adding Multiple Services (MinIO)

55:32 that. We've changed it. Is it gonna ask me to change it next? No. Okay. Cool. New new thing. We're gonna add menu to our setup. So is this a new server? Yes. Okay. So when we use the term server and namespace world, this is a software component, a package, something like that. Yeah. A package can have multiple things. We have servers, we have tests, we have binaries, and and then these are the things that live so server is a type of thing that lives in in a a package, but you can have others. Okay. I've decided to call the file this time

56:12 Class Stateful

56:16 random dot queue because I don't think it cares and I wanted to confirm. But if that is wrong, please feel free to just tell me to shop and rename it. We can try it. Again, we have a name, we have an image, we have no integration this time, but we do have something called class stateful. And then we have what I suspect is familiar to everyone. We have the ability to do environment, our services and mounts. This feels very Kubernetes deployment a. So let's focus on the class stateful. Yeah. It's it's another one of those decorators.

56:48 In this case, it's not language specific, it's behavior specific. And it will add a couple tests to our server. So for example, if you want to use a persistent volume, you need to you're you need to have a stateful class declared. You or else we won't know exactly how or how you want to manage that attached storage. It will also as you could imagine, when you're deploying to Kubernetes, it will instantiate a stateful set while other servers will instantiate deployments. But even those are not fully set, like, we we go from what's the intent of the user, that's what

57:27 we're trying to capture here. Like, okay, you want a stateful server. And then depending on the current deployment strategy, we'll we'll might be doing different things. So in development, we'll do a stateful set. In a test, we actually just deploy a single pod with with this server. Okay. So the integrations and the languages that support, there's a finite list. I understand that. The classes, what's the discovery method for those? How do people work out which classes are available for their servers? Yeah. Good question. We we've been working we we we chat a lot with the Q team.

57:45 Integrations

58:07 We're big fans of Q. One of the things that is starting to surface in that space is a powerful LSP, so we want to have integration to do Versus Code. And we've been designing our configuration to be as standard queue as possible so that we can benefit from from and contribute to that ecosystem as well. So we don't have that yet. We actually had a version of an LSP, but it wasn't with some changes that we've been doing, it hasn't kind of kept up to date. But that would be the discovery or or one of the discovery mechanisms that you would have.

58:44 And then, obviously, we also have a syntax reference in our documentation. We're also happy to hear ideas of how other folks kinda prefer to to kind of get to learn about these concepts. We we're trying to do a mix of documentation examples, and then in the near future also integration into into editors. Okay. Awesome. Hey. And those be for a second because this is a question I don't think it's gonna make sense to anyone else besides you and your team. But why server colon rather than like a queue definition? Like, what's the because I as soon as you said LSP

59:06 Why server colon

59:24 and standard queue, I was like, oh, but how does it know this the types for the server definition? Yeah. We haven't really used that. So is that I I was just curious. That's all. Yeah. So that's that's you're you're hitting exactly why our previous LSP work and currently doesn't work. So we the the first version of namespace requires you to do those uni unification. So you would have to say, hey. This is a server that is unified with this data type as well that kind of serves as as a type definition in queue. So you would have this server is a doll

1:00:02 hash server and which is very common and folks that are familiar with queue are are very familiar with. But we found that to kinda be distracting, honestly, and as we're kind of getting people to get to know about namespace. So we've removed the need for that for now. And what we've worked with the queue team is that we'll have, this is not implemented in queue yet, but we'll have the ability to define at the module level which base types are unified with a package. So there will be a kind of a base package, if you will, that we can

1:00:34 define, and then we say, well, if there's a server, that server is preunified with hash server. If there's a test, that's preunified with hash test. And that's how the LSP then will know because it will see kind of the unification. Alright. Next. Cool. Thank you. So let's see if my random dot queue worked then. I'm assuming I can run dev and we call this menu. Mhmm. And it's good to just pull in the menu image. One thing that we also don't we don't really it's not perhaps obvious to to folks that are that are seeing,

1:01:20 but your machine, you're running I guess this is an m one or m two Mac. And we what we did was we kind of used that information, like, know the environment that we're deploying to to to pull and manage images automatically. So you never have to think about well, obviously, there are cases in a in a developer's life cycle that you might need to know about the platform, but most of the times, you don't need to manage platforms. So if you'd go and do prepare EKS, for example, and your EKS nodes are only x eighty six sixty four, we'll automatically

1:01:58 build for x eighty six sixty four and deploy images of that target platform, which we obtained from the from the Kubernetes clusters. We actually build for that cluster. And we know that we're building for Linux because that's what's running inside of Docker, ARM 64. What do you do if it's not a homogenous cluster? We do a multiplatform builds. Nice. So we unify over all of the platforms that we see in the cluster, and and we build for all of them. And and, yeah, so that's something that Namespace manages for you. It automatically generates indexes. It parallelizes,

1:02:36 like, multi platform builds. I discovered some sort of UI that I didn't mean to click on, but I have clicked on. Yeah. This is a early stage of of something that we expect to be more that folks will interact with with more over time. As you see, it's it's it like, the the bits on the bottom are are not really working, so you may even have to I I actually don't know if if if you may have to go back. But what what's one of the challenges that we face as developers of this kind of multi

1:03:16 server world is there's so many logs. There's so many terminals. There's, how do I there's so many ports. Like, how do I manage all of this? And our take is if you say that there's a service, and by default, we assume that the service has a particular kinda meaning for you as a developer, we automatically forward it. So you're connecting inside of the cluster that is running inside of Docker. So this is not something that is available in your local network. We actually are doing that port forward automatically for you. If you press the server, you'll get logs.

1:03:50 If you go into the terminal tab, you get a terminal. And and that's something that we do for any server in the stack. And main servers is is a hint that you can actually have many of these. So you could even run NSDEV with with multiple servers and and and manage them all from a single place. Cool. Alright. I see buttons. I can't help myself but click. So Alright. Yeah. I like that. So the UI was nice. I'm clicking on one more. I thought I was checking on the manual UI, but it took me to this, but it that'd let

1:04:28 me get to the manual API as well. So nice to all of that stuff is just right there waiting for me. Alright, let's see what's next on our tutorial. I appreciate that. We're getting close to like our plan time and I feel like we've got so much more that we need to show people. So maybe you can kind of guide us. What what what should we be looking at next? Should we keep working through this or is there something that you think, hey, people need to see this. Let's maybe I think we can just you can

1:04:57 just copy this one definition, and we'll connect it to servers, which are the it's a thing where we we feel like brings value to to folks. I think this is you you probably can copy the whole thing. And Okay. It's just a change to the the server dot queue. Right? Yes. Exactly. So here, we're kind of connecting the two things. We're saying that, hey. There's this MinIO server. The Go server needs your MinIO server for folks that are familiar with Docker Compose. Like, it's a similar idea. But what's interesting here is that Namespace understands these definitions across multiple packages.

1:05:39 So when you go and set up, MinIO, you don't need to kinda figure out, okay, what's the port that is listening in and and what's the what's the host? You just say, like, in set up my environment, my s three endpoint from a service endpoint. So that means from the API that MinIO is exporting. And you see this it's kind of condensed into this string format, but it's package colon and then the service name. And you can refer to anything across packages. And as you're going to deploy, you'll see that that environment variable will have kind of

1:06:13 the right value for this deployment. And then as you do different deployments, that value will also be updated accordingly based on deployment that you have. There's just one nit because you named your module something else. So here we say example.com/nsexample. So that's our our package system is trying to that's that's how we're telling our package system, hey. Here's where to find this. But, actually, the current module, you named you yeah. I think you named it Rawkode Academy. So you we'll need to update one of them for for this work. Oh, alright. Okay. So this is the service from the

1:06:49 manual deployment. This doesn't change. This is a package which is the same. However, this bit here is Exactly. Okay. I've got you. And it was lowercase as well. Yep. And the same under requires. Every time something goes wrong, I'm gonna show it to my fault also. I'll take the blame. I'm also happy. You know, what's what very much all like, I so so very often as developers, as we're working on an application, he's handling errors. And, you know, something goes wrong, and and so I'm always happy to just see, hey. How is it failing? Because it's also an

1:07:27 opportunity for us to kind of understand, are we already doing a good enough job telling you how you can kinda fix things yourself, or or do we need to do better? Okay. So we have updated this with the requires block. And since I I probably shut it. Did I shut it down? Oh, no. It's still running. Okay. So this should already have happened then. Right? So it didn't because and this is this is influx actually. We're you you run NSDEV and then one server. So we're looking for changes to that server. We're not looking for changes for any change

1:08:05 inside of that repository. Okay. So we see here now that it actually said that there were two servers that were deployed, and, it will, by default, focus on the one that you specified, which was the Go server. But if you press s, it will cycle through and show you so one more once more. Yeah. And then you'll see then support services. You'll you'll see also the endpoints of these other servers that you're deploying. Nice. So if I open a new tab, run NS tool, keep control get pods. We have our menu on our server. And

1:08:43 Connecting Dependencies & Service Discovery

1:08:53 I'm assuming if I describe this pod, that s three environment variables just gonna be kinda chilling, which it is. Okay, cool. I like how it's connecting everything together and quite this is quite a neat way of doing it. It's like it's not overly I don't need to apply too much cognitive awareness to what's happening in the system. I'm literally just saying I need this variable and it's coming from some computed value within the stack, which is neat. And as it changes, like, you sometimes you'll rename the service or or now maybe that server is something else,

1:09:31 it will tell you. Right? Like, I think that's one of the most powerful things, which is keeping these semantic references in the code. Like, what's your intent as a user? You you actually want to connect these two things. So whenever that's not true for some reason, we can tell you, hey. This is not working the way that you would expect it to work versus just leaving some ports over there, and then you go and change MINI O to be on port 7,000 and and it just doesn't work anymore. Okay. So if I come in here, I'm

1:09:59 some Yep. Wildcard developer that's decided. Actually, this is a a TLS API and I just go and change this. I'm assuming this is not gonna complain and tell me that it can't fulfill It should complain. There we go. It it complains in a in a a fairly hard to understand way. Well, that's completely. Yeah. Exactly. Exactly. So it so it's still not it's still not exactly the problem that we should be seeing. Yeah. This is a different problem. It's just me not using DNS appropriately. Okay. Now we've got API is not exported by this module, which is a lot easier to

1:10:42 understand. Yeah. And you know why it it took a second to do to do that? Because it, internally, there's two phases. So, there's actually three phases. There's parsing as well, but there's planning and execution. And some of these values are only resolved after we go and deploy something. So we may have to deploy something until we know, hey. This is actually the endpoint, and we can go and update your, Go server deployment to point at that value. And so so that also means that we're out of the box doing things in order. So if you say when you say that

1:11:18 the server, kind of Go server, depends on MinIO, the first time that you're deploying, we'll deploy MinIO first, wait for it to become ready, and then we deploy the Go server. And this is less important during development, but especially in tests, which exactly the same environment is used for. It's really important that you kind of know that your test is starting at the time when the server is actually ready, and then we're initializing the right order. Okay. Cool. Got it. Since because just being mindful of time, but, obviously, David, you know best, I think what

1:11:50 Multi-Tier Application Example Walkthrough

1:11:55 could be interesting is just show a quick example of a a few more things coming together and from our examples repository. Yeah. That's good. And and then we can just maybe highlight a couple things from there. So if you could clone our examples repository, then I can guide you through it. What's the gahub Namespace labs slash examples. Alright. Where would you like to start? I would go to one of the most complete more complete ones, the multi tier. And then we have whatever the latest number it is. Sorry. It's actually hard for me to read. Yeah. Sorry.

1:12:48 The sidebar doesn't seem to scale with the font size. So we've got simple web secrets and web resources. With numbers. Right? So whatever number is the highest at the beginning. Yeah. And in this multi tier example, we have a Node. Js front end. We have Go server back end, and then we also have, a database, server. And, this covers a lot of the functionality that Namespace has. Maybe we could just run NS dev for it, but you'll need to run prepare local first. Okay. So do I go into the multi tier directory, do I do it from the

1:13:31 top? It's it's up to you. You can also do it from here. Alright. Well, let's do a multi tier, all three. That puts me here, and let's prepare local. So and and because we ran local before, we, by default, target the same cluster, but we use different Kubernetes namespaces for different application workspaces. So the previous deployments that you saw will continue in our local cluster. As we deploy things from a completely different module or repository, they'll they'll be in the same cluster but in a completely different Kubernetes namespace. And if you run NSDEV front end

1:14:17 So it's it's now there's a bunch of things happening. So the front end depends on the API back end, which is a Go server. The front end is written in Node. Js, and the back end then depends on Postgres. And there's different options of how to deploy Postgres. We're going to use, the in cluster setup that we have that is going to deploy a Postgres server in the same cluster. And, what you're seeing is we're we're actually building the Node. Js application. We're done with that. We built the Go application before as well, full parallelism. Some of that run-in BuildKit. Some

1:14:54 of that was local. We fetched the Postgres image. And now what we're doing is, we're kind of orchestrating these changes. So first, we're going to deploy the Postgres server. It took a little bit of time. We actually have a bug in the timings there, so but the o OSS Postgres server was the first thing to deploy. Then, the Go server uses a database that is created out of that cluster, so that is running what we call a provider. So that's code that will run inside of the cluster to go and set up the database for you.

1:15:30 And and then we run the ghost the we we deploy the ghost server, and then we deploy the front end. And if we look then at the server definitions, it's all kind of references very much like before. So we yeah. As you should Oh, no. I got it. I think you have two gets. No. It it's it's correct. I think it's just because you have two gets. It happens to me all the time. Yeah. There we go. There we go. Okay. Cool. So, yeah, we got a separate namespace. We have all of our services on this namespace.

1:16:09 We have our web UI, which lets us get to the front end of the application. Mhmm. Very neat. Very Yeah. And and if you type in into this application, it's sending your request to the go back end, which then is storing that data in Postgres. And I think what what the what we're aiming for is imagine you're in development team. You just joined that team, and and there's a stack that everyone is is kind of using to to kinda build their application. And how often is just a bunch of documents that you have to run through,

1:16:41 install this, and and run this command, and get authorization for that. And here, we kinda we we really try to encapsulate that whole experience into well, you have NS. NS will kinda manage the whole environment for you, and we'll get you to an to the same sort of development environment that anyone else in your team has because all of those dependencies are pinned. We know that they're the right versions, and and it gets you to that final setup. Okay. So you mentioned earlier when we spoke briefly about Node. Js that it works a little bit differently from Go

1:17:12 and that the system is actually synchronized. Now I've seen this with a few tools in the past and the challenge always seems to be that speed of reflecting those changes over the fail thing can be a little bit slow. So do you mind if we if we test that? Sure. So if I change the title of the page to Stargate is better than Star Trek, I like to be talking first. It reject the change just because it's wrong. Oh, no. I don't have Actually I've got local host. Where's my title? My browser doesn't show one. Okay. Let's make another

1:17:49 change. Mhmm. So this I'm assuming I'm gonna have to change some CSS or something background to be white. Oh, wow. That was fast. Yeah. But it's not good enough yet. It's it the the reason for it is we're we're we're we weren't inspired by some of these other tools. And what we see folks do is they look at your file system, looked at the files change, kind of ship that over to the container. Now that means that there will be kind of subtle cases where the changes inside of the container don't fully track what's in your workstation.

1:18:35 So we have a few ideas of how we can improve that. Notably, bringing to Kubernetes something that feels a lot more like volume volume mounts from from Docker, where it's literally like your file system that is exposed. And we're kind of keen to to do some of those. We just haven't gotten to it yet. But but we want to get to a point that developing in an environment that is actually Kubernetes feels as good, if not better, than local development. And we know that the TypeScript and JavaScript ecosystems are ahead. Like, it's much better for you to go and do

1:19:13 NPM dev or YARN dev today than deploying to Kubernetes, but you you lose so much on the way. Like, there's so many compromises that you're doing around how production like your environment is, and we want to get to a point where you feel that you're developing in a production like environment without making those compromises. So we're we're we're we'll continue to invest in in getting there. Okay. Let me throw a question out to you then. Like, we've really focused today on what that developer experience looks like with Namespace. Let's assume, know, people wanna start kicking the tires unless

1:19:48 they wanna loop it into their own systems. Is there any is there any penalty and it's drawbacks for people that only want to use it in-depth or is that like or is that like free go for it? This will this will help you and then it's up and to production or do you feel as a loaded question, but should people go all out and use Namespace from from start to finish? Yeah. It's a great question, and and it's it's a common question. So thank you for asking because it can sometimes feel daunting when you have

1:20:16 this all in one systems. Like, okay. Now I have to go through this decision of of changing everything that I'm doing. And we we're developers ourselves, and we know how costly these are. So our philosophy is we want to use the bundling on on for the benefit of the developer, for the for for the benefit of the development teaming as well. So you can go and use develop Namespace only for development. That's totally okay. You can have Terraform to deploy to production. You can use our image building system, but you can also just keep your Docker files

1:20:52 or anything else that you use to build production images. You can do that. You can then go from development and add whole system testing, and that's kind of a very incremental cost. There's actually almost zero cost for you as kind of a user of Namespace. And and and now you have a set of capabilities that probably you didn't have before compared with to the way that you were doing development before. And then you can also do production, and we care deeply about production as well, but it's it's opt in. It's not something that you have to do from day one.

1:21:27 Okay. Awesome. Thank you. I know how you are for time. If you have a hard stop or if we could maybe go a little bit over, but what's kind of top of my mind right now is that we've taken a look at this more complete example where if I stop playing with the JavaScript code for a moment and look at the server Is that this just says, hey, I'm an integration web. I speak to some sort of back end. When we go to the back end, we see that that has its own server dot queue.

1:21:31 Deep Dive: Resources, Classes, and Providers

1:21:57 This has some resources. So it's looking database and obviously this is like, you need postgres. Can we take a look because I'm assuming people are gonna get to a point where they wanted to do this. Already wanna replace all of my development pipelines with namespace. And my first thing is that, okay, well I use MongoDB for the CMS. I use Redis over here. There's gonna be a point where I have to write my own definition of what this server looks like. So I pulled up your foundation and I thought maybe we could take a look at the postgres one and

1:22:29 anything else that you think is interesting. So people get a feel and a taste for at some point, I'm gonna have to do some heavy lifting and as that actually heavy lifting or is this 20 lines of queue? Like, I don't know yet. And yeah, I'm happy to to still cover that. I I think it's interesting to kind of cover this this resource part of of Namespace as well. If you don't mind, just one minute before we jump into the into library, which where those definitions are, I would just say, I think it's just useful

1:23:00 to kind of introduce people to some of the concepts here so that we can so that we can map them back to to what we're going to see. So previously, we had, like, requires. So this server requires this other server. Now this is under the resource block, and this this server says, well, I declare a resource, which is a to dos database. It's of a class database, which lives in their database Postgres. So that's kind of the type is provided by this library OSS Postgres, so you can imagine that that means Postgres. And there's something called intent, so that's the

1:23:37 the definition that is used as an input for this resource. And in this case, it's, well, what's the database name, and do you have a set of series of schemas that you want to be applied automatically? And then we there's a kind of an advanced feature here that we can pass resources to resources. So it's actually a composable system. So we we pass the Postgres cluster in, and which we can have as many clusters as you want. So this system is kinda recursive or supports recursive definitions. And so class is a type. Provider is is a thing

1:24:17 that provides an instance of that type, and that's what we're going to see back in the code base. Okay. So I wanna see if I can make that a little bit more concrete for people to understand. Sure. So the class says, this is what the definition of a post grad database looks like some sort of contract. Don't think it's important what that looks like, but the provider and please correct me if I'm wrong, could be OSS postgres may be running the container image from the official Docker Hub library. But in theory, this could be RDS postgres

1:24:52 and that's what actually spin up and manage postgres on AWS. So the provider changes that. Is that correct? That's exactly correct. Okay. Sweet. The intent, I didn't fully understand other than is inputs and I understand that this is the database name and this is the some local schema that it wants to run. So I'm just going to take your word for that. That's what happened. So assume that is this defined by the class? Is this what determines what else is in this property, these fields here? That's right. It's the class defines two types. One is what are my inputs

1:25:31 and what's my output. And so in the case of inputs for a database is name and a set of schemas schema files, and outputs are the Postgres database endpoint credentials that may be needed to connect to that endpoint and the database name as well. And those are then injectable into your server. So if you go up in the same server definition, oh, or perhaps down. Sorry. Oh, sorry. We actually so there's two ways that we do injection. And in this example, I had forgotten that we we dynamically consume that. We automatically generate a config map that gets in that

1:26:22 gets mapped into your server with runtime configuration, including these resources, including what are your own services that you're exporting, what was your own image, which development environment are you coming from. And this example is reading those from that file, not but you could map them. You could inject, for example, the Postgres endpoint into an environment variable the same way that we did it before. But rather than being from service endpoint, we would say from resource field so we could actually go and fetch that value. Nice. Okay. Cool. Is there anything else in this file that

1:27:01 we need to go over before we look at the library? No. There there's just another concept here, which is tests, and and maybe I think folks can also explore that by themselves. If you run NS test, what we're going to do is we are going to instantiate the same stack that you saw so we cannot deploy the same set of servers, but in a separate namespace. We call it an ephemeral namespace, which is used for this for for that test. And then we deploy a client binary, which is your test code that you write yourself in whatever language that you want that

1:27:04 Automated Full-System Testing

1:27:34 is running as a client inside of the cluster. So it can do it can it has private networking. So it can reach the the server endpoints directly. And our philosophy for testing, not for everything, but but is but for, comprehensive testing is having wholes whole system tests. So kind of writing a client as if it was a real client that is exercising kind of the real API passage as you would inside of the cluster, and that's what this testing subsystem allows you to do. Alright. I'm gonna give a huge shout out to your teammates who have tackled a bunch of

1:28:09 questions. Well, we've been They're awesome. Yeah. But there is one right at the bottom there that hasn't been answered yet. And I actually feel it's quite important, so I'm I'm gonna throw that out. But why I'm always asking about secrets? How do we handle secrets in the Namespace environment? Yeah. So we we have a secret implementation. So secrets are a first class. We understand secrets in Namespace. And the implementation that we have manages secrets on your behalf, but we focus on, I think it's fair to say, like, simpler team, based scenarios. We don't focus, like, full productionized

1:28:53 secret scenarios yet. Like, the secret implementation for production will be pluggable, like other parts are in are in our system. It's not the case yet. What we do today is you you can define a secret here, and then we'll tell you, well, do you need to specify a value for it? We'll, we'll give you a command, which is n s secret set that writes a local file that you should commit into your repository, but it's encrypted. And, it uses a series of a series of keys for those that are familiar with age, which is kinda a

1:29:28 tool that is kinda commonly used for for kind of encrypting data. We we we manage age keys for you. So each developer has an age key. Then everyone that should have access to that secret file is added to that secret file. Every time that you add new secrets, we re encrypt the file so that everyone in the team has access to it. So whenever you go and deploy something, you have direct access to those secrets. They're never committed clear text into the repository. Then when it goes to stored? Only in your local machine. Never never revealed to us, never revealed to the

1:30:03 repository. Okay. So what if I change machine, do I lose access? By default, yes. You can generate a new key. So you go NS keys generate, and and that behind the scenes uses age to generate a new key, and then you can ask a team member to add you to the to the secrets file, which when we add a reader, it re encrypts so you you will have access to it. And or or you can use your previous machine to do that yourself. Okay. So I should probably just back up that private key, like, if I plan on

1:30:40 working for multiple machines? Honestly, like, the model that we followed is more like a key per system. So it's kind of easier to reason about revocation. But having said that, this is not designed for production. This is really just to assist development. What often ends up in these files are secrets that are used for development, And we'll have a we'll have something that integrates with Doppler and something that integrates with Vault later on as part of Namespace when you go to production. You've already mentioned my favorite secret provider. I use Doppler every day for everything. It

1:31:17 is fantastic. And the free tier is ridiculous. Like, I just work in projects on my own, I never have to pay them any money and they do all the time. Yeah. There's there's I must say, I mean, there's incredible software out there and we we try to lean on it whenever it makes sense. And, yeah, someone else already tackled secrets for us. And for us so so for us, it's really just about going and doing a good job of integrating with them so that it doesn't feel like, you as a developer have to go out of your way.

1:31:45 And, you know, I I I can imagine how many people have gone through this, and I I'm just tired of to use tool a, go to login, go to settings, generate key, copy key. Like, we we're trying to kinda really streamline that whole experience, and and we even have ideas of how to incorporate OAuth into OAuth web flows into some of these so that when you go and provision something that can be can use OAuth authentication, we actually kick you into a OAuth authentication flow so that you don't have to manage any more keys. But these are these are things that will

1:32:25 come in the future. They're not there today yet. Very cool. Alright. I've already kept you way over time. Is there anything else you wanna show before we wrap up to this session? I think we could kick off or or we could just finish this since we're here. We could just do NS test because it's one of those things that I think folks I hear people, like, we cannot run, like, comprehensive tests whether whether we're testing our front end or we're testing our our back ends, and it's a lot of folks end up leaning on mocks and and and

1:33:04 really kind of underrepresented systems. And we're trying really hard to make that simple but also cost efficient. Like, you don't want to wait for hours for it to for these run tests to run. So when you define a test with with NS, as I said, like, we're just going to deploy that stack for you, and, and then your testing code becomes a client inside of the cluster. And what we saw here, like, you're running all the tests that are in the repository. By default, the strategy is one at a time. So we build in parallel, and that's

1:33:38 why you see a lot of things going on right now. We're basically building all of the tests and all of the examples. But then we're deploying one test at a time, and that's what we saw before. And then we said kind of collecting collecting post execution server logs. We'll see that at the end that we run that test, and then we not only collect the logs of your test driver, but we also collect the logs of all the servers involved in that test, during the duration of the test. And in the future, we'll also collect

1:34:11 metrics. We'll collect network traffic between those tests so that you can understand, what actually happened. You can debug, and you can, better review some of the changes that you're doing and how they affect some of the tests. So we use tests a lot because that's a way for us to have confidence that, that we're shipping things at work. But you'll go through our code base, and you'll see very few unit tests, actually, but, quite a bit of these kind of end to end tests because we we really want to go for that full represent representation.

1:34:44 And so here, it's going it's because it's running everything, you'll see it just takes a little bit of time. We have options that allow you to increase parallelism, and we have a product coming next year that, we call them instant clusters where, whether it's in our infrastructure or on on your infrastructure, you'll be able to spin up, like, a cluster from scratch in a few seconds, and and the target is is exactly kinda running this test. We want you to be able to run twenty, fifty, one hundred tests in parallel and but in fully isolated environments.

1:34:50 Secrets Management

1:35:21 Yeah. This is pretty neat. I had no idea I was running every test in the entire repository, but I'm also kinda glad to edit because it's been kind of fun watching us kinda scroll by and then tick, tick, tick, tick, and then we got past past past past. So, yeah, that was cool. I like that. It's it's also satisfying in some way that it does all that work and then and then obviously when when things work, it's good. But but, we developer experience is so important. And, right now, we'll tell you that, well, you're testing in past and and that and

1:35:54 and here's why. But we have so many ideas of how we can improve that whole troubleshooting, journey that you go as you're developing software. We want to bring into Namespace so that, again, you don't have to go and figure out how to do diff use different tools for that. You just run NSS. It gives you a in a kind of a URL that you can click. It will show you what happened, whether it's logs, whether it's traffic, who called who, and and just kind of so that you have, like, a tool that helps you as you're building a robust software. We care

1:36:31 deeply about robust software. Awesome. Well, that was really cool. I I think you're you're sneaking in just before the December 1 is like potentially my favorite project of 2022. Like this is super cool. And it's one of those spaces that I am very passionate about. Think especially with Kubernetes adoption, microservices, cloud native health to some degree, is that a lot of the things we used to take for granted that were simple and they become inherently complex. And Namespace with the way that it provides these obstructions and reusable components and provides a developer experience. I'm very excited about it. I've gotta be

1:37:10 honest. I'm I'm very very happy that you joined me today and I can't wait to see what comes next. I think there's so much more. Sorry. I'm just gonna keep blabbing at you, I think there's so much more we we can show and I think we're gonna have to schedule some sort of other episodes in the coming weeks or something like that just to to get through all. I would love to come back, and I I really appreciate the opportunity, David. And on I've been a follower for for for a long time, and so it's it's actually

1:37:15 Conclusion & Future Outlook

1:37:36 a pleasure and an an honor to to be here because I I I really appreciate how you approach kind of problem solving and and and your passion for for the ecosystem that we share. I welcome folks to try it out and, you know, and come and talk with us. Like, we we're developers just like you, and, we want to build great software. And, we'll only be able to build great software if we, learn from all of you on what's what's kind of important and what are the things that we should be looking at. And if you're passionate about open source and want

1:38:08 to contribute, we're happy to have you. So come on board. Awesome. Well, I'll make sure that the links to the Twitter and Discord are in the show notes shortly after this. So if you've watched this session and you like what you see, join the Discord, ask questions, get help and even contribute back. There's lots to be done. But the start that we have here is very exciting. So Hugo and to your team in the comments, thank you so much for all your hard work. I'm going to experiment more and hopefully we'll speak soon. Any last words? The big props to my

1:38:37 team. Yeah. They're they're awesome. Like, I wouldn't be here without them. So thank you. Alright. Awesome. Have a wonderful day. I'll we'll see you all soon until next time. Bye.