0:00 Holding Screen

0:00 Alright. We are now live. Awesome. Hello, and welcome to today's episode of Rawkode Live. I am your host, Rawkode. Today, we are gonna be taking a look at Dagger. It's a new programmable deployment system. That's a little bit vague, but we're gonna dive into that in just a moment. First, we've a little bit of housekeeping. If you are not subscribed to the YouTube channel, just now would be a great time to do that. Click the bell and you will get alerts for all new episodes of Rawkode Live. There are episodes weekly and we try to explore as much of the software

0:57 Introduction and Welcome

1:27 available to us on the cloud native landscape together. It is vast, so we will do our best to present it in a nice, digestible, fun, and educational way. If you want to chat with other cloud native enthusiasts, you can join us on the Discord server at Rawkode.chat. There are now over 400 of us in there talking all things Kubernetes, cloud native, and everything in between. Come and join us, say hello, and I look forward to meeting you. Okay. Now for today, to guide us into our introduction to Dagger, I'm joined by Solomon and Andrea from the

1:50 Guest Introductions

1:57 Dagger team. Hello, both. How are you doing? Hello. Hey. Thank you. Good to hear. Thank you both for for joining me today. If we just do a quick round of introductions, we'll start with you, Solomon, since you're next to me, and then we'll move on to Andrea. Hi. I'm the I'm Solomon. I'm the cofounder of Dagger, and before that, cofounder of Docker. And, hopefully, not gonna go through any more startup names that start with d and finish with r. Awesome. Thank you. Andrea? Yeah. Hi. I'm Andrea. I'm the cofounder of Dagger. And before that, I was neuro engineer

2:38 at Docker. So I worked in Docker for seven years or so. I was part of the original team with, Selman and Sam. He's not around. Awesome. Well, thank you very much. Yeah. There's Oh, sorry. You go. There's three of us. Yeah. There's three of us. And Sam, Andrea just mentioned, had more important things to do than hang out with us. Am not offended whatsoever. It's awesome to have you both here and, you know, maybe in a future episode, we'll get all three of you together for a bit of fun as well. But for today, we're gonna explore and introduce our audience here

3:13 to Dagger. And I think the best thing to do would be start with a little bit of a TLDR. What is Dagger? Sure. So Dagger is your application delivery super glue. So it it takes all the existing tools and systems that you already use to deliver your app. Could be your source control, your build, your CI, your Kubernetes cluster, your Lambda functions, whatever it is that you need to ship the app to the cloud, and it helps you tie them all together in one place, in one system that you can look at and manage with a simple interface.

3:20 What is Dagger?

3:51 Usually, you already do that with a bunch of scripts and manual operations. And later, those scripts and manual operations, when your application grows, you call them a platform, but they're still mostly scripts and manual operations. And so we that's your glue, and usually the glue has problems. It's hard to change. You forget how it works because the person who wrote it left, and so it slows you down. And so we replace that unreliable glue with a better glue. Basically, that's what Dagger is. Alright. So Dagger is bash for the cloud native editor. Is that is that a nice way

4:34 to summarize that? I think that would be an accurate way to summarize it. Yeah. Yeah. Because it's programmable. In Bash, it's programmable. Yeah. And just one more thing. I think that what we're aiming for is to solve the problem of making application delivery actually programmable. Like, it's actually code. And right now, we're almost there as a as an ecosystem, but we're so we're getting just just close enough that we're in the uncanny valley. You know? It's a little almost like code, but it's not actually like code, and so it's weird. So we're trying to bridge

5:10 that gap so it's actually really like code. Yeah. Okay. Can we then maybe give a very concrete example of a a use case for Dagger? Like, what you know, as a an application developer working with cloud native technologies and I wanna do something with Dagger, what would that something be? What's a good use case? So from the point of view of the developer, basically, what you want is something like Heroku, except, that actually works with the application you have. And so if you're you if you're lucky, you can actually use Heroku or something like it for the whole thing. But most of

5:14 Dagger Use Cases

5:50 us are not so lucky because not everything fits in something like Heroku, and so you end up, you know, back with the glue. And so, what a developer wants is one command or one click, and there's a staging environment. You know? There's a pull request. Boom. I've got another environment to review in the production like environment if that works. And, of course, then I wanna continuously deploy to to production. But just this idea of defining one place, what does an environment look like with everything running together, and then how do I get how do I get one of those? You

6:24 know? So from the developer's point of view, that's what you get. And from the delivery person point of view, so DevOps person, the DevOps team, the platform team, whatever you wanna call them, the SREs, the expert knows how to deliver. You wanna be able to program that exactly the way you want it and change it over time, you know, to to improve your infrastructure, to improve the workflow, whatever, without having to change the disrupt the the workflow of the developer every time. And so you kind of have this separation of concern. So there's two different users,

7:00 interacting with Dagger. One at the front end, the developer wants to deploy, and then one at the back end, you know, the delivery expert who wants to customize the the deployment. Awesome. So then pick your application, pick your stack. You know? Whatever it is, you wanna be able to get a staging environment of it with Dagger. Okay. Awesome. That's a a nice description of it there. I like that the crosses the developer and the operational side of things together and tries to get this concrete representation. I think you said what an environment is and how to deploy my application and so

7:35 forth. Yeah. You mentioned that it is programmable, and I I know that this is is built on Q. Do you wanna maybe just give us you know, they need to tell us what Q is, but maybe why you went down the road of using Q for for Dagger? Sure. So Dagger is built on two really important components, both of which both of which are open source. One is Q, and it's a programming language. So when you program your, delivery environment, It's not like Bash. It's not Bash scripts. Instead of Bash scripts, you write Q. There

7:45 Why CUE and BuildKit

8:10 might still be Bash scripts in there. You can orchestrate those Bash scripts if you want. But Q is a declarative language that lets you declaratively describe what the nodes of your, delivery graph are and how they're connected. And the key insight is that delivery is a graph. And so CUE is a language that works very well to describe a graph and how the pieces of a graph are connected. So, anyway, we can go into more detail later, but, the other piece is build kit. And that's the part that actually runs the runs the operations in the graph. Because with queue, you

8:48 can describe what what you wanna deploy. You can even describe how. But at some point, something needs to actually happen to deploy, and queue won't do that. You need an engine to interpret those those that that that description and actually do it. That's where BuildKit comes in. And BuildKit comes out of Docker, and it basically lets you orchestrate really powerful, pipelines of operations with running in containers. And plugging in inputs and outputs from those operations, you get built in caching. It's it's very, very powerful. So Q plus build kit, that's the the tech foundation of Dagger.

9:31 Well, I'm a a huge fan of Q, and I love seeing it in the wild. So this is really exciting for me. And I haven't actually until Dagger, I've never heard of anyone using BuildKits kind of Dag environment or programmability or whatever you wanna call it. And an application that would be to be a bunch of tasks to run. It's normally just building a container image, but you kinda just made a bit more generic, I guess. And it seems to be working really well. So I think That yep. Sorry. That's that's a testament to how good the build kit

10:03 developers are. You know? They basically, their job was, hey. Can you just make Docker build less terrible? And and they just felt like just something that there was so much more than that. So we're just taking advantage of it the way we think it should. And I guess that explains the name Dagger. It's a dag. Is is it that simple? Yes. Awesome. Alright. Well, I invited you on here to get hands on with Dagger, not to talk about it too much, but thank you for using a little bit of time there to share that. I hope our audience appreciates that and now

10:29 Getting Hands-on with Dagger

10:36 has a little bit of a clue of what we're gonna be diving into today. So I am gonna pop my screen share up. What we have here is the Dagger documentation, which you have kindly tweeted out in the last twenty four hours that you're very worried about someone going through this. So it's gonna be a lot of fun as we we work our way. So Just because we've seen other people go through it, we know what happens. So Alright. The Dagger is alpha. It's very much in the the testing experimentation phase. I'm not sure how you wanna describe

10:45 Installing Dagger

11:08 that. But if people do wanna get access to this after watching this video, what is the the procedure there? Can they reach out? Is it open alpha? Or is it? Absolutely. Yeah. Dagger.io. Just go there. Click request access. There's a very short form, and then you're on the waiting list. And, we're sending invites as fast as we can. GitHub, if you can hear us, we can only send 50 invitations per day. That's the bottleneck. So, yes, that's the way to to join. Alright. Awesome. Yeah. Dagger.i0 for everyone who wants to request access after working or watching us go through today's demo.

11:48 Dagger CLI Basics & Cloud Service

11:48 All right. I'm gonna move on to step one, which I'm assuming is to install Dagger. I see that we have a brew tap here and I'm gonna get this running quickly. I made the horrible mistake last week, maybe ten days ago of installing Mac OS Monterey 12 and had to wipe my machine about one hour after it because nothing worked. So it's been a fun last week having to solve every single tool in the world again. But hopefully, brew won't take too long with this. Well, as you know, I'm on an iPad, so I don't have any of those problems.

12:31 I have all sorts of other problems. Yeah. We've already discovered a few of those with the streaming software so far. That's been a whole lot of fun. So if you could see Solomon's lips moving and no words coming out of his mouth, it's because he touched his iPad. Alright. Let's assume that is good to pull down just now. Let's pull in that baiter release. Yeah. There we go. Alright. Oh, my computer is struggling. There we go. Cural bash is also an option for anyone who's on Linux. And then you can also pull down the release. All pretty standard stuff so far.

13:08 Oh, that's a expander. Let's go on to our basic usage here. So I have Dagger on my machine. Let's see if I run it. There we go. We got a whole bunch of sub commands here. Is there anything interesting that you wanna point out before we work through the little tutorial that we have? Yes. One thing I wanna point out is some of these commands, if you run them, they will panic and say nothing. And we have we have merged deploy requests. They're hiding them, but I don't think we've released it yet. Alright. I mean, the main thing is it

13:48 it changes a lot. We're we're still iterating based on feedback. So the most important one is up. Alright. Dagger up. I also see a Dagger login and a Dagger log out. Is that an indication that there's a cloud component to Dagger that runs my tasks for me? So, no. It doesn't run the tasks for you. We're it's we're never gonna run the tasks for you. It's always going to be a % on your infrastructure. But, optionally, there will be a cloud service that you can connect all of your Dagger engines to and tells you interesting things about it and

14:24 lets you control them as a fleet. But it's always gonna be kind of optional management and never actual infrastructure. Got it. Okay. Also, you type Dagger login, that Dagger login, it will panic and say not implemented. I know you're gonna have to fight the urge to take that, but I'll do my best. Alright. We got a a couple of comments there. So first one is let me drag that away from your head there, Andrea. And Nuno says, it's not Admiral Dagger. He's very disappointed. I'm not sure if I should know what Admiral Dagger is. I feel like there's maybe

14:48 Audience Questions (CD Comparison)

15:00 some sci fi reference that I'm gonna purely miss, but cool. DS has got a question. This sounds like a CD pipeline. How does Dagger differ from other continuous delivery platforms? Do you wanna tackle that now, or do you wanna wait till after the demo? Sure. I mean, I can maybe a few pointers now, and we can dive in later. But one is it's programmable. So you can actually, you can you can express an arbitrarily complex, delivery workflow in code, and you can actually export it as a package, and someone else can import it. And it actually works the

15:33 way you would expect. That's one big difference. Native artifact support and encrypted support secret support. So you can compose not just the flow of values. You're like, oh, the a little JSON object can go from step one to step two. Great. You can actually compose artifacts. And those artifacts can be injected by the user. They can be generated on the fly. They can be downloaded from elsewhere, but you can actually plug these components together, and you get a you get a chain of artifacts, bundled into, a reusable software component. To our knowledge, nobody else does that.

16:00 Dagger Example: Static Todo Deployment

16:11 But I guess that's a little abstract maybe. So I'll just kick that can down the road. Okay. Well, let's clone the exam. Yeah. Sorry. On you go. I'll clone Ashish now if you wanna say something else there. No. No. I I talk a lot, so I gotta stop myself at some point. It's all good. Alright. We've got three examples here. We clone this. The first one we're gonna be tackling from this tutorial is the to do application. Let's just take a look at it. I've always been really bad at reading docs. I always just wanna run the thing. So

16:20 Exploring the Example Application Code

16:45 I'm gonna pop open Versus Code and we'll see what we've got in this directory. Come on, computer. Getting there. I'm not sure why it's struggling today. Okay. This is a a node application, I'm assuming, by this package dot JSON. If my computer catches up, we'll see it. It is alright. It's a React to do application. I think I've seen a thousand of these in my lifetime. Mhmm. Oh, beach ball, that's not good. No. Okay. Well, cool. We've got a Docker file, which is on this playing catch up with my computer today. Maybe I should get an iPad. So that's

17:32 intentionally very simple and almost boring React app that you could substitute with your own if you want. Yeah. So this is just a Dagger. This is just showing how to use Dagger, I guess, to to build a node application container image. Does it do anything else? Like So we actually it so well, you'll see. But Yeah. Yes, it does more. You can just tell me to shop shop and read the docs. That's perfectly acceptable answer here. Alright. Dockerfile, like, it's pretty standard. Right? This is the There's nothing yes. Nothing special on that Dockerfile. We didn't I don't think we even wrote

18:11 that Dockerfile. I think it's the standard one from the example. Right? I'm not sure, actually. Okay. And we have don't we have. Just the Docker file. Nothing. Just the yeah. Yeah. That script is actually that that there's a little story behind that one, but I I will wait. I'm curious I'm curious to see see what conclusion you you you reach. Alright. Alright. We have let me fix your bash header here. So user and ENB bash. Oh, go ahead and think about interruptibility here. First pull request. Alright. So it's looking for some sort of keys fail

18:58 and does Dagger help? H. This is using the h two. Yeah. I don't know what's going on here. Should I should this should this make sense to me? No. No. There should be a comment at the top saying, you can ignore this. You should ignore this. You're gonna run you're you're gonna have to run it later, and then the the tutorial actually explains what it does. Alright. This PR will be pending after this call. There we go. Yes. So I think the first thing that's jumping out to me, like this is just a standard JavaScript application with a package dot JSON,

19:35 some random script you've already told me I can ignore with some JavaScript code. We do have a dot Dagger file. That's the only thing that appears to be different. So the way that we work and interact with Dagger, is it through this dot Dagger directory, or is this something else I should ignore? Correct. No. That's it. But Alright. The answer is easily easily answered by reading the next the next paragraph of the docs. We don't need docs. I'm gonna make this up. So Okay. Go. Go. I see values fail, which which makes me think of helm.

20:02 Understanding the Dagger Plan (S3 Deployment Example)

20:12 And then we've got some queue files. I can see there's a YARN package, an s three package. Okay. I know what this is. So this is a Dagger. What would you call it? Would it be a Dagger manifest, a Dagger package? Is it just a do you have a name for it? Yeah. It's a plan. A plan. Dagger plan. Oh, in fact, it's right there in front of me in the the directory. Okay. This is a Dagger plan that builds a container image, pushes it to and then does a YARN publish, guess, upload it to s three and

20:48 I get a sapphic website. Is that close? No. Actually, it does less than that. The the it only it it deploys a a mini staging environment of your React application, just the front end, on a shared s three bucket. And the the ECR thing is actually an implementation detail of if you go to the names file so the names file, that's a little a custom custom logic written by the delivery expert to run, you know, the the funny name generator from from Docker, you know, the container names? Mhmm. So it actually runs a Docker image

21:33 with that little that code that name generator inside it, and that spits out dynamically a cute name, and then it injects that in the configuration. And it does that so that everyone doing the tutorial can deploy the same s three bucket, which is ours, and everyone has a separate little directory in that s three bucket. So it's sort of like a poor man's multitenancy so that you can deploy something right now, basically, on our s three credentials. So you can actually deploy something without having to fiddle around with infrastructure. Alright. I got yelled out from the audience

22:04 because I didn't drag my terminal up so they could see the last line, so I have now fixed that. Apologies at all. And I should know better. I do that every single day. Okay. So we've got funny names. We've got a Dagger plan. I mean, I could continue to ignore the docs and be entirely useless and just run a dagger up and see what happens. Dagger up. Yes. Run dagger up. Alright. So it's telling me that it tried to load some identity from this dagger keys fail, a script that you told me to ignore. So okay. Exactly. So you should run

22:23 Executing the First Dagger Plan (S3)

22:38 it. Should run it, but ignore it. Ignore the content that just ran it. Run it without reading it. It sounds like a really bad idea, but okay. Yeah. Let let me explain let me explain what you're doing. So by the way, the docs explained this. So the the so here's our thinking. Oh, we should do a tutorial where you can do something really easy, really quickly from just right away. But oh, but how would we do that? How do we get you to deploy something without telling you, oh, first, go to your Amazon account and, you know, get some credentials

23:16 and inject them. So what we did is we preconfigured an environment, with our own s three credentials properly locked down so you can't just, like, do stupid things. You can only upload to that one s three bucket, and then we preloaded it in the environment in the Dagger environment. So when you run Dagger up, you're actually some of the input values are preconfigured by us. And that's actually a really useful collaboration pattern where your infrastructure team can preconfigure things for you. And then you by running Dagger up, you finish deployment. And so you get this kind of

23:50 Heroku like experience where you don't have to know all the credentials. The problem is in this usually, you do this in a small team or in a team where everyone knows each other and someone decides, okay. This person has access to the credentials and this other person and no one else does. In this case, it's kind of a weird special case. We want the whole world to be able to deploy to our s three bucket, and that's actually not supported by Dagger maybe for obvious reasons. So, like, because Dagger it will not allow us to not encrypt

24:21 the, the s three credentials. You know what I mean? So, basically, our workaround is to give the whole world access to this particular private key that is used only for encrypting these s three credentials. And so the script basically adds to your key chain this special key that will give you access to the tutorial credentials. So, basically, it's a script to hack the tutorial environment. Okay. So you have a published s three bucket. Rename it to hack tutorial that I say to be more then everyone would run it for sure. Exactly. Alright. So there's an s three bucket that

24:57 you provisioned. You're distributing the private key, which gives us access to the encrypted secret for Dagger to be able to upload my application to it. Okay. Right. Let's try Dagger up again. Oh, and I should really have Docker running, shouldn't I? Yes. You can also have a build kit daemon. You only just need build kit. So if you have a a non Docker build kit setup that that sometimes happens, we support that too. You just need build kit. Alright. Okay. Well, we'll just give Docker a few seconds to start up, and I will use that

25:34 opportunity to read the documentation there. So look at that. The next line, import the tutorial. Alright. Okay. So we've done this now. Actually, yeah. Yeah. It's basically exactly what I said, but in written form. Well, it's nicer to hear it from you than it is to read it from the the documentation, so I'll take it. I already mentioned I like talking, so not a problem. And then next command is Dagger up. Assuming I started Docker, it's on the way, and I think we kinda covered what we expect to happen here. So Mhmm. We also have access to a Dagger list, which is

26:10 going to, I guess, list all of our plans or environments as it says here. Yeah. So yeah. So everything happens in an environment, and Dagger lists lists your environments. And then each environment has a plan, which is like the source code of the environment. It defines what it should do. Alright. Awesome. And then we got input less. Okay. I wanna be able to run these Dagger commands, so I am not gonna jump ahead here. What we'll do is we'll try our Dagger up again. It's working. There we go. The first time it's gonna be a little bit slow because

26:55 it's starting up build kit. So it's gonna take like fifteen seconds or so, but it should be fast more time. Yes. Those get runs as a spawn container on the Docker system, doesn't it? And then yeah. Okay. Yep. That's how we get all of that funky caching stuff that you mentioned about earlier. Yep. No. No. That's correct. Because of Docker desktop, we all have access to build kit already. Very convenient. Correct. Also, any modern CI system has some sort of way to run containers, we can piggyback on that too. So you can embed Dagger in,

27:34 you know, your GitLab or GitHub actions setup. No problem, which is very useful. Okay. So let's let's see what it's doing here. Scroll up. So it generated our funny name. So does that mean that I've pulled down that container image from ECR and actually got it to spell a name? We got something to do with source computing and completed. I don't really know that yet. We got our app source to the same thing. Should I know what that is or do we skip past that? No. You don't need well, as the as the developer who just wants to deploy your thing,

28:17 you don't need to know what's going on. Alright. Okay. So this now looks like this is doing our container image build based on our Docker file perhaps. Yep. Now it's there. So we just have to wait now for a Yarn install. Those are notoriously quick. Right? I don't know if the sarcasm came across there, but it was Yes. It did. We knew exactly what you're talking about. Yeah. Another type of sat watching NPM installs or Yarn install. Not it's actually not building that from the the the Dockerfile. I I don't think that the what it's doing here is it's actually

28:53 building a container on the fly that will then perform the YARN build. So it's basically yeah. It's building a lot of the components that you saw, like the s three component and the YARN component. They do they do things in containers. And so the developer of those packages, at some point, specified what that container should be and how to build it. And so the first time you run, an an an Netlify operation, for example, or, no. Sorry. What am I saying? In this case, it's s three an s three operation or or Yarn operation, you're

29:26 you're building the container as the author of the package specified. And then it's all cash, so it will be fast. Nice. Alright. Well, since we've got a moment because of this YARN install, we'll just pop back over here because that's where I left my little thing here. That's the wrong one. Oh, funny that his comment came out at the same time, so I'll show both at the same time. There's an episode of the podcast being released this Thursday of which I believe, Solomon, you had the honor of chatting with with Pop. Right? Yes. So if you wanna watch that or see

29:59 an announcement when it launches, you can follow podcast pop on Twitter. And I've heard it's it's pretty good. It's just surprising for Pop because his material is crap. Sorry, Dan. I just I just you bend a pineapple pizza and I'm still raging about it, mate. Like, I can't believe you threw a whole pineapple pizza in the bin, but I'll I'll hold it back. You should definitely watch that episode if you wanna see what happens when you record a when you record a a podcast. On an iPad? Outdoors without without checking on an iPad. I was on an iPad, but

30:32 it did not crash. But, I I did do it outdoors and did not check the weather forecast first. So Was it stormy, rainy, snowy? Which random weather did you get for the day? Thunderstorm. First thunder and then and thunder, and I thought, oh, that was noisy. Sorry. And then I didn't even think to go for cover. So later it went. And now you know. Well, Yaron is now telling me that I have a problem with my network so that's wonderful as well. Oh, I think we actually have that issue. It's the Yaron registry thing. Right?

31:10 It might be a Yaron registry issue. If we ignore it, will it go away? Maybe you try rerunning. You say you don't wanna give it thirty seconds? You want you want me to control c? I don't know. Either way, recall. Alright. Well, The sad thing is if we if you control c now and you're on you where you're not gonna get cached, the the the you know, built it only caches a successful operation. So if you if you control c now, you're gonna have to rerun everything. Of course. So it's a dilemma. I know. It's

31:52 catch 22. Because we could just run it again and get presented with the same trouble with my network connection. I'm sorry. Solomon likes to talk. We'll just let you go. What if you go for us? What should I talk about? Let's see. Well, one thing we could do is maybe I don't know if your setup supports it. I mean, I know on an iPad, I can do it, but, maybe while it's running, maybe you can look at some code at the same time, and I can comment the the plan, actually, the source code of the source code describing

32:13 Deep Dive into Dagger Plan Code (Q & Artifacts)

32:29 what's actually happening on the screen. Right? Yeah. Alright. Let's take a look at the queue then. Right. So if you look at main dot queue, I'll show you the line that's currently having trouble. We can we can, do a little virtual troubleshooting. So I think that's the one. So it's a Yarn it's Yarn dot package. So we're basically here declaratively saying, there's a Yarn package here in, one of the nodes in our graph is a Yarn package. And so let's wire up that YARN package with the correct inputs and outputs. And so it's basically gonna plug in source code as

33:05 an input, and it's going to run a YARN build in a container and spit out the build directory as an output, and then you can wire that. So, for example, a little lower, you see s three bucket? So that's another node in the graph. It's an s three object that's gonna be synchronized, up to the s three API. So you give it an input, which is an artifact, and and that's gonna sync that up to the the s three bucket as specified. And you can see the source key for that bucket is app dot build, which is

33:35 of reference to the build key in in the Yarn package, if that makes sense. So it's basically that line right here. That's a link in the in the DAG. This is what causes this to have a dependency of the app Yarn dot package. Right. It's basically saying take the take the the build the Yarn build and stick it into the s three object as its contents. And so then the Dagger engine loads that and sees, okay. That's a dependency. From there, it will run things in the correct order. It will parallelize if if needed, etcetera.

34:14 Or or rather, it will arrange for Billkit to do that because Billkit does most of the heavy lifting. Okay. So because I have a little bit of a familiarity with q, why don't we kinda dig into this this specific couple of lines and let's dig into what's actually happening with these lines. So this is in queue, this is a module called yarn with a definition called package. So that means that Dagger has explored some sort of schema definition to say this is what a package looks like. What we are defining here is that the source as the source which

34:24 Composable Packages & Future Ecosystem

34:47 comes to another line, we'll skip that for now. But there's nothing here, this is entirely declarative. Right? This is just me saying that a YARN package So that means that the Dagger engine understands what a YARN package is and how to work with that. So I'm assuming Dagger has a few of these integrations and maybe we could talk about the ones that are available. Sure. Yeah. Actually, you could look at the source code of YARN. That's a good example because we developed it, but we developed it in queue using the regular, Dagger APIs. And so anyone can develop it.

35:22 And, actually, just earlier today on the Discord, someone said they have a pull request they wanna open for a Maven integration, I think, and they wrote it themselves. And so, looking at the YARN package would be actually very useful because that's a big differentiator of Dagger. Actually, that's what I should have answered earlier in the question, why is it different from CD? Well, it's a good example of the programmable thing. Like, in Go or in Python or in JavaScript or whatever, any real programmable environment, any real platform, you write code. You know, you have your program, like your

36:01 main function or whatever. And then at some point, you think, oh, that part could be, moved into a reusable package, and I can import the package, but then maybe someone else can or me later. And you do that. And your main configure your main, sorry, your main function is written in Go, for example, and your package is written in Go. It's all Go. You know, the library is Go, and the main thing is Go. And then you kinda it can be Go all the way down because the library can import another library, etcetera. That's basically how it's supposed to work. But

36:32 if you're writing, like, a a CD YAML thing, that's not how it actually works. You know? You have a YAML your main thing is YAML, and then the next level is maybe Docker containers or shell scripts, then there's no third layer down. You know? So there's no you can't take your YAML and spit it out into a sub YAML or a sub sub YAML and expect it to work the way you would want. And same with Terraform. You know? You have Terraform HCL configurations, which compose, Terraform providers. But Terraform providers are not written in HCL,

37:08 so you can't spin off part of your HCL and say, oh, I'll make that a provider now. Oh, and how about a sub sub provider? There's modules, which are a separate thing, but they're kind of a superficial it's always two different layers of programming. Right? And writing a provider for ter provider for Terraform is more like developing a driver. You know? Sure. Can write any driver for any peripheral, but it's not you know, you're not just gonna do it on the fly as you develop your application. You know? No one has time for that. So

37:39 what Dagger does, it's it's just basically queue on one side and queue on the other. And you spin out your queue thing into another queue thing and then they import each other and you can keep going down and it just works the way you you would expect. Awesome. Alright. Well, I've pulled up the the yarn package that you mentioned. There is. I'm kind of curious like, you know, as Dagger starts to evolve and you start to approach GA and such like that, do you see or are you going to be encouraging pull requests to the standard

38:08 lib to support every tool in the world? Do you see that being something that there's third party slash external packages or libraries? Yeah. How do you envisage that working? So right now, yeah, we do encourage pull requests because it's the only practical way right now to share, you know, a package like this one with everyone using Dagger. But that's temporary because the queue developers are some of which I think are listening right now, are actively developing queue modules. So right now, there's kind of the basic infrastructure for importing. You can actually import you can define queue packages. You can import

38:49 them. I mean, we're doing it right now. But, what's missing is the, the discovery and distribution system. Like, with Go, you can't discover a queue module anywhere and then download it and then manage versions and upgrades, etcetera. So that's what's being developed now, heavily inspired by Go, which is not surprising because a lot of the queue developers are also Go developers. So, basically, soon, you'll be able to be able to queue get or whatever. And then you could you know, then as a result, anyone could publish their package anywhere they want, like, their own GitHub repo, and you can

39:23 just import it right here. So then there will be two ways to share and reuse packages, and probably our built in standard library will play more of a curation role. You know? Like, you can go out there and find any package for anything. But if you wanna play it safe and you wanna know there's a set like, a distribution of things that are tested together and are known to work together, then kind of like, I guess, a Linux distribution. Right? But for queue configurations, then you just you you play it safe and you import our standard library or you

40:02 can mix and match. You know? Awesome. Brian Carrollton shared something with me the other day that I thought was really cool. We're we both do a lot of stuff in the queue stuff as we both do a lot of stuff with queue as well, but someone compares a queue to web assembly and was running it in their browser, which I thought was a a really cool use case for doing that. Yeah. Okay. Compared Q to what's sorry. I missed Someone compiled Q using tiny go as a web consent a a web assembly and was using the

40:32 the Q API inside the browser, which I thought was a really cool application of it as well. Yeah. It was really nice. Yeah. That is cool. Yeah. There is all there is also there is a an interactive Q playground that actually runs in WebAssembly. I think it's on the official Q website. It works really well. Oh, nice. I've actually used that, and I never thought about how it worked before. But now that you mentioned, of course, I was using WebAssembly. So okay. So are we feeling confident? Do what do we wanna look at my terminal?

41:01 I mean, I can't speak for the YARN registry, but sure, I believe in them. It worked. Yay. There we go. We talk enough and then we come back and magic things happen. So let's try and work out what actually happened here. So we That's my life philosophy. Yeah. It's like that x k c d comic was the it's compiling and the chair sword fighting. Like, it's still relevant today as ever was even though we don't compile as much. It's like that, but if you stop sword fighting, it stops compiling. Exactly. Alright. So we got our Alpine

41:36 image ability stuff. We got our yarn, which eventually did finish. We can see here that it does a build and the build output finishes here. App.build says we're complete. That's our dependency then that we noticed and say that the queue failed. So this now is an artifact that's available. We then perform the next step in the DAG, which is the s three bucket, which completes. And then is this just I don't know why it's doing things there, but it's just just uploading the output to Amazon s three. I think that's maybe what happened there. Yep.

42:04 Reviewing Plan Outputs (S3 Site)

42:14 Alright. Now there was a command and we can already kinda see the output from it here, but there was a in the documentation, a Dagger output list. Was that right? Mhmm. Yep. Or or that was input list, I assume an output will work too. And that's just what this is. Is this a can to, like, what we expect from Pulumi output, Terraform output? It's just a variable that we wanna expose or present to whoever's consuming Dagger? Yep. Pretty much. The the the nice thing is right now that the engine can do more than the command line tool can in terms of presentation.

42:51 So for example and you could have an output artifact, and, you could have it copied or mounted into your local directory as a result. So you could have, for example, if if you want if you wanted and if the CLI supported it, you could you could have, you know, the the the result of the YARN build, copied to somewhere in your local directory as a intermediary artifact, you know, to inspect or whatever. Okay. Let's browse to it. Let's see if it worked. We got busy and human. If actually Solid name. The link below that. If you go back on the output list.

43:37 Yeah. That was the bucket URL. Good catch. Oh. Yeah. We should probably we need to Oh, and then I I I went from that way anyway. There we go. We have our React to do application. Awesome. Yay. Okay. So these are the outputs. I'm as those are I'm assuming those are just defined in the queue file and maybe we can take a look at exactly what actually happened there. We have a whole bunch of inputs now. I don't remember seeing any inputs, but I'm assuming these are just all of the things that I could provide in

43:58 Working with Inputs and Secrets

44:11 my queue plan Yeah. The Dagger plan to be able to override or change the way that this plan works roughly? Yep. Yes. Yeah. So packages can define inputs. So for example, when you the s three bucket part, all, you know, all the inputs within s three bucket, that was actually defined in the s three package, like access key, secret key, etcetera. Okay. And the way secrets work in Dagger is just they're just encrypted, and you have to distribute the key to people and Right. So that's that's what the each thing was doing. Right. There's Dagger input

44:53 Dagger input secret to set a to inject a new secret value, Dagger input dear to inject a local directory, etcetera, etcetera. And Dagger input secret will take care of the encryption part for you. Awesome. And, yeah, then on on the other end, you you need to distribute keys via a UI that is not yet implemented. Sweet. Alright. We got a question from Noel. I don't understand, but maybe you will. So input values support both the value or the pointer. I'm not sure if that's cool. So the asterisk the the asterisk, that's a queue syntax thing. It means default value.

45:39 Oh, that was a queue question. Got it. Okay. Well, I don't know, actually. I I I'm just inferring from the from the comment. I I there there's no pointers in queue. That's for sure. Yeah. There we go. Okay. Yeah. Oh, I see. Yeah. I don't know what what is that? Yeah. That's that's mostly just, an unpolished UI thing, but, that's queue saying in very rough ways that we should hide. This comes from the engine saying, by default, it would be an empty list, and it actually is an empty list. So, basically empty lists. Yeah. I do like this constraint model that

46:16 Q presents. So like, yeah, think here's a really nice example where we're seeing that this defaults to both, but really we accept any string value. Like, yeah, Q is is a really cool tool. I I think it's really awesome that Dagger uses that under the hood. There's just the the potential is just huge. And, honestly, we it's just the frustration is, you know, there's a whole year worth of work just to kind of build up to the potential of of queue in terms of presentation. Like, for example, we have an earlier prototype, which is not open source, but we're gonna

46:44 transpose it into Dagger, which generates from your queue configuration a complete interactive web interface. So you get if you ever used Heroku, for example, there's this add on system. You know, you could say, a database, and then you have this nice web panel to configure your database. And each add on has its own custom made panel to configure things specific to that. Right? So let's say you wanna add, for example, an s three bucket, then it would ask for your s Amazon credentials and the name of the bucket, whatever. Here, that's all specified in the queue configuration.

47:23 So we have this rough command line interface to that, Dagger input list, Dagger output list, and, you know, sometimes it says weird things like asterisk, you know, brackets, etcetera. We actually have this prototype where we generate. We do the same thing as those add on, web interfaces, but generated. So you would from this, you would get a little web interface, that shows you everything I just said in in a in a nice web form specific to exactly that. So it's it's really magical. Personally, I'm really impatient to bring it back because you write 10 lines of queue, and

47:59 you just develop the web interface for your deployment thing. It's it's really pretty cool. Yeah. I'm really frustrated that you said all that because I've tried to build some for as a a command line application for some of my own queue stuff and failed miserably. So, you know, if you ever open source that up, I'm gonna steal all of it completely. I the plan is to make it available in the open in in Dagger. So, yes, open source. Yeah. That would be awesome. Yeah. My use case isn't important. I'll share that with you another day. But, yeah, really cool

48:30 that you build that kind of tooling. One it's one of the things that QAPI does try to make easier. I'll just blame my own limited skill set for failing miserably. But we've only looked at the to do application there. So let's let's jump through a couple more of these Okay. Learn Dagger things. Anyone watching does have any questions, please feel free to drop them in the comments section and we will do our best to answer them. I mean, I will I will proxy the question for you, but we will get some really awesome answers here.

48:38 Introducing Dagger Environments

48:58 Okay. So we did some basic usage to do application. We did a Yarn build. We able to do the test three. We were able to browse through it. Already very, very cool system. So Dagger one zero two is where we dive into what an environment is. So can we think of this as just Pulumi stacks, TerraForm, whatever they call them. I can't remember these days. Like, an environment being staging production. It's it's Yep. Yep. Okay. That's where it is. Yep. Alright. So I should probably be reading this with a bit more vigilance, but I'm gonna keep going

49:37 and going and going. So this wants me to build, like, a new directory in order to do application, add a new source dot Dagger, or is this building out the same thing that we kind of just already ran but in a different way? It's so what we ran is one environment called s three that was preloaded, and we're gonna create a second environment called multi bucket that deploys to s three and Netlify. So guess yep. Yep. Okay. So it's guiding you through writing those there's like three little queue files to write. Okay. So the first thing I wanted me to

49:44 Setting up the Multi-Bucket Environment

50:23 do was create multi bucket. Did I have the CD into that before I run my QMod? I don't think I yeah. No. The you should run everything in in the to do app directory. Standby. Oh, yeah. Like I said, I installed the Mac OS 12 very early developer beta. Very nice. So we we actually have this we we had this conversation recently with the Qdevs, but, probably, it would be useful for us to like, it's actually not strictly necessary to install Q to use Dagger. It's just that at some point, if you wanna get serious about

51:09 Dagger development, you you probably wanna get serious about the queue tool and all the all all it can offer. Like, if wanna speed things up for this tutorial, you can you can start without installing it and install it later, but you're already on your way. So Yeah. I'll just pull this then. No. I don't save the pocket. I have never wanted that ever. I think you can do brew install of q length slash tab slash queue, and that should work. No pressure. That worked. Awesome. Alright. So we What are the comments, by the way? You should have a speech bubble.

52:03 Well, maybe not an iPad. I'm not sure. I'm scared I'm scared to click. Oh, I see it. But I should I click it? I don't know. Yeah. You can always reload the page. What's the worst that can happen? Oh, there it is. Alright. Come on, queue. We wanna build a multi bucket thing here. Alright. So we're running QMod in it, which just sets up some really simple directory structures for the QModule system. We're gonna pop back over here. We're gonna go beach balling. There we go. We created that. So now we're creating our first queue file.

52:42 So this is just source dot queue. And I'll just do this. Did you see the comment about the last the last line of your terminal being hidden? It's probably thirty minutes ago, but I did. I dragged it up. We're we're all good. Yeah. I do that every week because I have a a window manager. Like, I I come from Linux and I three is my main working machine, but for streaming, I really do need to use a Mac and I I'd like things just to be full screen all the time. And I have a window

53:09 manager that puts a full screen but adds a border but the border is not big enough. It's a whole thing. Oh, that's what you were saying earlier. Yeah. Okay. So we got our new multi bucket package, we're importing the Dagger engine. We've got something called source, which we're seeing as an artifact in Dagger and a Dagger input. Is an artifact something you wanna take a minute to explain to people what that is, or should we just accept that as it is? Alright. Okay. Go for it's it can't can't hurt to explain it. I mean, that's one area we need to

53:42 explain better in the docs. Yeah. Basically, one reason regular CICD systems can't just haven't solved this problem is many of them don't have composition. But when they do, you compose YAML values or JSON values. And so you can only compose things that fit in JSON. That includes the URL of a source directory to download maybe, but it doesn't include the actual directory. And so that means any composition of anything that involves artifacts needs to have first class support of artifacts right there in the code being composed. So you it's not enough to say, oh, that input is a URL

54:29 of a thing to download. Because if you say that, then you're you're hard coding the concept that, you know, you can only you need a third party now to which the thing will be uploaded and then downloaded. So you're not really covering the full picture. So here, what we're we're doing is we're the the engine actually supports any value in your tree actually being an artifact. We're representing an artifact. But so the but the way we do that is it's not actually the data of the artifact. It's the specification of how to produce the artifact on the

55:02 fly. And so, for example, if the actual artifact is, in this case, we're basically only specifying here. SRC will be something that can produce an artifact on the fly, and we're simplifying that to an artifact. So it's really a just in time artifact. And so what you do then is, as a user, you're gonna specify what that is. And so you could say, oh, I want SRC to be the contents of my local directory over here. Or you can say, actually, I want SRC to be that remote that branch of that remote, Git repository. Or you could say, I want it to

55:40 be that tag at that remote registry. And so then we fill it in, and we generate that, dynamically using BuildKit. So the the as long as BuildKit support, it can be expressed as a BuildKit pipeline. You can make an an artifact. And because BuildKit can do a lot of things, you can cover almost any possible use case by composing these these artifacts. So it's it's honestly, I have to find a better way to explain it. It's just insanely powerful. Yeah. It's what makes Dagger just very different. Yeah. It sounds just so flexible. Like, you can almost just do whatever you

56:17 need and the tools kind of stay in that way, but providing the constructs to do that. And I think that's really, really awesome. Okay. It's composable. And composable. Yeah. I mean I mean, yeah. I'm curious what came first. Right? Was it you you obviously had an idea to build the product and then you made the decision to adopt Q. Now did all of these amazing features come out with the fact that you chose Q or was the reason that you chose Q because you'd already decided that composability and flexibility were something that you needed, like

56:44 I think Andrea can answer that one better than anyone. Like, I mean How Well, I'll the the short version is Andrea basically built built a version pre q almost by his by himself, and it's a lot of work. I mean, is that a good summary? Yeah. We ended like, we were reinventing q. We we had, like, our own made stuff, and then suddenly we did And built in. And we ended up yeah. And so we're just reinventing our thing. And and after a while, we saw we saw queue, and we're like, okay. Like, this is

57:21 doing exactly well, not exactly. It's doing way more than what we're, like, hacking together. So that's when we decide to just switch over to to using queue. Awesome. Alright. I'm gonna copy and paste the last couple of bits of queue here, and then we're gonna go through exactly what is happening. But we do have a few comments that have come in in the last couple of minutes. So we've got one from Amit who's saying, thanks for the stream. Exactly what was needed today. Awesome. Glad we could help. Crazy Max is saying, hey. Hi, Amit. And then now we have a question from

57:52 Moody. So would you say the Dagger is like GitHub actions, but not tied to GitHub or any specific provider? I think it's it's it's a useful comparison. I I guess it depends what you see as the primary role of GitHub actions. It's it's it's hard to compare directly tools in this space because it's just kind of a mess of partially overlapping things. If you think of GitHub actions as the place where you're gonna define the glue between everything in your delivery, then yes. If you think of GitHub actions as a really good CI that takes your GitHub

58:33 world and makes it just really easy to automate, then, then they're very complementary. So, yeah, I think if you took GitHub actions and became very, very maximalist and say, I wanna actually automate everything to end, glue everything together with GitHub actions. Eventually, you would run into the problem that, a, it's not programmable, and, b, it's just tied to GitHub. And even if there's a standalone runner, it's always gonna be tied to GitHub. And so how do you integrate that into GitLab, Atlassian, you know, non GitHub everywhere world? It becomes kind of there's a lot of

59:09 friction. So yeah. You can use like, I would say, a good a huge portion of Dagger runs today are run inside the GitHub action, so they they work very well together. Nice. Okay. I believe I've copied the appropriate steps for our multi bucket demo here. So we got our source dot queue, which is just defining that we have some property called source, which is an artifact. We then have a yarn dot queue, which is a has something defined as an app, which is a yarn package with the dependency on our first source. And then we have our Netlify deployment,

59:51 which defines something called a site, which has a Netlify site and the contents of that are one of the output fields from our application, which is the build. So we can kinda see command called Dagger doc, which maybe you might wanna use at some point if you're curious. I'm always curious. Alright. Dagger, doc, multi bucket. I'm just gonna make us up there. No. Oh, well, I guess that could that actually could work, but currently, it doesn't. Yeah. Currently, it's wired to, the the exact thing you import. It's gotta be what you import. Ah, right. Okay. .Dagger.io.

1:00:33 Yeah. Got it. Okay. There's also a web Nice. In that in the docs. So it shows me the inputs that I can use and the different definitions. So Netlify has something called an account, which makes sense. We then got multiple sites we could deploy to that account and we have the ability to pass in our account token via secrets and stuff like that. Yeah. That is a pretty handy command. Nice. I like that. Okay. So the documentation also requested that I now that I've copied something I don't wanna copy, but we wanna create the new environment.

1:01:18 And Okay. So what does that actually mean to Dagger when I create an environment? Is that just a directory in my dot Dagger? Yeah. Yes. Yeah. It's basically a bug. It's it's like a it's a it's a box to put code in content a code in data, code in state. Okay. So now to add my code to this environment, I just copy everything here. And is that the, you know, the standard way that people should think about this? Like or would they just run the Dagger new environment first and then actually just craft the queue inside of there? Do you recommend keeping

1:02:01 it outside or That's that's work in progress right now. We're actually we're gonna get rid of the copy right now. So we're trying the the the principle is we want developing for Dagger to be as identical to regular queue development. So if you already know queue and you have a workflow setup, it should be instantly recognizable and compatible with Dagger. Right now, the the problem is we have we we have you put your queue files in this dot Dagger slash n slash something slash plan, and it actually works fine at runtime. But the problem is once

1:02:40 q modules will come out and you have this it's like go. That code, you might wanna make an importable package by someone else. And then what what name do you give to the module? You know? So, basically, the natural way for to develop something in queue is to have it somewhere normal looking in the in the GitHub repository in the Git repository. And so, we're gonna get rid of that copy step. So the the way you develop those queue files instead of having you copy them into the the environment, now you can actually configure the environment to

1:03:16 load them from where you develop them. That's actually it already works as of this week. We just haven't changed the tutorial yet. So it's both work. Yeah. It's early days. We're figuring out the the the perfect developer experience one step at a time. You know? Well, yeah. And you're adding 50 developers a day and getting feedback. It's you know? Right. These things take time. So we we don't have any more documentation here, so we're gonna go slightly off script. But I'm assuming, you know, I'm trying to think about this now. Okay? I'm I'm using us too. I

1:03:42 Configuring Environment Inputs (Netlify Example)

1:03:51 wanna start to plan The Dagger up for me, I wouldn't expect to work right now. Now it seems like the s three package provider does ship with those credentials. We do have the key imported, but I would expect the Netlify one to potentially fail. Is that correct? Yeah. Actually, the s three package itself does not ship with credentials. It's that environment in the example repository that we preconfigured. Yeah. So, yeah, it will fail for the reason you mentioned. Okay. But if you imported s three, it would also fail. Okay. So when we have multiple environments, we have

1:04:25 to specify the environment that we wanna run. Yeah. And that's gonna send a plan off to build kit. It's gonna try and do a few things, and I'm assuming that's gonna fail right after probably the yarn build, I would expect. Ideally, it would fail right away, but it's like oh, yeah. Perfect. Okay. Okay. So Yeah. So exactly what you said. Okay. So it actually wants me to provide what source as as well. So how do we provide inputs for Dagger? Is that through the the CLI and not queue? Dagger input. Yep. Correct. Alright. So let's say we are actually

1:05:07 the an input is actually it's a little configuration telling Dagger how to dynamically inject little bits of q. So it's gonna be q in the end, but Dagger will generate that queue for you to make things easier. Alright. I'm just making up now, but will that work? Yep. No. That's perfect. Oh, the environment. Okay. Is it dash e for short? Yes. Yep. Okay. So I'm curious about that Perfect. What that's actually changed here then. I must because you said that input is generating queue. That's gonna have to live somewhere. Yep. My computer would keep up today and

1:05:49 maybe be able to see what that So it changed your values dot YAML file. Right. Okay. So we can Which you can you can actually you can actually try Dagger edit. That one, might enjoy. Definitely do not manually edit it here because it's it's it's managed with SOPs. You know? So it's all there's because of the encrypted values, if you manually change a part of the file without going through Dagger edit, the the the hash will not match. And so it'll say, oh, this file has been tampered with. Alright. This computer is going to the bin

1:06:39 after today. Alright. Multi bucket. Okay. So I can modify this safely with Dagger edit? Yes. Okay. Yep. So we've already set that. And you can see yep. So that's basically it's like a file representation of of these command Dagger input commands you typed. Okay. So I have two more inputs that I need to provide in order for me to be able to deploy this sample to do application to Netlify. One is account Token and Netlify. Name. The input I'm wondering if I can just make this up some more. I have to say, I'm not sure I

1:07:21 would remember it, but, technically, you can. I think the the you have to use the path that was given as if, so you cannot do nesting. So it should be, like, netrify.account. Name and so on. Alright. If I were you, I would do one of those by the with the command line, like, Dagger input something and then maybe replicate it. That's what I do because I never remember it. Alright. So we want input, notify, dot account. It's input. First thing you give the type input, That would be text. It's like a text value. Oh, yeah. Because we used dir previously. Okay.

1:08:06 Mhmm. And I don't know what this is yet, so I'm just make it up. Oh, yeah. And then I'm going in here. Alright. Okay. Yeah. So the inputs. Oh, yeah. Okay. So this is the type followed by the value. Alright. Okay. Got it. Which means we could also do You must be just watching me going, I really wish you would stop doing this stuff. No. No. Actually, it's very interesting. It's good data. Keep going. Alright. So I'm assuming, does it care if mine I can't talk it as a secret or or text? Or does it just expect it to exist?

1:08:44 It does. Okay. Yeah. Because we we if it's a secret, we're going to secret we're gonna handle it very differently. But as if I provide the input as not a secret and just I'm happy just to to expose that, will it still work? No. No. Is it's gonna try and decrypt it and, you know, it won't be a valid ciphertext. Okay. So the Netlify provider expects that. And in fact, if I use the doc command, which you already showed me, I'm assuming it probably has, yeah, a dead sitter. Dagger dot secret. Okay. So that means that we probably wanna do

1:09:27 a can It was fine using edit. Like, you can just type it in plain text and then Oh, you're right. It will just encrypt it. So Oh, I forgot about that. That's so cool. Okay. Well, I mean, I I don't have a Netlify. Should I get one? Do we actually wanna spin this up? Or is there something more something Do you have an account? Yeah. Yeah. You should spin it up. Yeah. Well, let me give you I can give you one. Oh. He switched window. No. It's it's alright. Let's let's jump to that if I I've I've got a login.

1:10:08 Yeah. Let's see if we can deploy our application. Welcome back. I'm fine. I'm back. Alright. We were enjoying you on mute. So log in. Sam and Andrea like to make fun of my iPad setup, so I'm gonna hear about this for days. You know what? It's been working a lot better than my Mac setup. So so how do I I think you should click on your on your user icon on the top right. And, yeah, you should see user settings. Yeah. And applications, perhaps, and personal access tokens. And if you generate one, go. There we go.

1:11:05 You've all got ten minutes if you wanted to play stuff to my. There we go. Go. I did that with an Equinix Metals token one day where I created a Kubernetes cluster, and I said I'm gonna be so disappointed if there's not a crypto miner running on us in thirty minutes, and there wasn't. Alright. So we've provided the token. I'm gonna run a edit. Okay. And now we can see the secret token. The account name, I mean, that is Rawkode. Right? That's just my actual account name. It's not like a site name or anything like

1:11:41 that. Okay. Yeah. I think actually the account name, that's actually optional because account name, that's when you have a team, I think. Oh, no. No. Yeah. Site Name, you need. That's the site that's the site name. Site.account.name, that's that that's what they call a team. And I think by default, you just wanna leave that empty. Okay. So that's right there. Your your personal team. That that's I guess we should change that to team because it's it makes sense to Oh. I think a coding style for the standard library should be use whatever terms the native service uses. Yeah.

1:12:27 Yeah. I think all the inputs should have been prefixed with side notes. So if you're on edit, you should be able to like, just Dagger edit, and you should be able to change that in batch. I don't know why I did there. Yeah. Let's let's do that. Okay. So we expect is that just me failing to be able to paste now? I really am on fire today. There we go. Okay. Let's take that one out. Oh, I see. Yeah. But then net there's no is that Netlify? Is it site dot Netlify? That's what the output side. I'm not yeah.

1:13:10 I wanted site Netlify account token to site, let define name Oh, okay. Which I believe I've now set appropriately. And we're gonna run the Dagger up, and we're gonna see if we get us to do application deployed to Netlify. Nice. One note. If let's say you're setting that up for a team, this this setup you could do once. And because the values are encrypted, you can commit it and share in the repo. And then downstream from you, app developers who just wanna deploy to that to Netlify to, like, let's say, shared Netlify account, those inputs will be preconfigured for them.

1:13:16 Executing the Multi-Bucket Plan (Attempt)

1:13:48 I mean, is it safe for those like, say, I wanted to just Multi tenancy. Deploy my own blog with Dagger and the repository's open source. It's public. I mean, is that is it safe because of that encryption for me to do that as long as I don't give anyone access to my private key? Yes. Yes. Just don't give access to the the key. Is there a plan for I mean, I mean, you know, you should apply you should apply common common sense additional layers. Like, if it's a critical credential, you should prepare for the possibility that the

1:14:21 key is leaked anyway as some you know what I mean? So Yeah. Just apply security common sense. But, yes, it it it is I mean, it the the project is very robust, so it is safe to commit credentials encrypted with SOPs and age on the Git repo. Yeah. In fact, this is actually my exact next question, and I think this answers already for me. But the secret back end can be swapped out for cloud KMS as well. Right? Yes. Alright. That's pretty sweet. Now we just need a one password option, and then I can do everything on my

1:15:01 local machine. That would be pretty cool too. Oh, that's cool. Yeah. Yeah. Actually, it would be nice to to have, you know, pluggable back ends so that you can use your favorite KMS and, you know, just pick your back end. We did that in a in a past version before open sourcing, so we'll have to bring it back. Yeah. Well, I can't use Google Cloud's key KMS key, whatever they call it, because it doesn't allow you to delete the key rings when you're done with it, and I just can't handle it. So I always end up

1:15:28 using AWS just for their KMS but then deploying other stuff to Google. Just weird little annoyances. But yeah, one password on here would be pretty sweet. Although age is a pretty good way to do it as well. I could store that key somewhere. Alright. Let's see. We're doing a a yarn step, so this may take a either a few minutes or a few hours. We'll just need to wait and see. Again. So I'm curious about the different environments. I'm assuming because the input has changed that does it know that the inputs don't affect the YARN build? Could that have been cached or

1:16:03 are environments all the caching happened per environment? Should have been Caching shared. Assuming that yep. Although, I'm assuming here, there might have been some modifications or something going on. But, yeah, in theory, this should have been like a no op. We should have just, you know, not built anything. And that's because it doesn't require anything environment specific that the build step itself should have been shared across different environments. Yeah. Actually, it's shared across the entire, like, build kit instance. It could be, like, your cache, but it could also be, like, someone else's. Like, if you share a a build

1:16:46 kit server, could be your entire team or coworkers. Yeah. Oh, nice. Do a lot of cool stuff on the infrastructure side. Yeah. I'll just assume my Versus code. Netflix. Sorry. I'll figure Netflix, they use Bill Kit, a lot. They they do a lot of for building. And and, you know, they have build kit clusters backed by container d clusters, and and so you get some some pretty amazing you know, it scales up really nicely because both build kit and container d are really well engineered, and they got a lot of usage. And so you can just you can do

1:17:26 some some pretty impressive stuff, and and so we just piggyback on all of that. So Nice. Alright. Audience, last chance for questions. We'll be finishing up just after we hopefully see our Netlify Netlify site here deployed, which I'm pretty confident about. I'm sure it should be good. I'll assume the cache here wasn't used probably just because of my Versus codes modifying or doing stuff in the background. Yeah. I don't know why. I think Is it is it the queues queue files? Yeah. Yeah. I think because the input is dot, and since we put, like, the queue files,

1:18:08 we we want to change the cache. Yeah. There we go. Yeah. That yeah. In theory yep. Yeah. I think if we remove them and Dagger up again, it should be cached, but I'm not gonna I think we're too close now. Let's just let it go. Yeah. Yeah. Yeah. We we should auto auto exclude them. Right? Because Bill Kit you can tell Bill Kit to exclude certain patterns from from a local directory. So I guess we could we could we could do that. Queue dot mods. Maybe if if dot Dagger We can even be more specific. We could say ignore only

1:18:41 the queue files, which are part of the plan, which you're currently executing. Ah. Cool. Alright. Let's see. It seems to have already fetched the dependencies. Let's do the linking step, which I've never understood with node. Hopefully, we got a nice pretty quick build. So this is the build of the actual to do app? It is. It looks like it to me. It's crazy. My computer is I don't know what's going on with it today, but last week or two, and it does slow down a lot. I'm not sure what I've got running. I feel like we're

1:19:27 we're at a point where web builds are just so ridiculously long that there's a whole feel and innovation in making really cool progress bars for web builds. Like, really slick. That's not healthy. Really, what I should just do is get an iPad and use GitHub code spaces and not have anything running locally except my streaming software. I think I'm now sold on that idea. That's what I do. Yeah. Exactly. Let the cloud deal with this. Why am why am I running Docker on my local machine? Well, does GitHub space they have an an editor. Right? Yeah. Yeah. You gotta fill a

1:20:03 Versus code, web based Versus code. So it's really good, actually. And it syncs with all your settings locally. So, like, whenever I open code spaces, I get the exact same Versus code with the same extensions with the same color theme. Like, it's really impressive. Wow. I use VI and SSH, so I I that's also something Andrew and Sam like to make fun of. But I I I used to be a new low maintenance. I like NeoVim, but the the Versus code with the language server protocol stuff just, like, blew me away when I first tried it. And I know

1:20:35 that NeoVim now has quite good language server protocol support, but I kinda got I got used to Versus code. I actually quite like it. I I hear ads for Versus Code every morning when I log in. Alright. This may or may not finish. So we got a comment from Crazy Max saying, yeah, they do. Netflix do contribute to build kit. There you go. Yeah. We can and we bother them a lot. Oh, it's there. We did we did hit we did hit a lot of, well, not a lot. We did a few build kit bugs.

1:21:26 When when you're hitting bill kit bugs, you know you're you're really a serious bill kit user user because bill kit doesn't have a lot of bugs generally. That and you're using the queue, which I I'm sure Marcel would say queue is still prey early in the adoption cycle, so you're kinda making maybe Queue has more bugs, but they fix them really fast. No. But they, you know, they they they fix them really fast, so it's it's a it's a good combination. Alright. Is is this has it worked? Is this the Netlify? It looks like oh,

1:22:02 that's actually so that's the the initial build of the Netlify container. So the the container that will run run the Netlify upload. Alright. Okay. Okay. Because when you're when you develop a package, like, you you decide actually, we may actually have to set a convention for that, you know, like, official repositories in in Linux distros. Everything has to be built from scratch in a reproducible way. Like, we we will probably end up imposing a rule like that for official upstream packages. But in theory, you could have a package that just you know, there's a docker pull on a

1:22:39 binary image and then just runs it. You don't have to build on the fly, but probably you you should if you're serious about your your, being able to trace your supply chain because then you can see what's going on end to end. Right? Yeah. The sec store project, I think, is really exciting for that whole supply chain thing right now. They've got the new software bill of manifest or materials support that they've been working on, which I think is really, really, really, really cool. Oh, what project is that? A six store. Oh, yeah. Yeah. I I I see what

1:23:15 you're talking about. Yeah. It's a You know, there's a whole little group of people on the on the, Dagger Discord that are really ex you know, really focused on on that, the software bill of materials and, just the new requirements coming up for it in The US, you know, for federal contractors. So it seemed like a lot of vendors are getting excited about that. I'm really starting to dislike this error message from you, Aaron. And now thinking that someone should invent BuildKit as a service so that all this stuff just runs on someone else's computer for me.

1:23:49 I guess that's what you were saying about the team. Sorry. Here you go. No. I agree. I agree. Yeah. I mean, I do have access to bare metal. I work for Echo XPEL. I should just spin up a really beefy box running build kit and expose myself a little Yeah. Docker host thing. You should. I mean, the the first cloud provider that does that, we will we will point Dagger users to it because we're not that's not how we're gonna make money. You know, I think Google Cloud had a maybe they still do. There was an attempt

1:24:18 at doing that for Bazel. But the problem, I think, for Bazel is it's really, really hard and high friction to go for to to to do remote execution. It doesn't you can't switch it on on demand. But with Bill Kit, you can. Alright. Well, we'll give this a few more minutes to finish. Why don't we have a quick chat just before we wrap up then and maybe you can kinda let me just pop this back over here. Maybe you can share with us what's what's coming next for for Dagger. Like, where are you and the

1:24:40 Roadmap and Future Plans

1:24:49 like, are you pushing towards beta? When should people expect GA as our new any features providers are coming? You wanna give us the the quick road map? Yeah. Sure. So the road map is, you know, we started opening up to we're running towards private beta. You know, we're sort of transitioning from alpha to beta. I'm not sure where the cutoff is, but we're bringing in all these people that that are asking for access, and we're getting feedback and fixing. So the goal is to have some sort of publicly available beta in the fall, I guess. You know? Can't really launch in the

1:25:24 summer anyway. So we're just gonna improve as much as we can and get as many testers as we can and then fix things as they break. There is some major UX improvements. You know, we we need to finish the UX. It's still pretty rough, and we need to get kind of everything in order, and then we'll launch. Yeah. And then from there, we'll see. We also wanna make it clear from the start where the the cloud service fits, and we wanna be very transparent with the community early on because I think confusion on where the open source

1:26:00 part ends and where the cloud and proprietary part starts, that cause problems down the road. So we wanna avoid that. And, you know, I think everyone can agree. If it's if you like the project, then you probably want the people building the project to make money in a healthy way so that the project doesn't die. So we're gonna test all of that and roll it out. So, yeah, roughly, in the fall, things will open up a lot. And if people wanna play over the summer, they should subscribe now or they should register now, and, we'll let them in at

1:26:32 a rate of 50 people per day. And, again, get up if you're listening. Yeah. That's that's that's the that's the road map. Awesome. Pretty simple. Well, I mean, I'm looking at this, and I I don't think this is gonna finish. So I I I think it's What's the problem now? I think it's just my computer. I think it's just going to slow. So what I'll do is I'll play around with this another day on another session and and try and do something a bit more fun with it. I've been meaning to migrate my blog over to it anyway and actually deploy

1:26:47 Conclusion and Call to Action

1:27:06 it that way. So yeah. I'll blame my computer for now. On the fly Docker Compose to Kubernetes synchronization or something like that. Yeah. And then go to Lambda. Can I deploy to Kubernetes yet? Does Dagger do? Yeah. Yeah. Yeah. You can deploy to Kubernetes and ECS in parallel, and you can have you can have cross cloud deployments. You can have lots of cool stuff. Alright. Well, clearly there's a lot more fun to be had playing with Dagger, so I'm sure this will not be the last stream where I'm messing around with it. But I wanna thank you both for joining me

1:27:43 today. It's been a bit of time just sharing your knowledge or expertise or the history. All of that is is really cool and getting a look at how Dagger uses Q and BuildKit was a whole lot of fun. For the people watching Thanks for having us. A last reminder, the Solomon is on the podcast on Thursday. Check it out on Twitter. Alright. Well, thanks again. Any last words before we we we set ways? Come check it out. Write code. Try and come and break it. Help us fix it. Alright. Yeah. Check out Dagger. Thanks for your

1:28:17 time. Dagger.io. Sign up. Have some fun. And again, thank you. Have a great day. Bye. Bye bye.