0:00 Introduction: Problem with Multiple IaC

0:00 I have a problem. I like to experiment and play with infrastructure as code tools. And in fact, I don't just use code for infrastructure. I use code for everything. Setting up my personal email addresses, configure DNS records, deploying Kubernetes clusters, you name it. As you may be aware, I used to work for a company called Pulumi. They're one of the leaders in the infrastructure's code tool, but not the leader. So since I've left Pulumi, I've been wanting to make sure that I keep up my skills with these other tools, notably Terraform, Terraform CDK, CDKs, and Crossplane.

0:35 However, when you start to use more than one infrastructure as code tool, you have a problem. As you start to use a micro stack pattern, using Pulumi c to spin up a Kubernetes cluster, Terraform to manage DNS records, CDK to deploy to the Kubernetes cluster and so on. You have this explicit set of dependencies, which are quite hard to make explicit inside of a code pipeline. These can typically be in different repositories and chaining repository actions together is already very tedious. So there has to be something out there to help make our lives easier. Right?

1:12 Exactly. Today, we're gonna take a look at Env0. Let's dive in. First up, what is Env0? Well, it is a tool for infrastructure as code, but not an infrastructure as code tool. What do I mean? Well, you can't use Env0 to provision cloud infrastructure, but you can use an infrastructure as code tool like Pulumi and Terraform and CDK and have m zero act as an orchestration or workflow layer. M zero can even manage your Terraform state if you need somewhere to keep it safe. So we're gonna take a look at a demo today where I've got like a real world example

1:51 Demo Scenario: Pulumi, Terraform & Dependencies

1:54 of me personally wanting to spin up a Kubernetes cluster and for today I'll use Civo Cloud and I'm going to do it with Pulumi. But I also have another stack, an application that maybe wants to deploy something to this cluster. But step one is actually to connect a IP address with the DNS record because my applications aren't that good if I can't route traffic to them publicly. So let's dive into the code and take a look at how m zero works. So let's take a look at the Pulumi stack first. This is a very simple Pulumi program.

2:25 Examining Pulumi Code & Env0 Config

2:31 We import the Civo package. We create a firewall. We create a cluster. I'm reserving an IP address so that I can use an annotation on the ingress controller and tell it to use this IP address within the cluster. Lastly, I export the load balancer IP and the kube config just in case I want to use these in another stack. Now this is a very traditional Pulumi program. In fact, there's no concept of n zero here at all except for the Env0. Yaml. Now, we just need a little bit of information here that tells Env0 how to get

3:11 ready to run this Pulumi stack. It already knows how to do a Pulumi up. It already knows how to do a Pulumi preview and it already knows how to do Pulumi destroy. What it doesn't know is whether your Pulumi program is JavaScript or TypeScript or Node. Js, Python, Java, Go, etcetera. So what we're actually doing in the m zero dot yaml is providing some steps for deploy and destroy. And all we're saying is that after we set up any variables, run an NPM install because we need all of the NPM, all the node packages to be available for Pulumi

3:47 to run successfully. And this is just a simple debug command, but before we run a Pulumi login, it may be useful to having your logs the Pulumi version. So let's do that. We also repeat the same steps for destroy. Next, we have a Terraform program and there is nothing m zero about this directory. You'll see here that I have a civil token and a Cloudflare token. We then specify the Terraform provider block and we say that we require civil and Cloudflare. We then configure our civil and Cloudflare providers and then using two data lookups. One, I'm looking up the Cloudflare zone for

4:03 Examining Terraform Code

4:28 the Rawkode Academy domain. Next, I'm doing a lookup against Civo Cloud to grab the load balancer IP address. Lastly, I create a Cloudflare record. If you've used Terraform before, none of this is particularly new. And we don't need any m zero yaml here because Terraform is Terraform. It's HCL based. We don't need node modules. We don't need Python modules, etcetera. When we have dependent stacks like this, we need to be able to define a workflow that says, well, we need the platform to exist first before we do anything else. And this case, we want to spin up this evil cloud

4:55 Defining Workflow Dependencies

5:07 Kubernetes instance, and then hook up the DNS record. Why? Because there's a dependency. The DNS record needs the IP address of the load balancer before it can begin. So you'll notice in the top level of this repository, we have an m0. Workflow. Yaml. And all we're specifying here is a list of environments and any dependencies that we have. It's a small bit of glue, but it does powerful, powerful things. First one, we define the platform is called platform with a template of platform. Don't worry if you don't know what the template is. We're gonna take a look at

5:47 that through the m zero UI in just a moment. Next, we have an environment called DNS, which has a name of DNS, a template of DNS and it needs the platform to run first. This is how we encode our dependency and guarantee the running order of these two stacks. Not only that, when we do a destroy, it'll run them in reverse making sure all the dependent stacks are destroyed first before any of the base layers. Pretty neat. Right? So let's pop over to the m zero UI and get this up and running. Here we have the m zero UI and I've created

6:20 Env0 UI Overview

6:26 a project called level up. If you click on level up, we'll see that we have no environments. Just a moment ago, we took a look at the m0.workflow.yaml which listed the environments. We have or we want a platform environment and a DNS environment. And those had something called templates. So let's go to organization templates, where I've already configured our platform and DNS. But let's click on settings and see what they look like. First, a template we have to tell it which infrastructure as code tool we are using. You'll see here I've selected Pulumi. However, I could have picked Kubernetes,

6:50 Configuring Env0 Templates

7:11 CloudFormation, a workflow, Terraform, or Terraform. But let's go back to Pulumi. We got the template a name here and calling it platform just because that's what I have in my m0.yaml. Feel free to provide a description if you want. You then get to select the version of Pulumi that you wish to use. I am going to use latest. You get to define reshi logic on failure, whether to do it on deploy, destroy or both. And if you need an SSH key to access any of your code because they're in private repositories, you can configure that as so.

7:49 Next, we click on VCS. Here, we have to point it to the repository. For today's example, I'm using gitlab.com and I've already authorized myself as a user. From here, we can type in the name of our repository. For this, it's Rawkode Academy DevOps tutorials m zero. We choose the branch that we want it to run against. Main is a pretty stable choice. And then if the code, the infrastructure is code that you want to run as this template lives and a sub directory like mines and the Pulumi directory, you could that here. Next, we specify the variables.

8:30 So I know that I want to store my state using Pulumi cloud. So I need a Pulumi access token key. And you can paste in your own value there. My Pulumi code also speaks to the Civo API. Now, I could use Pulumi secrets, but instead I've opted to use Pulumi token as part of the environment variables where I've also saved an API token. Lastly, we tell it the projects are allowed to consume this template. I don't need to save this because nothing has changed. For DNS, we'll open it up. You'll see it's configured as a Terraform template.

9:10 The name is DNS and just like Pulumi, I get the version, the retry logic and any SSH key. This is pointed to the same repository, the m zero DevOps tutorial repository on the same branch, but the directory the code lives in is different. In this case, it's Terraform. On variables, almost the exact same thing. However, I was able to use this wonderful load variables from code button and we'll take a look at that in just a moment. But m zero has really good knowledge and understanding of Terraform programs. If you add a new variable block to

9:48 the code, you can tell emcee to scan it and make it available on the UI. And lastly, we've set the project to level up two. So let's pop back to our project environments. Here I'm gonna say create new environment. Now we have a choice of just using our two templates, but then there wouldn't be any dependencies mapping between the two. So instead I'm going to say use a version control system. And this time I'm not specifically configuring Terraform or Pulumi, but instead a workflow. It's in the same repository with the same user, m0 like so.

10:02 Creating the Workflow Environment

10:34 I want to use the same branch, only this time the m zero workflow file is in the top level directory. We can indicate that with a dot. I click next. We don't need any variables and we can give this any environment name that we want. I'm going to call this production. You can provide a workflow workspace name if you want, however, is not mandatory. We have a default checkbox on which will say redeploy every time we push to get. M zero actually has really good integration with GitHub and GitLab and that when you open a pull or merge request, you will get

11:10 a comment with the plan results. We'll see that in a moment too. If you want to enable drift detection, this takes a cron syntax like so. However, the minimum time or the minimum drift interval is one hour. So we'll say to run at five past every hour and click done. As part of this new environment, this new workflow, a run for both of our templates will now begin. So, we'll come back in just a moment when it's completed. Okay. The workflow run is finished. We scroll down. We'll see that the platform has a tick and the DNS has a

11:33 Initial Workflow Run

11:57 tick. This means that both runs completed successfully. Looking at the platform, we can see the clone, loads the MZ file, does some preview and stuff. And eventually, we have the longest part of the run, the Pulumi up at three minutes. We can expand that and see all of the Pulumi output as well as the resources, as well as the explicit outputs we defined in code. So, let's make a change to this repository. First, we want to see the variable scan. So we're gonna add a new variable to the Terraform and we'll call this name. Not very original, but it'll it'll do.

12:30 Modifying Code & Variable Scanning

12:44 Let's also add a comment to our DNS record. And we'll just say hello and zero. And let's make a small modification to the Pulumi program and we're just gonna request an extra IP address and we'll call this platform app. So let's be a good citizen within our organization and create a branch. New million dollar idea. We can do a git add. Be the worst developer in the world and commit changes. And we'll do a git push. So let's pop over to GitLab and open the merge request. And create. If we click on changes, we'll see the

13:38 two modifications that we made or three modifications that we made to our program. And if we pop over to pipelines, we haven't configured any CICD for this. There is no dot GitLab YAML workflow file. This is m zero speaking to GitLab as a GitLab application. And if we click on our pipeline, we'll see that m zero is running a plan for our production environment. So let's click on it. And we'll see that it's already succeeded. And if we look at the Pulumi preview, we'll see that we have one resource to create. So let's go back to our merge request.

13:44 GitLab Integration & Plan Preview

14:24 And if we scroll down, we have comments. One, we can see that Pulumi wants to add one resource and four are unchanged. And if we expand Oh, someone forgot to set a default value. So let's do that now. Let's pop open our Terraform. Set default, David, add, commit and push. So let's open the merge request. We click the button and we say create. And what's happening, mZero has a GitLab application, which has injected a pipeline for us. We have not created a GitLab workflow file. This is all handled for you by using mZero. If we click on the pipeline,

15:28 we'll see that it's running our DNS and platform jobs as part of our production environment. You can click on this and it will take you to m zero. And here we can see that this plan was a success. If we scroll down, we'll see that the Terraform plan wants to update the Cloudflare record with the comment that we added. So let's browse back to our merge request. And if we scroll down, we'll see that we don't have to comment yet. So let's just give that another second. There we have it. Now we have our comments. The Pulumi plan has succeeded and it

16:13 wants to create one more resource, which is, of course, our new reserved IP. Down here for the Terraform job, we see that one resource needs to change. So, I'm not gonna wait. I'm just going to merge this immediately. And done. So let's head back to M 0. And you'll see that our production environment already has a deploy in progress. Now that our merge request has been merged, it's going to run the Pulumi up and the Terraform apply. Let's just give that a moment. Well, Let's head back over to our project templates and click on settings for DNS.

16:25 Merging Changes & Automatic Deploy

16:58 Let's go to variables and you'll see that we still have our Civo token and our Cloudflare token, but we did add that extra variable to our Terraform code. So let's click load variables from code. I love that feature. It has scanned our HCL detected that we have key of name with a default value David, which we can override like so and hit save. So as we've seen so far, we can take multiple infrastructure as code tools, repositories, or subdirectories, build workflows with explicit dependencies. This part's great for the options that we've already seen on the m zero UI.

17:45 Terraform, Terraform, Pulumi, CloudFormation. But what about CDKTF? I'm a big fan of Terraform CDK. So let's see an action as part of m zero's workflow. First, we'll create a directory called CDKTF, where we'll run CDKTF in it. Let's go with TypeScript, and the name and description is just fine. Now we're not going to use Terraform Cloud because m zero is actually managing all of our state files in the background. We didn't actually specify any state storage for the previous Terraform program and nor do you need to. Now for this example, I'm gonna keep it relatively contrived and we're going to use the

17:53 Adding CDKTF (Terraform CDK)

18:34 random provider. Let's go back to our code. We have our new directory, a main dot ts, and we have an empty stack. Now, we don't actually have to add anything to the stack in order to show it working with m zero. We can actually just go to the CLI and ensure that CDKTF synth runs, which it does. Now the way that CDKTF works is you write your code in TypeScript, you run a synth, and it generates HCL for a normal standard Terraform run. So let's configure m zero to do this too. Let's go to our code and we'll create

18:38 Configuring Env0 for CDKTF

19:18 an m zero dot yaml. We'll go to the Pulumi one and copy it to make this a little bit faster. We'll delete destroy for now and just focus on deploy setup variables after. Here we're doing our NPM install. What we can also do an NPM install global CDKTF CLI. Now because this is a global install, it will require sudo. From here, you can actually just run CDKTF since. However, we want to copy all of the generated code and to the current directory. We just need to put the stack name in here. If you're unsure, run the CDKTF

20:12 locally, the CDKTF synth, because you can just expand the dot out directory, the stack, you'll see the stack name and then we have our JSON. And we want to grab that to the current working directory. And we'll want to copy this for destroy, Like so. So let's modify our workflow file. Now we can add CDKTF, name CDKTF, template name CDKTF. And we'll see that this needs platform and DNS. In fact, let's remove the DNS and we'll see that m zero will run these stacks in parallel, which is also kind of cool. So, let's commit this

21:07 and push. So if we come back to our project, we will have our production environment. Now it's inactive and passed, but if we click on deployments, we'll see that our last run failed. And it's failing because we do not have the template CDKTF. Let's go to organization templates and create new. Now because we want us to run the Terraform eventually, we just select Terraform. We call this CDKTF to match what we put in a workflow YAML. And click next and configure a repository one last time. And the directory for this is CDKTF. Now we don't need any variables, so we

21:41 Creating CDKTF Template in Env0

22:16 don't need to parse anything here. So let's just click next and done. Awesome. Let's go back to our project environment. Click and we'll see this field run here. Let's kick off a new one. Main branch and run. Fingers crossed. And now we can see from the graph that the platform is running and that our DNS and CDK Terraform are dependents. So let's give that a few moments to finish. Ta da. Now we have m zero workflows handling dependencies and workflows for Pulumi, Terraform, and Terraform CDK. Sweet. So m zero is a really cool tool. As you explore infrastructure as code,

23:14 Benefits of Env0

23:18 mZero gives you a few key benefits. One, it understands Terraform pretty damn well. You don't have to worry about your state file anymore. Let mZero worry about it for you. It can also scan and pull out variables and bring them into a UI for you to configure. It gives you a nice graph of all the resources that are being created, integrates through your source control system for pull request updates and plans, and handles all the workflows and dependencies across these tools. I can't state that enough. As you adopt infrastructure as code tools, these dependencies are always very implicit.

24:01 We always have the big bang problem. What if I wanna spin up a new environment from scratch? Well, it's not a single Pulumi run. It's not a single Terraform run. It's actually many of the above. The m zero brings a way for us to codify that too, which is pretty neat. Go check out m zero at the links in the description below and on the screen now. The code is available at gitlab.com/RawkodeAcademy, where by the time you watch this, I'll have cleaned up the commit history. Promise. So go have some fun. We'll catch you all next time.