1:57 Introduction

1:57 Hello, and welcome back to the Rawkode Academy. I'm your host, David Flanagan, also known across the Internet as Rawkode. Thank you for joining us today. Today, we are continuing our exploration of cloud native and advanced event driven architectures using any new forward future facing software that we can get our hands on, and today is no exception. Now today, we're gonna take a look at adding a new feature to Guidepad, and that is a project that we have covered once or twice on the channel before. The secret about today's episode is is that we are joined by both members of the

2:30 founding team. Hello, Matthew and Reggie. How's it going? Hello. Good. Thanks for having me. Great. Thanks for having us. Really happy to be here. Absolutely. My pleasure. I think I've made my feelings about Guidepad very clear to people over many sessions now, but let's get back to that in a minute. First things first. Can you both just say hello? Tell us a little bit about you and what you're up to. Yeah, Rich. I'll defer to you. You go first, and and I'll Sure. Pick it up a second. Yeah. Yeah. I'm the cofounder and CEO of

2:56 Guest Introductions and Guidepad Overview

2:59 Guidepad, and my background is in consulting. Did a little a a lot of work at cloud about ten years ago. Really got interested in distributed systems, built a platform, sold some software to the government, and met Matt about four or five years ago. We got started building software, and we built Guidepad as our own internal developer team, tool. And, it's kinda taken us over the last few years, and we're really excited about what we built and where we're going. We are a unified developer platform, so we could cut across a few of the main pieces of building, you know, software

3:35 such as DevOps, AIML ops, data integration, and workflow automation. And I'll let Matt introduce himself, and we'll go from there. Cool. Yeah. Thanks. So I'm Matt, CTO, cofounder. Yeah. Like Reggie said, we met, you know, four or five years ago. My background is a little different. I was in academia for about seven years, did biotech for a bit, bounced around in there, fell out into normal tech, and then, you know, met Reggie and the rest is sort of history. So, yeah, Guidepad is sort of our attempt to crystallize all of our, like, experiences over our years,

4:13 our joint, you know, combined years, like building these distributed systems. You know, when building these sorts of systems, it's it's hard to, like, standardize things. It's hard to make things repeatable. It's hard to make things observable. And so Guidepad sort of takes all of our learnings about how to do those things well, bundles it up into a a platform, and then tries to, you know, make it available to other folks. Awesome. Alright. You said something that I hadn't heard before, but made a lot of sense when you said a unified developer platform. So maybe the both of you can dive

4:44 into that a little bit and just help the audience understand, like, what can Guidepad deliver and what problem is it solving? Sure. Yeah. So, you know, when you think about building distributed systems today, you're you're probably talking about, you know, half a dozen tools off the bat. Something for your your CICD, something for your your back end, something for your logging, your issues tracking, data engineering, right, managing your infrastructure. So Guidepad has the building blocks to put all of those things into one tool. And then whether you wanna do stuff at in a user interface or at the command

5:20 line or in like an interactive coding session, Guidepad has ways into all of those things so that you can do what you need to do without having to jump between a bunch of different tools, essentially. But it's also, like, not that opinionated that you couldn't sort of integrate Guidepad with stuff that you're already using. So, you know, if you have a preferred workflow management system, but you wanna use Guidepad for, like, generating endpoints or managing like your units of work that go into that that pipeline system, you can do that. If you don't have a workflow management system

5:52 but you do have maybe a CICD thing, you can use Guidepad's work plan system to automate, you know, the tasks that you're doing. And sort of, like, as you get more comfortable with the system, you can, you know, you can put its tentacles out into more of what you're you're doing. Yep. Nice. Anything to add there, Roger? No. I think I think he hit the the nail on the head. You know, a lot of our customers right now have come have come to us. Some of the use cases that we've seen, a lot of our

6:22 clients, larger enterprises are looking for, like, a unified, like, cloud control plane. You know, most most customers we talk to are hybrid in nature. So they have workloads across AWS, Azure, Google Cloud, and some still have workloads on premise. And so having one tool that can help manage all of that, manage and control everything. And then data engineering and workflow automation, has has been another big thing for us. So, you know, as as you'll see, once once Matt goes into the demo, we start diving into the tool, you know, very little code that has to be written. We we

6:58 do things dynamically at Rawkode. Yeah. Yeah. So we have covered this before, and the very little code to be written really highlights one of the key strengths of Guidepad. And something that you promote on the website as well is that you can build a platform or software as data. But for anyone that hasn't seen anything previously, maybe we can go into that in a little bit of detail just for a few minutes before we go into the actual live portion of this. But what does that mean? How do people build their new pieces of software and their platforms using Guidepad as that

7:09 Guidepad's "Software as Data" Approach

7:28 substrate? Sure. Yeah. So it really you know, when we think about like a platform as as data. Right? We're we're talking about moving, building a system from sort of a a build the system sort of thing where you go in and you write code for each of the pieces. This is sort of like the old sort of infrastructure as code thing. Right? Where you have a bunch of files, they're spread out. Maybe maybe you're good at organizing them and they're in one repository. More likely they're not in one repository. They're in many repositories. They all have distinct, you know, like ownership

8:00 and maybe, you know, maybe the folks that own those things don't get along all the time. So there's a lot of there's a lot of friction, right, when you're stuck sort of in file land. And so what we try to do with Guidepad is move you out of file land into data land where you can be more collaborative. It means that you're rather than writing, you know, scripts that orchestrate your your software and your applications, you are writing specs, so configuration. And then Guidepad takes those specs and produces things for you. Right? Whether it's the the

8:35 commands that need to go out and deploy a web server in your Kubernetes cluster, or Guidepad needs to manage an ingress for you or Guidepad needs to create an s three bucket for you. Those things become configuration problems rather than sort of code problems. And when you're in data, you can sort of move the paradigm to I'm not building a system, I'm programming a system. So things become more repeatable, things become more observable. You can write then the scripts that you're writing are actually orchestrating all of the components of your system that are stored within some data store storage engine.

9:14 And we'll we'll talk about this more when we go into the platform, but Guidepad is sort of data storage agnostic. It has what we call the type system, which is a layer on top of what you would consider to be like an ORM. But that ORM, you know, allows us to plug into, you know, different data storage things whether that's a database, Postgres, you know, MongoDB. You can do things you can actually store parts of your system in files if you want to. You can have a data store that's an s three bucket. You can have a data store that's

9:41 an API. So there's a lot of flexibility as far as like where you store the parts of your system. And what that what that allows you to do is to tailor that storage to the performance requirements of each sort of piece of your system. So it's like, this particular data structure works the best in Postgres. So let's put it there. Well, this part, we actually don't really care about the structured stuff so much. We wanna be able to store a bunch of disparate, you know, polymorphic types of things. Let's put that in MongoDB. It doesn't care so much

10:10 about the schema. There's less overhead about maintaining stuff. Yeah. I'm rambling now. So I'm gonna stop doing that. And if you have a question, have it answer it. No. It's not rambling. You're, you know, you're just sharing how the system works. I think a lot of us be solidified for people as they see us explore the platform, add new features, and kinda show how to work with it. Maybe that should be a good time then for you to discuss or share what is our plan for today. What are we going to do? Let me should I share my screen now?

10:30 Project Outline: Building a Node.js Lambda Replacement

10:39 I can do that. Yeah. Go for talk through. Great. Let me actually move the the call window over so that I'm not sharing all of us. Here we go. Great. Okay. Perfect. So is it can you see it? Yep. Awesome. So I guess maybe before we get into what we're gonna do today, I can spend just a few moments talking about, like, the high level concepts within the system. That might help folks as we sort of move through into the actual implementation. So this is the Guidepad UI. This is the instance that I created just

11:02 Guidepad Concepts: Instances, Environments, Services

11:24 for this, you know, this livestream. So every Guidepad installation we call an instance. Typically, it's, you know, it's one per customer. They all get their own domain name. You can we can host them or they can be hosted on premise. We have an installer that sort of stands one of these up in like a couple minutes. And with this installation, you get a what we call an environment. An environment is a bundle of resources. So draw a logical box around like an AWS account or Kubernetes cluster or an Azure account. And that's an environment. Right? Environments can be big or small. Just

12:02 depends on where you wanna draw the boundaries around them. You can see in this particular instance, we only have two right now. We have, a Kubernetes cluster here, which is what is sort of hosting all the infrastructure for this. And then we have a build excuse me. We have a build environment, and that uses the packer from from Terraform. We'll we'll actually use that as part of part of the demo. So environments are bundles of resources. Into those environments, we put services. Right? And so services are the things that provide capabilities for your they're actual, like, parts of your system. Right?

12:40 And so in this particular instance, we have the things that come along with every Guidepad installation. So we have the user interface, which is what we're looking at now. We have two things to manage work plans, so the automation piece here. We have an open search data store, which is used to store the logs and the events that go on in Guidepad. I think No, not issues. Issues are stored in the main system. Anyway, I'll get to that in a second. And then we have this thing, which basically just manages ECR credentials so that this cluster can access all the

13:14 images that it needs. And then this one here is the generative API, and not generative in the Gen AI sense, but generative in the I read a bunch of operation specifications and make endpoints. And it does that in all the frameworks that we support, and that right now that is REST, GraphQL, and I think we do it over message exchange. Yeah. We have a it's not deployed in this instance, but we could deploy wrapped in queue server, and then that server would accept, API requests and give the results and all that good stuff. So I talked about operations,

13:47 and we'll also use operations in our, in the feature we're building today. So operations are portable, repeatable units of work that have predefined, inputs and outputs, and a handler. So you can think of them sort of as, portable functions. And those you know, these operations, they're they're data records, and then you can sort of summon them up at will and and use them. But the Guidepad API uses those definitions to create the endpoints that I've I just I just mentioned. And the nice thing about operations is that they can execute in any environment that you've

14:04 Operations and Execution Environments

14:21 already set up for your instance. So for this particular demonstration, the operations that I run can either run locally on the machine that I'm looking at here, or they can run-in that Kubernetes cluster that was in the environments list. And I'll show you I'll show you an example of that, like, as we as we get into things. Okay. So, we have environments. We have services. How do you manage the services in the environments? Well, we do that through what we call state plans. Right? And so I'll go into a state plan and sort of show

14:49 you how that works. So state plans are the actual commands that are used to get the service in that particular state in a given environment. And we'll take a look at this deployed plan for for Kubernetes. This is sort of the default one. Every service when you create a service definition, if it doesn't have state plans defined within that definition, it just uses the default ones, this is one of them. You can see it's just one one little node. All of the state plans are DAGs, right, just directly cyclic graphs. They're made out of phases, so every node

15:23 is a phase. That's this box here. And then inside a phase, you have, serially ordered stages. Right? So this one only has one stage, I could add more, below it. And those would op those would execute top to bottom when this phase executes. So if I click into the stage, you can see there's actual commands in here. I apologize. I have to, like, move stuff around. I'm on my tiny laptop screen. But yeah. So these are the actual commands. You can see it runs, a series of Ansible steps when it executes to create a volume claim,

15:56 service definition, and then a deployment. They're parameterized. You can see in here that we're doing templating. Right? So this can pull information from what we call the context of the state plan's execution, and that context includes things like the service that you're deploying, the environment it's being deployed into, any requirements for the service. And I'll talk about requirements more later, because they're they're a very flexible way of specifying either what a particular thing needs to to execute, or individual pieces of data that a part of the system needs, like at runtime, to to do the thing.

16:37 So yeah. So, you know, you use state plans, to put services into particular states. If I go to the, environments, over here, you can see that I've been deploying things. And I think I mentioned this, I might not have, but the the UI is a couple steps behind the the core capabilities of the system and we're on a develop branch. So, any visual bugs, you'll just have to forgive me for where we are aware of them. But I wanted to give you the latest and greatest, so that's why this is showing not a number. Anyway,

17:15 Yeah. So you can see that, these are these were all deployed earlier. And so they went from transitioning to deployed, transitioning to deployed, to deployed them a couple times. And then these are all the services that are in the system. Like, it shows you the ones that, like, you've deployed into this environment and then these are all of the ones that haven't you sort of been haven't interacted with this environment yet, but they could. Yeah. Okay. So environments, services, state plans, state machines. I haven't talked about state machines yet. This view is broken, so I'm not gonna

17:43 State Machines for Automation

17:45 click into it. But what state machines do is they orchestrate state transitions. Right? So it's a it is like the literal, you know, computer science definition of a of a state machine. Right? It's got states and transitions. But what Guidepad can do is it can use your state machine. I'll actually show you this, like, in code form because I have it there, when we we hop out of this view and into a different one. But the state machines are can be set into autonomous mode. That's this little toggle here. And when I do that, there's that service

18:14 called the orchestrator that manages the state transitions for the state machine and then uses the state plans autonomously to move the the service that's attached to the state machine from state to state when the requirement conditions are are met. And, David, we actually that's the thing that we did for the, the s three bucket, that we were managing autonomously. So, like, enforcing that the permissions were set properly. If someone deleted it, it recreated it. Yeah. So that was using state machine concept, in that in that orchestrate, Barry, that I that I mentioned. Which is for anyone who hasn't seen the

18:49 previous videos. We've done a couple of cool things with Guidepad to date to show the kind of benefits and power behind the platform. One of them was we built an s three reconciler that made sure that an s three state, like, IAC, was always reconciled to the correct state. Even if it was deleted, that was recreating stuff. That was very cool. People should go check out that video, and I'll put a link to it after the the livestream. And the other thing that we have built before was the URL shortener was some generative AI summarizing of the HTML components as well.

19:20 Yep. You can do a lot with Guidepad. And you've kind of touched on a lot of the primitives that are exposed to people. Right. And then the way that you connect these together with Guidepad that kinda makes them kick in and and do the thing that you need to do. So yeah. Right. That's that's sort of the magic is is, you know, we we there's been a term that sort of come into vogue recently, which is like compositional systems and compositional software. I think Guidepad is sort of, just by its very nature, like a system like that. Right? Because you

19:50 have these primitive entities. You can extend them. You can customize them. And then you can put them together to make basically anything that you want to make. And I think maybe that's a good segue into talking about the architecture of the thing that I wanna build on the stream unless you have another question or anything before I before I do that. No. Take it away. That's great. Perfect. Okay. So what we're gonna do is we're going to try to build an AWS Lambda replacement for node. So the ability to run node code serverlessly through the Guidepad framework.

20:16 Project Architecture Diagram

20:33 And so if we think about what if we think about what Lambda is doing, right, so you've got, AWS up here, and then you've got, you know, your Lambda function. So this Lambda function, and that's a colon that should not be in there. So get your lambda function, that, like, executes I don't know. There's some black box. Right? There's some containerized black box over here that they run your function in because, know, you can attach a particular, like, image to it. So you need a Docker image that contains contains the, you know, your code artifacts and all the requirements. That's, you know,

21:19 if you wanna use any packages that aren't contained within Lambda's default runtime. So then, you know, and then out here, you maybe have a, I don't know. Think they released a thing back in 2022, which is like, you know, Lambda URLs. Right? So you can have like a Lambda URL in front of it. And then you've got your little client over here. Let's find a nice icon for a client. You know what? I hate the mirror changes. Let me just find client. No. You're not gonna help me out. Alright. Fine. Whatever. I'll just use, this again.

22:01 Okay. So you have your client here. Right? And it's gonna send the requests, to the Lambda URL. I don't know why I made that so big. Let's make that smaller. Thank you. So it's gonna send that over there. And then Lambda URL is, you know, attached to your thing and this is gonna use your Docker image to, you know, start the thing in the black box. And then when the black box is done, it's like, oh, hey, here's the result. So it's gonna go back to the client. So, we're gonna try to build something like

22:34 this in Guidepad. And what does that look like as far as, like, Guidepad components? So, what we're gonna do is we will have our environment. And I'm gonna draw that with a dotted line. Alright? So this is our Guidepad environment. What's called the Rawkode KDS environment? So, we got this thing here. And then inside there, what we need to do is we have to have some we have to have two things. Right? We have to have something to represent the code that we wanna execute, and we need to have a place to execute it. So the place that we're gonna execute it

23:16 is what we call a language executor. Let me just draw one of those. And full disclosure, I kinda already have this for Python, but we have a full stack engineer on the team who isn't super well versed in Python and would rather write stuff in JavaScript. And so I'm trying to be nice to him, and, you know, give him a way to write the operations in the way that he would wanna write them. Yeah. Okay. So, I got that. It's a solid. And so this is gonna be our node I don't know how that happened. Node JS language

23:56 exec Nobody can type when they're being watched. It's just like an unwritten swear to I swear. Like, my my TPM oh, no. Yeah. My words per minute go down to, like, 60. Right? When, like, when I'm I'm being observed. Yeah. Yeah. It's Schrodinger's words per minute. Right? Like, they're they're incredibly fast as long as no one is looking at me. Okay. Yeah. Okay. So we have language executor, and then the other part we need to have is the operation. Right? So we need a Node. Js operation. And this Node. Js operation is going to contain a couple things.

24:34 So, actually, let me let me delete this internal label and I'll make it a little bigger so that I can put some stuff inside it. And and the stuff I'm putting inside it are gonna be they're, like, represented by database references, so they're not literally contained within this object. And I should say that these are objects. Right? Like, in in a database somewhere. Right? Like, we're gonna have a language executor record that exists somewhere, and then we're gonna have an operation, that also exists. So we're gonna have the language or sorry. We're gonna have the Node. Js.

25:12 Js operation. Right? And within that operation, we're gonna have a couple things. We're gonna have the input type. We're gonna have the output type. And we're going to have the, the implementation, basically. Right? So let me make this a little bigger. This a little bigger. Because I have to draw some other things. K. And Miro recently changed yeah. It recently changed the thing where if you drag the highlight, it also highlights the thing in the background. So I can't just, like, select. It's it's so annoying. Anyway, so we've got this stuff here. The implementation is interesting because in order to store

25:52 Artifacts and Storage Backends

26:10 it right? So if you think about the implementation, right, that's just a handler. It's the code. But in order for it to be portable, it needs to be stored somewhere. So what we do in Guidepad is we have this notion of an artifact, which is stored in what we you know, I hate saying we call it because it's such like a simple word of like, of course, you would call it this, but we have what we labeled storage back end. And a storage back end is just an abstraction layer on top of different, like, storage

26:36 things for file like stuff. So for for data like stuff, things that are either structured or semi structured, have types and data stores. For things that are unstructured, we just like file blobs. Right? Those are artifacts and those go in storage back ends. So for the implementation, what we're gonna do is we're gonna store that in an artifact. Right? So we're gonna have an implementation artifact. And this is gonna contain the code of our little Node. Js execution. Full disclosure, it'll probably be like console dot log something, just to make things not not that complicated.

27:15 But we can get more complicated later, just just a proof of concept. Right? You're gonna walk first, or crawl first and then walk. So you got that. The other thing that might be obvious to folks, like an obvious question, right, is like you have this operation, you have the executor, like this thing is supposed to run the function. Like, does it how does it do that? Right? Like, how do you get from from here to to here? Well, the way we do that in Guidepad is we have this idea of a service exposure. So service exposure typically sits on the boundary

27:39 Service Exposures and Public Interfaces

27:47 of a environment. Right? Sorry. I can close this. Right? Yes. Thank you. Okay. Perfect. Yeah. You can just click paid on that if you mock that. Oh, this too. Perfect. Okay. Thank you. I wasn't sure if that showed up to the viewers or not, but I'm glad that I I had it. So, anyway, so typically a surface exposure sits here, like, this boundary. Right? And it allows traffic into, a particular, service on a particular protocol. And so Guidepad supports a couple different protocols, the usual ones. Right? HTTP, HTTPS, TCP, UDP. You can make service exposures through all that

28:26 stuff and then point them at your services. And then the environment that you're living that that thing is living in is managed by Guidepad will expose it in the way that you want it to. Think of if you think about, like, the Kubernetes service concept, that's what this is except one level up. Right? So it's not just Does your exposure map to the executor, or does it map to something else first? It seems a bit weird that it goes straight to an executor. Right? Well, so the way that it it does go straight to an executor.

28:56 Because in the service exposure, what we have is a reference to the exposed service within the record. Right? And so that just points right to it. And then it's the job of the environment's public interface, which is another entity, to take the service exposure and then manipulate the environment in the way that it needs to to make sure traffic can be routed properly. But that's not what we're gonna use it for here. What we're gonna do is link the operation to the service exposure. Right? So Guidepad has the ability to say this operation is actually handled by

29:34 the service pointed to by the service exposure. So when you call execute, you don't hand like, because a node a default operation in Guidepad has an execute method attached to it. And that runs normally, takes the input, calls the handler, produces the output, and returns it to the caller. When you link a an operation to a service exposure, it tries to see if Guidepad knows the protocol of the service exposure and then will opt then will handle the operation over that protocol rather than call the internal execute method. And so that's what we're gonna do here.

30:08 Right? So that it, you know, so so when this thing is called, it's gonna be like, oh, hey. I'm, like, actually linked to the service exposure which is linked to this Node. Js executor. I'm gonna send my request over there, run, and then return the results. And what it also does is sort of shunts this into here so that it can run. Right? Like at run time. Like, you know, transfer. Right? So say calls and then calls and then I guess this thing, yeah, this thing uses that. So anyway, so this is what we're gonna do, right, in order to

30:47 get it to to work. Does that make sense or any questions or things I should clarify? Potentially. Okay. No. I think I think I'm definitely following along. Right? We're I I we're trying to repeat it back, and you can tell me anything that I get wrong. Sure. And we'll see where we are. So let's start with the easy part here. The service exposure is some sort of domain name, port mapping, ingress to our environment. Just a thing. Right? It could be Rawkode.ishere.com, and then comes into the service exposure. That service exposure has a direct link or relationship

31:27 with a Node. Js operation, and that is all bundled together for the executor to know what to execute, I'm assuming, where the implementation is coming from artifact storage, which could be, let's just say, SC compatible API or something like that. And then the executor takes all of that information, the service exposure, the operation, and the artifact executes the thing and then returns our response. That's kind of my understanding at the moment. I don't know if I got any of that wrong. Yeah. You got most of it. The only thing is that it's the actual operation that handles

31:59 all of that negotiation stuff, and the executor is just dumb. Right? It literally just takes a command. So it's a it's a pretty stupid service. I'll I'll show you I'll show you its implementation. But yeah. So that's that's the idea. Right? I think you got it. Is Well, let me ask you a couple more questions then. Of course. Because, obviously, right now, are watching. They get a pretty deep insight into how Guidepad works. Now we're doing this for Node. Js. You mentioned the Python exists. But one of the things that we have discussed before is that

32:34 Guidepad can make this work for any language. I mean, it could be Go. It could be Elixir. I'm assuming it could even be a bash shell script. It's not really that opinionated. If you can write an executor, then Guidepad can do it. Right? Yep. So, I mean And it's very small. I'll show you the executor. Yeah. Let's take a look. Let's go for it. There it is. We only see. Oh, no. There you go. I did it for suspense. Here. Here it is. Alright. So there's an attribute called service type. There's a deep copy. Where does it execute?

33:18 Yeah. That's a good question. So remember we have this operation. Right? Yes. Like, this is the thing that actually does the operating. The service is just so you can have something deployed that can handle the operation request. Right? So that's why it's so small, is it's just like, I need to have a service that I can basically attach an an image to so that when I deploy it, the right image gets deployed. And that's what the service is doing. The operation I'll show you the operation, and that is not in the plug oh, sorry. I showed you a plug in and

33:51 I didn't mention plug ins. So let me let me do a quick aside on the plugin and tell you what that is. So, you can see this is a a repository. Right? It's called Guidepad demo. So Guidepad has the notion of plugins, and plugins allow you to layer functionality on top of the core system. And then what we typically do with plug ins is that we'll have one per sort of, like, industry vertical that we run into. So for example, we we have a services division at at the company, and we're doing something for, an emergency transport

34:24 company. They want us to make a, like, a ride sharing thing. And so what we did is we wrote a plug in for ride sharing on top of Guidepad, and then we wrote another plug in on top of that that is just for this particular customer. Right? So we've specialized Guidepad in this in their instance for transport related concepts, and then we made more customizations just for that particular customer. And then if we had other customers in that vertical, we would customize again. But what I'm doing here with this demo and every plugin is a repository. Right? It's

34:56 in Python at the moment, but, you know, we have plans for SDKs, for all the, you know, all the sort of popular stuff. It's in the road map. We'll see when we get there. But, anyway, so, you know, this is sort of the structure of of a plugin. Right? You have, like, the top level sub package. You have a little entry point here called plugin dot py. This is sort of where everything gets exported from your plugin, And then you define, like, custom operations, custom services, like this demo JS executor. You can do custom types, which we actually

35:28 haven't talked about too much, we will. I I promise. We'll get we'll get to everything. And then work plans, if you wanna have custom work plans. And then at runtime, what Guidepad does is it reads the plug ins that are configured in your instance, of reconciles if there are changes in the remote that you don't have locally. If it does, there are changes that pulls them, installs all the requirements, and then adds everything in the plug in to your runtime environment so you have access to all this all this stuff. And so what we're doing with the, the

35:58 demo JS executor is we're override we're extending a, service that comes from Vanilla Guidepad, right, here at the language executor, which is that's that's the the thing that you were talking about, just a moment ago, David, about, you know, if we wanna do this for another language, right, like, do you what would you need to do? Well, you would, you know, extend this executor, and then you would also, go into I I don't have this in the plugin yet. It's just a normal Guidepad. But you would go and also extend the operation. Sorry. Here. So

36:41 this is the language exec operation. Right? And we wanna know JS one of that of this, basically. Right? So Yeah. Luckily, I've already written it. You can see it's also very small. Okay. So now it's starting to click for me. I didn't when you showed me the class at first, I didn't realize the state machine was above it where it's kinda defining that it's gonna run inside the container. Now we can see the command is no word followed by the file name. Whatever. So now it's starting to come together about how this actually works. Can we take a

37:14 look at that state plan and the the plugin? Oh, yeah. Sure. The The state machine. Sorry. Yeah. That's fine. It's it doesn't work yet. So this is this is gonna be part of the maybe the advanced session after this one, is getting this to work because I thought This isn't the advanced one because my brain is already fried. Come on. Oh, no. Oh, no. I've I need to slow down. This is supposed to be yeah. This is supposed to be like Guidepad, maybe maybe 102. I I didn't realize I was giving, 602. So I can I can maybe

37:52 tone it down a couple notches? But anyway, yeah. So this is this is the state machine that that I wanna use to eventually manage the executor. Right? Because, like, if you think about Lambda, when you run a Lambda function, you don't have to tell Lambda, like, hey. I'm about to run my function. I also have to go manually, like, deploy, you know, the runtime for that function is going to to execute. So we'd want the same thing in our Guidepad replacement. Right? And how would how would you do that in Guidepad? Well, you would do it with a state machine.

38:22 And what that state machine would do, I've I've sort of started to to put it in here, but I haven't I haven't fully flushed it out yet. You would use Guidepad's requirements system to say, if there is an operation that's trying to run, but it can't, so it's in the pending state, I need to take this service and rebuild it and then redeploy it. Yep. Right? And so that's what I'm hoping to encode here in the state machine, for each of these. So I have the transitions built up. Right? So I I have a a transition from not deployed to built.

38:55 Right? So this thing doesn't exist anywhere yet, but you need to go build it. And then the next thing is, oh, I wanna be able to go from deployed to not deployed. So after maybe some cool down period, no one's used the operation in a while, I'm gonna undeploy the service to free up resources in my cluster. And then this one here is, I'm gonna take this from deployed to built. So this is the thing's already out there. Someone has updated the configuration for my executor to require additional packages. So, you know, I've I've updated my handler.

39:30 I now use x y z package in my little node script or whatever. The thing's already out there. I'm gonna rebuild it to include that package, and then I'm gonna redeploy it right after. And then this one just goes from built to deployed. So this is like, build's finished. I'm gonna go deploy it. Yeah. But they're not wired up to all the requirements yet, I can't actually use it. But it at least it sort of lays out conceptually, this is you know? And if and if the UI was working, you'd be able to see these states, like, logically, sort of, like,

40:03 mapped out. Right? Like, these are all the valid states for this type of service. This is how it moves between them. All that's missing are the conditions for the transitions. So Alright. Okay. More questions? Here's the state plan. I'm fully up to speed there. So Okay. Okay. Here's the state plan. Right? So this is the actual commands that happen as we move from phase to phase. Right? Yeah. Yeah. So this is it uses all the default ones, so that's what's happening down here. I'm just pulling in the defaults from, you know, Vanilla Guidepad. I'm customizing the build

40:38 state plan here, And I'm adding build commands. And you could put them in line. Like, you put you could just put them in the same module. Right? But in order to do that, have to kinda do something like this. You know? And then I don't really like doing that. So what I tend to do is put them in a file sort of next to the service. Yeah. That means And then validate. Yeah. Yeah. Exactly. So we can look at that. Is this? Of course, I've broken the syntax highlighting. I did that after I wrote it the first

41:21 time. But I'm using templating, right, to only do this when they're when certain things are defined. So this uses like Guidepads. It's using Ginger two under the hood, which sort of everybody does if you're doing string interpolation. Right? Maybe we'll write our own one day if we ever ever make it big. But anyway, so we just have custom block opens. Right? So that if you're writing something in Ansible or something else uses Jinja, there's no conflicts there. Because if you use the standard ones, it'll just replace all your stuff or make you write, you know,

41:54 curly bracket, percent raw and then that. And then that's also very annoying, so didn't, you know, didn't wanna have to do that. But, anyway, what this is doing is, if there are any system package requirements defined on your service or anywhere in the context that the service is execute like, is in when it enters that state, it will install them here, which uses APK, because we're using an output the node alpine, image as the the starting point. And then down here, it installs basically, if you have NPM requirements. Yep. Yeah. So, yeah. That's that's what that does.

42:35 Oh, sorry. Yeah. System package. So, yeah. So so APK for system packages and then NPM for any node modules that you need. And we can we can add some of these and rebuild when we feel like it. But let's, yeah, let's not do that just yet. Sorry. I I was just peeking at the time. Okay. So more questions or are we we good to start doing things in the terminal? Yeah. Let's go for it. Okay. So sorry. Gotta stretch. I always debate, like, whether I want to do this stuff in, you know, in a notebook. I like working

43:06 Demo Setup: Connecting to Guidepad Instance

43:13 in notebooks a lot, Jupyter notebooks, just because it's sort of repeatable. I can just kill the kernel and, like, rerun everything. If I'm in IPython and I'm making a bunch of changes, it's a little harder to, like, do it. So I'll probably do it in here today. So okay. Let's let's go back to, like, our diagram real quick and just think through, like, what we what we need. So we need a Node. Js operation. We need, this implementation artifact attached to it. We need the Node. Js language executor, and we need the the service exposure.

43:45 So I've shown you the class for the Node. Js language executor, but I haven't shown you the, the instance of it yet. And I actually made that already. I know that's kinda kinda cheating, but I did. So I apologize, for that. But, here, let's take a peek at your in let's let's go under the hood, because I showed you all, like, the deployed stuff, like, in the, user interface, but, like, let's actually use kubectl, to, like, look at the stuff, like, in, you know, in your in your instance. So you probably don't remember this, but in

44:15 the user interface, the actually, I'll just go over there and I'll show you. Sorry. So if you go back to the environment, we go to the settings. So this just so your audience knows, like, this form is auto generated. So what it does is it reads the type definition for whatever type of thing you're looking at and then generates the the wizard, basically, off that type definition. We haven't talked about types at all, but but types are our language agnostic database representation for for schemas. And so everything that we've looked at so far is a type at the end of

44:57 the day, and that's how it's stored in a database. We have a type for environment. Right? And we have a type for Kubernetes environment, which is what this actual type of environment is. So we go through, there's sort of like the vanilla, like, stuff, but then you get to here. Right? And you can see, like, oh, there's a Rawkode namespace. And that means that everything and this is only on Kubernetes environment types. Right? Like, other environments don't care about a namespace, so why would they why would they have that? And so this namespace is actually in the cluster. Right? So if

45:27 I do k a this too small? Should I increase this or is this looking good? No. I think we're okay. Okay. Great. Let's get all the pods. Here they are. You can see this is from my, setup last night. You can see how old this you can see how old this, project is. It's about a day old. That's when the stuff was that's when the stuff was put in here. Anyway, so these are all the things that have been deployed. Right? Like Guidepad is managing all of these bots, essentially, as part of the the deployments.

46:02 And we actually already have a little Node. Js executor in here. Yep. We'll come back to that. But I just wanted to sort of show you, like, here's some stuff. It exists. And now we'll go back into that notebook I was looking at, and I'll show you the data record for the for that service. And, you know, this is sort of the standard way that, like, we interface with Guidepad, like, in an in in an interactive session. You typically don't have to do this. I'm just doing this because I might make changes to my local

46:42 code without I don't wanna have to push it up every time, basically, so I'm sort of cheating, and just setting my Python path. If I remove this, when this happens, Guidepad will pull down that repository for the demo plug in and put it on my Python path for me. So I don't have to do that. I'm just I just want to. So I'll do that. It's gonna initialize Guidepad, and we'll use it for the rest of the session here. It's starting a kernel, so it's taking an extra second. Okay. Cool. So I guess we can walk

47:15 through what what what just happens because this is probably your audience's first time seeing it. So we load the connection details from a local environment. Guidepad can be configured either with a direct connection to the primary data store that I'm using and that's like super admin access. Or you can use it in, API proxy mode where you just have and this is what we do for, like, end users at at our customers. Like, you just give them their username and password and they you can actually use Guidepad through our API. So it translates all of

47:48 the data store requests into API requests, and then uses the RBAC, like, for that user to sort of sandbox all their permissions and stuff. But since I'm me, I don't have to do that, and I can just connect directly. So that's what I did. So, let's take a look at that Node. Js executor thing, that we had just talked about. So I'm gonna import the type registry. Types dot registry. Import the type registry. So, the type registry is an object inside Guidepad that knows everything about all the types in your instance. And the nice thing about the type registry

48:03 Interacting with Guidepad Types (Fetching/Inspecting)

48:26 is you can ask for any type you want from it. So you can just give it the type name or the class name. I'll just do the type name to show that you can do that. And then if I were to look at the variables that are set up, you can see here that this service that I just assigned is a fully fledged class that was generated by Guidepad from the definition of what it means to be a service. Like a service container from OOP. Yes. Yep. Yep. So here we go. We get all the attributes that are defined on a service,

49:07 all that good stuff. But the other thing you can do with a type once you have it, right, is you can do CRUD operations on it. So I wanna find the Node JS service that I made earlier so that we can play with it. So what I'm gonna do is I'm gonna say Node JS is service.listsingle. And I don't remember what the name of that thing is. Actually, let me just get all of them and then I'll just find it in the list because I don't I'm getting I turned 40 this year, so short term memory is no longer one of

49:36 my my strong my strong suits. Yeah. I hear that. So I'm just gonna grab them all, and then we can take a peek and figure out which one I actually want. Why don't you why don't you refresh? Oh, you know what? It's because I'm in the wrong one. No. I'm not. Oh, there it is. Yep. Great. I guess it just took a second. So, let the the the let's see here. You. You. Perfect. It's not coming back as the right service, though, so that's a little troubling. Right? Like, it should be coming back as the,

50:19 as, like, the class. Right? So you can see the classes here, like, next to it. Right? Yep. And it should be coming back as a Node JS, like, executor service, but it's coming back as, a bare service. So I need to figure out, like, why that is happening. It might be I'll just I'll just verify that. So let's see. Node. Js is services minus one, and then Node. Js Wasn't fast enough. Yeah. It's coming back as a service. That's strange. Well, it doesn't actually hurt us too much as long as when we, like, do the

50:58 deployment and stuff, it's, like, coming back to the right type. But it's strange you know what? What usually happens when this occurs is that Guidepad doesn't have it doesn't know what that type of thing is yet, and so it just returns the default type. Right? So you can see here, it was able to find the specific types for these, and so it returned them. But, this is just a bare service. And, you know, this one in particular, that we're looking at now, like, can't find the the subtype for it. So it just returns, like, the the normal type so that

51:32 it doesn't, you know, completely throw up. That's okay. We can we can still work with this. So, going back to our diagram, like, have this service. Right? We have a service. We need the operation, and we need the implementation, and we need a service exposure to to point to it. So let's let's work on those let's work on those pieces. So let's pull in an operation. So we said we wanna do a Node. Js operation. So we'll just import that real quick from Guidepad dot operations built in language exec Node. Js Node. Js operation. I could do this or I could do

52:18 I'm gonna make sure I capitalize it correctly. As long as it's in Guidepad's sort of like global context, it should come back as the right thing. Yep. Yep. We got it. There. Oh, no. It's a string. String. Yeah. Yeah. Boo. Okay. Here, let's fix it. So I think what happened is it's just not in, the system, like, scope. So what Guidepad does is when it when it does the initialize, it pulls all the types that have been persisted to the data store and then also it sort of merges that information with what is defined, like, within the code.

52:53 So what that means that come came back as a string is it's either not been persisted into Guidepad's, like, centralized data storage or it's not imported in the system level packages the Guidepad maintains to, like, make sure everything's in scope, like, when when it starts up. So let let's just make sure it's in scope, then I'll refresh this, that should return what I want it to return. So we just have this thing, system types. It's where everybody lives. Very large. So I'm guessing yep. Just gonna send it here. And would that be why we got a

53:28 generic service type for the Node. J executor? Probably. And that one is a little like, that shouldn't be important here because it's defined in a plug in. Right? So the plug in should be loading it for us. But the reason the fact that it's not and, actually, thank you. Just talking through it, I now know why it's not doing that. It's because it didn't do this. So I said there's a plugin entry point, right, that, like, sort of exports everything that the plug in defines, but I did not I did not do that for my my

54:00 new my new shiny toy. So, that's why it's not there. And you can see I was I was doing a dog walker demo before this. So, that's why that stuff's in there. Let's see. Oh, it's demo node. Yes. Perfect. Okay. So I'm gonna save that. We're gonna save this over here once I get built in language exec node node. Perfect. Okay. So having done that, now I think I can just, like, restart the kernel and rerun all this good stuff, and then it should come back the right way. Yep. Yeah. So everything's everything's happy. Right? Like, you

54:55 can find all the stuff now, and everything's the right everything's the right type of thing. So what wanna do is wanna make an instance of this operation. Right? And the Node. Js operation, if you wanna know, like, what attributes it takes, you can always find that defined on the type itself. Let's do keys. And these are all the attributes. If you do items instead, it comes back names and attribute types, and they can introspect all the attribute types to, like, find out what they are if you want to. So what we're gonna do is we're gonna

55:01 Creating and Linking Operation & Service Exposure

55:34 make an instance of this of this operation. Right? And I believe all we need to do is say my node j s op is this, and the name is gonna be a demo Node JS operation. And let me just I'm just gonna verify that, like, there aren't any, Of course, it does that. Yeah. I mean, it's very straightforward. There's, like, nothing crazy I need to set in here. Same thing at this level. It's just this, but that's actually set. Okay. Great. So go back to here and then that. And then the only other keyword is exec support

56:26 provided by, like, Citibank or, I don't know, Publix. No. It's provided by the, the Node. Js, executor that we got up there. So all this is doing is telling to this operation that, hey, when you try to execute, if you don't see that there is one of these types of things because remember this is a type. I should have, like, named it better. Oh, no. Sorry. That's the service. It's not the type. Let's get the type instead. And we call that demo demo JS executor. That's right. You gonna update for me? Oh, yeah. Okay. That's good.

57:25 Perfect. So then we can put that in here, and then do that. And so now we have this operation. You can inspect it down here below. It's active. It doesn't run-in async mode. That's okay. I'll talk about async mode later if you want to. It's got a bunch of options for, like, you know, things that happen in the API, like batch, or you can disable authorization if this is just like a bare, thing that anybody can hit. By default, Guidepad puts all the operations behind our back on all of the APIs that it that it

58:00 uses. And so there's a whole way to, like, link operations to roles and then roles to users, and then that's how you can control access to them. Sorry. I'm just going I'm just going through here, making sure we have everything we need. Provided by is the thing that we wanna set. Right? So this is a link. If you look at the it's a reference collection. And the reference to it, if I go and and show you the actual, like, type for this, if I go Node. Js executor, and then we wanna look at, provided by.

58:41 You can see oh, you know what? That's an operation. Node. Js operation. There we go. Thanks. It's a reference. What's what is it a reference to? Oh my god. Reference object. Shush. Why why do you like this? Nope. It's a one to one. My bad. I'm just gonna go to the definition. It should be there. It should be there. Okay. Yeah. Reference type. I don't know why it's being so You typed a you referenced. Oh, did I type did I type a? Yeah. There we go. Perfect. Okay. So, yeah. So, you can see before, like,

59:32 in the diagram, right, like, we want to link this operation to a surface exposure. Well, how are gonna do that? We're gonna do that through the provided by attribute because it's a reference to a surface exposure. So the next thing we're gonna do is we're going to import the service exposure type. So, exposure. And then, so we have this thing and we wanna make a surface exposure. So, what are we gonna do? We're going to say s as a surface exposure. And if I show you the surface exposure class, again, it's defined like, that's the interesting one

1:00:10 of the interesting things about Guidepad is, like, you can define types in code. You can define them strictly in data or you can do a hybrid thing where some of it is in code and then you, like, add stuff on from data storage. But for all of, like, the core Guidepad types, we just implement them as code, like, in, you know, in the library. Right? So this is a service exposure here. The things that we're interested in on this are the exposed service, which we're gonna link to the executor that we created. We're going to

1:00:41 not care about the public interface ID or the suffix or the ports, but we are gonna care about the protocol. Right? Because that's the thing that knows how to communicate with the Node. Js executor that we have on hand. We wanna make sure it's active. And the other stuff doesn't doesn't matter a whole bunch. Great. So, so we're gonna make one. We're gonna call this the demo demo node JS, executor exposure. And the protocol is going to be sorry. Cube exec without a minus. So, this is not a standard protocol. Right? Obviously. This is a protocol that we came up

1:01:30 with so that our operations can talk to a pod, basically, running in Kubernetes. But what that means, though, is that any service that you want to use this particular protocol we implemented, you you can as long as you have Guidepad. So we're gonna do that. We're gonna say the exposed service is our Node. Js service. Let me just make sure I haven't overwritten that with anything. Nope. Still the right service. Great. And do that. And so, now, these objects just exist in memory. Right? Like, they're not persisted anywhere. Guidepad, like, has no, like, notion of them.

1:02:13 They're just locally on my on my machine. So what we need to do is we need to save them so that they get persistent to Guidepads, data storage, and that we can use them in, like, downstream stuff. So, we're gonna do Node. Js, save, and s save. And there's a lot that goes on when I do this, so I I don't wanna I don't want to What's that? Do you not need to save the operation as well? We will in a moment. Haven't linked those things together yet, so that's why I'm not saving them. I'm

1:02:49 just saving these two, for now. So it's gonna take a minute because the Node well, I should say that we're connected to a, a very small MongoDB instance. Very small, meaning an eighth of a CPU. So the fact that it works for me at all is wonderful. Yeah. Anyway, so those are saved now. So, if I wanted to, like, fetch them, right, I I could. I could just do, you know, my Node. Js. Oh, you know what? No. I don't know why I saved Node. Js. Oh, you know why I saved Node. Js? Yeah. Because now

1:03:28 it's exposed. I didn't need to do that. I've already fetched this, but now I can fetch the exposure if I wanted to. So, it's s is service exposure. So, I just wanna make sure that, like, it's saved properly, you know, so it's service exposure list. And it's a special by name. Oh, sorry. List single. List list without the single on it is for many things. It returns a generator. I I just want a single one. So you can see it down here. This is the exposure we just created. You can see its name, its protocol, cubics

1:04:05 deck. It's attached to the right service. We can see that if we go through here and here. And the name is that executor that we have. So, now we have completed this step and linked it to this. Now, we need to make our operation complete, and then, we should be almost ready to actually, like, run the operation. Alright. So and and, obviously, I'm I'm taking a little bit longer than normal because I I wanna make sure this all makes sense. Yeah. I appreciate it. Yeah. Yeah. Okay. So you have the Node JS operation. We wanna

1:04:47 link it to the service exposure so that it can handle the requests on our on our behalf. So we just do Node JS op provided by is s. Okay. And then if I go to the Node JS op, we can now see that the provided by is linked to that service exposure, so I can go ahead and save it. Okay. Cool. So does that service exposure have an endpoint we can curl? Like, how do we interact with that service exposure now? So you won't actually need to. Right? Because because the thing you wanna op you wanna

1:05:31 interface with is your operation. So this will have an endpoint. This will not. Okay. Okay. Because I haven't I haven't, like, actually attached this to a public interface anywhere. So it's sort of just like an internal exposure that Guidepad can use. Alright. Okay. So the service exposure is for the executor and then the operation is what's alright. Okay. This is what everybody wants to use. Right? You want this portable thing that can run, yeah, some node code. Yep. Okay. So we got all those bits and pieces. Let's pretend that I hadn't deployed the, the executor.

1:05:58 Triggering the Build State Plan (Using CLI)

1:06:06 Right? So let's let's build it and then, like, deploy it, just so we can see the magic happen. So what we wanna do is I'm gonna use the CLI for this. And I know, David, we talked about, like, me working on the the cool text UI. That's still something I'm working on, but I haven't gotten there yet. So we're I was gonna ask about that, but, you know No. It's not ready. It's not ready for prime time. Maybe I can preview it, but, yeah, we we still have to deal with the the multiple connects every time we run a

1:06:39 command, unfortunately. At least in this I mean, that's that's sort of why I work over here. Right? So that you only have to do, like, one initialize and you can go nuts. Service enter state. So, we're gonna use Guidepad CLI to have the service enter a particular state. What state do you wanna have it enter? You wanna have it enter the built state because we need to build the image that the executor is gonna use when it runs. So, I'm just gonna pull the, sorry. I'm gonna pull the name from here again so I don't have to, like, type

1:07:13 it. Great. Why'd you put floats around it? Okay. So, you need the name of the service, you need the name of the environment, that's just Rawkode. And then you need the name of the, the state that you want it to go into. So, if I hit enter, it's gonna connect to Guidepad. It's gonna look up all the state plans that this service has. It's going to look at the environment and be like, state plans do you have that are compatible with the environment that you selected? Well, there's this one. It is the build Node. Js docker image,

1:07:53 and it will put it in the build state. Do I wanna proceed? Absolutely. So I'm gonna hit yes, and then it's gonna wait a tick. So you can see it's actually, like, running the state plan here. Cool? Nice. That did not look right, though. That was way too fast. That was way too fast to build an image. So I'm guessing something happened and I have a good idea of what went wrong. Is that somehow the state plan for deploy has the build state attached to it. So this was actually the the deploy plan because you can see

1:08:15 Debugging Build Process (Incorrect Environment)

1:08:34 yeah. It just, like, did something super super fast. So let's take a look. We can peek under the hood and see what happened because there would be something new in here. I need to delete those. Why haven't I deleted those yet? Yeah. See, like, nothing like, nothing happened. Right? So something really funky went on here. So does that mean we have to look at the state plan? It does. Yeah. It means we need to to debug what's actually happening here. So you can see it's using this stage. Yep. Right? Oh, user error. It's not that environment.

1:09:22 That environment is a Kubernetes cluster. So, of course, it's going to try to, like, run these Ansible commands against Kubernetes. I want the build environment, not the like, not the production environment. Right? I need to build this image in Packer. So I need to use a different environment name. We'll go back over here. It's this one. Packer build. Yes. Guidepad default build. So, we're gonna do that instead. It's gonna do the connection stuff. It's gonna ask me again. I'm gonna hit yes. Okay. Great. We're not seeing Ansible, which is a great sign because that means it's doing something

1:10:21 funkier, which I'm very I'm both proud of and not proud of what it's doing right now. So, here, what it's doing is this. It's gonna be making a pod over here in a second. I hope so. I sure hope so. Let's check out the let's check out the manager and see what it's doing. Would it be in a different namespace? It shouldn't. It should know better than that. That's not that's not how I brought it up. Oh, yeah. See, it's it's going now. So, it should be it should be appearing momentarily. Yes. Perfect. Okay. So, it's here. Seconds ago.

1:11:08 Cool. Okay. So, what happened there is Guidepad ran a state plan. It said, hey, I wanna run the state plan against the Guidepad build environment. The Guidepad build environment has what we like, each environment has what we call a control plane, and that's the thing that takes the state plan instructions and, like, executes it against that environment type. For the packer build environment, there is a packer control plane. And what that does is it actually uses Guidepad's work plan management system to schedule a pod that knows how to build the thing using the instructions in the

1:11:44 state plan. So that's what's happening here. Right? Like, we're using our internal automation tool to automatically run and manage a build of an image using Docker and Docker. Like, they have to use Docker and Docker image to do that, which is its own can of worms. But we we take care of that can of worms for you, so you don't have to do that. And then if you look at the logs, you can probably see what packer's been up to. Unless I have debugging turned off, I might. Yep. All done. So this built a new image, and if

1:12:18 I go over the other thing, you can see that it it said it was successful. Right? So now we've we've built an image, for this, for the service. And if I wanna look at that image remember, have the notion of artifacts. Right? So artifacts are things that are file like. Well, we considered Docker images to be file like objects, so we put them in our in our artifact storage. So if we were to take the artifact type and get all my artifacts you know what? Let me yeah. I'll I'll just do a filter on it. Okay. Artifacts,

1:12:57 docker images. Nope. Right in the middle. Yep. List. The generator. I have to, like, make it real. Perfect. So, you see there's three in there. Right? There is the default image for Guidepad's API. There is the default image for the user interface. And then there is this one. And this one is the artifact that was built for the demo JS executor. Right? Alright. And you can see it was last uploaded just now. It's in this storage back end, which is the Uh-oh. Sorry. It doesn't have a storage back end because I'll I'll spare you the gory details. We

1:13:55 can see this is where this is where it lives. Right? It's in our UCR. Yep. So, having built the image, right, like, we can deploy it. So now I wanna do I want I do wanna use that other environment. I wanna go into the deployed state. So this is what we saw earlier, and that was why it was so sort of concerning for me Yeah. Because now it's deploying the thing. So if we go back into the into the cluster, you can see the old one's terminating Yep. Down there. And we'll just let it come back up.

1:14:54 And then I'll just show you that it's, like, using the right image just as, a a sanity check. Right? And then we can also see that it's node because that was the idea. Right? That we built a, like, a node image, right, like, the fly. So I should be able to get in there and run the node binary. So we'll do that too. Are you serious? Yeah. I'm so oh my god. I swear I'm confident. I promise. I promise. Sorry. You've had to do a lot of typing in front of a camera, which is never an easy task.

1:15:36 Well, that's very generous of you to say. I I I appreciate that. Okay. So let's just, like, make sure it's not doing anything because I'm pretty sure it just sleeps when it starts up. So, yeah, nothing going on. Great. Let's describe it. You can see here oh, let's look down here at the bottom. Yep. So, here's that image we made. Yep. Great. And if I get into it, does it have node? No. Wonderful. Maybe it's gonna maybe, maybe, node? Nope. Okay. Well, which image was it made off of? That would be good to know. Okay. So

1:16:34 Debugging Built Image (Incorrect Image)

1:16:34 here's what we're gonna do. We're going to run the operation that built this thing locally so that we can watch it happen, and then it's going to tell us which image it used as the source. So maybe what's happening is that it's using a different image that I'm trying to tell it to use. Actually, I know that's what's happening because right before this call, I fixed the bug and I did not push the bug fix. You get push and fix these. You shouldn't. Yeah. I know. I know. Well, the thing is I I haven't set up a CICD pipeline for you

1:17:09 yet, because your thing's only a day old. And I I don't know. I quote unquote grew up before a lot of the the modern, like, CICD stuff. So when I'm doing things myself, I like to have a manual touch sometimes. But this this is my fault. I I will I will own this, this this problem and we'll fix it. So, what happened is the thing, built the wrong image. Right? And it's because in the version of the code that your instance is using, it doesn't respect the requirements that I'm putting on the stage entity. So you can say,

1:17:48 hey, stage, when you are run, I want you to use this requirement. So what I did is I used that harness, to attach an image requirement to the build process so that it uses the node the node image. But the code that you have in your instance isn't respecting that. Right? So I'll just run it locally, and then we should be be good to go. So the way I'm gonna do that is I'm actually gonna take the, operation. So if you remember, this thing that ran oh, this isn't the one that's gone. But it was running an operation asynchronously.

1:18:07 Local Build and Redeployment (Fixing the image)

1:18:24 Right? So it was running a particular, like, build operation. I can take the build operation instructions from this pod and then just run them locally using Guidepad. It'll run the same operation locally using the code that I have. So I'm gonna do that. I'm just gonna pull this over here so that you don't see some stuff that you're not supposed to see as far as credentials and things because I can't guarantee this pod doesn't have them in it. Give me just a moment. Yeah. Of course. K. Great. Okay. Yeah. We're gonna have to maybe, scrub some stuff out of this later.

1:19:24 If you wanna make a note of the timestamp. I think I was. Okay. Thanks. So, I thought I'd be able to make it the whole the whole call without doing it. And I I almost got there. Almost got there. Anyway, so this is the operation that the that the work plan was running. Right? So I can run it locally myself just by, like, grabbing the, the stuff. So I'm gonna do that now. And I'll put it in log level debug so we can see it do the right thing. So you can see it's actually, like, using

1:20:16 packer, right, to build the image. And here, it's using node, which is what we wanted to do up at the up at the top here. And that's gonna do the push and all that good stuff. Alright. And we're good. So, now we're gonna redeploy the service. And it should use the right image and I should be able to run node on it. Cool. Okay. So let's try that again. Get pods. Let's wait for that to come back up. Just add a get dash w to your command. Oh, thank you. Thanks. And we'll just wait for it to

1:21:32 You can see how much I'm usually, like, in another thing and not not in here. Appreciate that. Germany, and there we go. Awesome. K. Right image. Move a trif. Yay. Excellent. Okay. Great. So we have the executor out there. It's it exists. Control d. Thank you. Okay. So exists. We can now because we have the let's go back to our our diagram. Right? We've got the operation. We've got the surface exposure. We've got the Node. Js executor deployed using a node image. We don't have this. So let's make that, and then we can probably try to run the operation and then

1:22:45 debug its inevitable failure, for the rest of the for the rest of the call. So where do wanna do that? Me go back over here. We have the class. I'm pretty sure. Yep. We have the Node. Js operation type, and we have the op itself. So one thing that operations have is sort of a, like, a helper. Right? If I go to the operation class, is it had they all have this method on them, which is register implementation. And they also have register text implementation. And those two those those two methods take care of all the legwork of creating that

1:22:52 Registering Operation Code Implementation (Artifact)

1:23:31 artifact that I was referencing in the, diagram. So because we have an operation instance, I can just call this method on it and provide it some text, and a like a dumb like a dummy function name and a dummy file name, and then it's gonna go make the artifact, and then we should be able to call the operation. Okay. So, we'll do Node. Js op. Actually sorry. Command. Yeah. We're just gonna do console. Log. Yeah. And then we're gonna say Node. Js Node. Js op register text implementation command. And then the function name is gonna be

1:24:21 hello, and the file name is going to be hello dot j s. And I don't think I need to provide a storage back end because it should just use the default one configured in the system. Let's test out that assumption, though, and hit enter. Hey, there you go. So now we have an artifact, that should contain that, our little function in it, basically. So now the moment of truth. Right? So we have the operation. We could just call it and then see what explodes. So I think we should do that. Some confidence. I like to I like to set my ex. I'm

1:24:51 Attempting to Run Operation

1:25:05 never disappointed. Right? Because Moe is like, it's gonna be a disaster. And then if it works, great. If not, I assumed it was gonna be a disaster. So yeah. Anyway, here, let's hit go. Nope. Didn't like it. Okay. So let's reconfigure some things and get it to go. Messaging environment? I do need to make a public interface. Right. That's that's the part that is is missing. So, I was wrong in my diagram. You do have to map this to a public interface. And it needs to be the public interface for your environment that you're in. Right? So,

1:25:51 we're gonna have a k eight s public interface here. And the reason we need to do that is that this operation and the service exposure need to know, like, needs to be able to get at how do I, like, actually run something on this pod back here. The way that you know how to do that is through your kube config file. Right? Because that contains, the service address and your credentials and all that good stuff. And we happen to have one attached to this environment. Because every time you create a Kubernetes environment in Guidepad, it creates a, you know, service account, attaches

1:26:29 the creds to it, so that that service account can only manage the stuff, like, in your the namespace attached to that environment, basically. Excuse me. So, yeah. So this is gonna be like this. And what that does here is it, like, gets credentials and, connection details. Oops. Okay. Yeah. And let's, let's drag this down here a little bit so you can, like, read that. Okay. But it's, you know, it's this. It's from this environment. So, we'll do that. Great. Okay. Actually, this lives in here. This lives there. Beautiful. Alright. So, the only step we missed was

1:27:24 setting up the public interface on the service exposure. It's just another attribute, so we can do that pretty easily. So, if I go through here and I pull in let's go back to where we made the service exposure. You know what? Let me just, I'll just do it down here. Think it's still bound to s. Right? Yeah. It is. So I'm gonna remove this. I'm gonna pull in the environment real quick. So, environment. And then code. Good. Good. You can see it's a Kubernetes environment, so it should have all the stuff that we need. It should also have a public interface

1:28:27 attached to it, so let's just double check that really quick. Here we go. Yep. Public interface. So, what we're gonna do is we're gonna take that public interface, we're gonna link it to the service exposure so that that function I just ran will work. So, we've got s, then we have public interface ID. I'm gonna say Rawkode public interface first ID. And I'll just double check that it is still down to s. It is. Perfect. And then we'll do s dot save. The nice thing about you doing this is the Jupyter Notebook is now you've got everything

1:29:11 saved as a re artifact of how you've done it. Exactly. And I can show this to people on my team. I can give it to other folks. I can revisit this when someone's like, how did you do that? Yeah. And we try to, as much as possible, like, encourage our team to to do this. I mean, that when we we have a machine learning engineer write a bunch of blog posts and he would just do all his work here. Right? And then put the markdown for the blog post, like, in line with it. Mhmm. It's very, very convenient.

1:29:42 So yeah. Okay. So we have everything linked up. That particular error shouldn't happen anymore. Let's call the operation again, and let's actually, like, save the result too. Okay. Okay. I'm gonna be like that. I get it. Public interface, surface exposure, public interface ID. That's not right. So the surface exposure should have that set. Yep. And it should be doing that there. So, let me just make sure. Maybe it didn't get set properly. No. That's set. And if I look for the public interface, Gosh. Sorry. Interface. Interface. I just wanna make sure, like, I can find it myself.

1:30:06 Debugging Public Interface Link

1:30:49 Oh, and of course, it's gone now. One thing I don't like about it is that the copy key bindings, like, don't work for stuff like in here. So, you have to select and then, like, copy and then paste. Kind of annoying. Okay. So we can find it when it does it here, and it's set properly on s. Maybe maybe it didn't take. So let's pull it back out. Let's inspect it. Nope. It's there. So we need do, like, some more digging. Right? We need to, like, figure out where somewhere in this somewhere in this call chain, it's, like, not

1:31:57 getting the public interface ID. Okay. So let's go to the trace back. It's here in kube exec. So this is the custom protocol implementation for the KubeExec protocol that we did. That's the whole backbone of this particular, like, running stuff against a something deployed in Kubernetes. So if we go to KubeExec protocol, we can see it's on line 93, which is right here. No. Not there. I see. It's calling handle, cube exact exposure, and then it's actually in handle cube mic. Okay. Got it. Yeah. Okay. Public interface first. Oh, okay. Well, it's saying you can't find that.

1:32:56 You know, keep this exposure. Usages. Okay. And so that happens in the operation. Yep. Sorry. I'm not sure if this is interesting or not. I'm just, so this is this is sort of how you're getting to see how the magic happens. Right? So, when I call this operation, right, like, when I do this, I'm basically doing the call, like, I'm I'm executing underscore underscore call on an operation, like, instance. Right? It's an instance of the operation class. And so, inside the operation, there's an execute, an execute method, here. And you can see this is another interesting

1:34:03 tidbit. It usually emits an event every time it runs. And so this helps us be a little bit event driven. This is how we can, like, wire things up to, like, when stuff happens. Because you can set a requirement on things, like your work plans and your state plans that are conditional on a particular event happening. So those events could include, like, an arbitrary operation firing, right, or returning with a success or with a failure, or those things. And we we might see that if we have another session where we go into some other details.

1:34:31 But, anyway, when you try to do execute though, up in here, it will do a couple things to figure out like, oh, hey. Like, are you asynchronous? Yes? No? Oh, I'm not asynchronous. I'm just gonna do execute. And then inside execute, what's gonna happen is it tries to find the implementation, and the implementation, could be one of three things. Right? It could be either, a surface exposure, which is what we're doing. Right? Or it could be, handled in some remote artifact and you download it and then compile the artifact and then run it in the artifact or run it

1:35:10 from the artifact or you're just running a local handler, which is what happens down here. So we're up in the first case, right, which is we need to we're trying to execute this operation through a service exposure, so we build a partial function and return that. And then we attach the target service exposure as the first thing. So the issue is you can see it's trying to use provided by first, and what we've done is we've saved the public interface on the service exposure, but we haven't refreshed the database definition of the operation. Right. Right? So the operation is stale, and

1:35:51 so its service exposure pointer doesn't have the public interface ID that we just set. So one way to solve this within Guidepad would be to have something like an instance map where at the process level, we have a map of all the objects that are sort of in scope, and then operations on one of them would change it for everything that has a reference to it. I haven't implemented that yet, but it's on my sort of, like, internal mat road map to do that one day. Anyway, so what what that means we have to do is just, like, we just have

1:36:19 to reload the we can either reload the operation, which will fetch the proper which will fetch the proper, service exposure details from the database or I could just, like, reassign it. We'll we'll do the former, and just fetch it again from the database. So or fetch it for the first time because we never we never fetched it after creating it. So, let's do that. So, we'll do that here. You know what? Let me insert. I know there's an insert above. I think it's a right click operation. Yep. No. No. Fair enough. Can I drag you?

1:37:06 You can also see, like, I I don't do this too much either. So let's cut let's cut you, paste you, and then we're going to take this guy, and we're going to fetch it again. Yeah. Perfect. Okay. So that's done here. And then we're gonna do a list single instead of this creation. I know I didn't need to do that, but whatever. Good. Good. Good. Okay. Great. And so let's take a peek at that service exposure thing just to make sure that it's wired all wired up properly. Should be. Provided by provided by First. Yep.

1:37:57 And then, there we go. Good oof. Should be good now. Okay. Let's do it again and see where we get. It's it's doing stuff. Stuff that's happening. Okay. It returned and What should I say? Oh, hey. Hey. It works. Great. So, yeah. Okay. So now you could put arbitrary JavaScript inside that operation. Right? And it would run inside the executor just by, like, reregistering the implementation. You could also so, like, you know, here. Right? Like, you could register a different handler. Right? And then rerun it. So, like, let's do it. You know? Let's do hi

1:38:46 Demonstrating Live Code Changes

1:38:54 instead of hello. It's not optimized, so it's not the fastest thing in the world, but Very nice. Yes. So, yeah. Should I re go over sort of, like, what's happening there? Or, like, what do you what what are you interested in in taking away from that? I mean, I think I need a nap after that. Okay. Lucky I mean, I have to go back to work, but I have a nap too. No. I think the way that you've walked through that really helps solidify all the ideas. You know, I think if people are watching this, right, and

1:39:24 Recap and Conclusion

1:39:43 you've stayed for nearly two hours to understand how this works, what they're gonna take away is everything is a type. And I I love the way that Jupyter notebook reveals that as well. You've been pulling things out of the registry. You've been using the methods for less less single, even creating some resources. All of this is just stored as a as as like JSON, right, and your your backing store and register Yeah. You wanna see it? This is you. This is your this is your instance. Yep. It's MongoDB. But, if we, take a peek at this and I know

1:40:19 you said you needed an app, I apologize for, like, dragging this on. But if I do this, so from Guidepad import datastore. Right? And do datastore, session datastores, datastores by name. It's not just the one. Right? There's three in play here. There's one that stores your secrets encrypted at rest. There's one that stores all that data. That's what this is what we were just looking at. Right? And then there's this one which we haven't looked at, but it stores only two types in there right now, log and an event. Those two types go in there.

1:40:54 But that's all transparent. Right? So, like, if I were to import the log type and then do log dot list, it would go out to that data store. But you wouldn't know that. Right? Unless you cared about it, in which case you wouldn't know about it. But yeah. So this is all all of it stored in here. Right? All of those things that we were fetching. Same thing that showed on the user interface, same thing that we interfaced with on the on the command line. Yeah. I I just think this is super cool the way it's all the keywords is

1:41:23 what you said earlier. Right? It's composition. And all these types come together to provide a very I don't wanna say a cohesive system. Right? But, you know, whatever system you're trying to build, you can architect those types to do the things that you want. And there there's so many things we could dive into. I mean, even the event driven stuff with the annotation on the function was really cool, but I think what we need to do is schedule a follow-up Sure. Where we can dive into some of these concepts in more detail, build something else, and just keep showing people

1:41:54 the power of the Guidepad the Guidepad framework because there's a lot to digest here. So I'm gonna say let's leave it there for now. Okay. And a follow-up scheduled. And, hopefully, you know, if you've watched us, please leave comments in the comment section. Let us know what you want us to build, what you want us to dive deeper into, and we'll make sure that we have a follow-up where we go into this in much more detail. I'm gonna pop over just to our big face mode now so we can Sure. Cool. Up. Just because I I also wanna get home

1:41:58 Final Thoughts and Farewell

1:42:22 to do my daughter's bedtime. So that Oh, yeah. Makes sense. But that was fantastic. Thank you so much, Matthew. Like Thanks for having me. And thanks for many times. But What's that? You can I've said this so many times to you, to Reggie, and and on previous videos. Right? The things it's really as limitless, the things you can do with Guidepad. Like, your own imagination is the only limitation. And I hope what people take away from that is just, like, understanding those atomic types, the primitives, how you can pose them together. You really can build anything on this. So,

1:42:54 you know, go to Guidepad.io, check out the website, and get in touch with the team and, yeah, make some awesome things together. Well, yeah, thanks for thanks for having us. It's been a been a pleasure. Thanks for putting up with my my mind. I mean, because Guidepad is is like it's my brain. So thanks for thanks for living inside it for two hours. I appreciate that. Definitely my pleasure. Was a whole lot of fun. Any last words before we we say goodbye to people? Oh, I think I've said enough. But looking forward to looking forward to the next thing

1:43:24 we get to do. Yeah. I had fun. So, again Me too. Thanks for having me. Yep. Alright. To everyone watching, thank you for joining us. And remember to get your comments into the comment section. Matthew and I will definitely be following up again soon. And, Matthew, thank you again for walking us through that. I'll speak to you shortly. To everyone else, have a great evening. Take care. Thank you for watching Rawkode Live.