1:47 Introduction & Welcome

1:47 Hello. I am off center. Sorry. I was talking to Ben behind the scenes. Hello, all. Welcome back to the there. Hello, all. Welcome back to Rawkode Academy. Today's session, we are taking a look at Teleport nine, which has just been released. You may have seen the release announcement yesterday. Two days ago? Yesterday. Yeah. Yesterday. With some awesome new features. So we thought we would jump onto a session and take a look at the new features and also do a bit of a q and a. So if you have any questions, please feel free to get involved, drop them into the comments, and

2:21 we'll tackle them. Well, I won't tackle them. I'll do my best. But we are joined today, fortunately, by, to the loo, Ben Abbott. Hey, man. How's it going? It's good. Thanks for having me, David. I know. I'm I'm gonna be tweaking my position trying to make sure that I'm I'm centered. I know. So thank you for joining us again. This is not our our first stream together. We always have a lot of fun playing with Teleport together, and I'm sure today will be no exception. For the people that have not seen you before on the the livestream here, can you

2:53 just tell us a little bit about Ben? Yeah. My name is Ben Arendt. I'm a developer relations manager at Teleport, and I've been in the developer tool space, I think, almost like a decade now in various roles, spanning UX product and developer relations. And so I've sort of touched everything. I also ran a Redis as a service product. And so I'm pretty excited about our Redis support since when we had Redis to go, many people would find their passwords in GitHub. And so which is a common thing. No. There's nothing too bad in your Redis database,

2:57 Ben Aaron

3:32 but it's not something you wanna expose. And so, yeah, I'm excited to demo Teleport nine, our machine ID, and the long tail of other things we've added. Awesome. Thank you for sharing. It feels like there's just a new release of Teleport every other week. And of course there's not a new release every other week, but for a company and a team, user moving really, really fast. It's just so exciting to see all these new features coming along. I think I've completely just bought into the Teleport eFalls now and I just use it constantly. And the fact that we're getting new features

4:07 on a regular is just great. Now, so that I don't mess it up, what is the official release cadence? Do you have one Or is it just when something's cool is built, we're gonna ship it? No. I think we've moved to a three month release cycle. Three month. And we also made our roadmap public. And so you can go to our docs and see what our roadmap is for the next three months. We also changed our semantic versioning last year to sort of represent more breaking changes between the different versions and version compatibility. There was sort of a time in which

4:27 Semantic Versioning

4:40 we wouldn't like, we would ship lots of features, but we wouldn't change the version or our version wasn't very clear. So another thing since we're, an open call company, we have a request for discussion. And so you can go to our Git repo and sort of understand, like, what our, like, versioning strategy is and then also look at our road map. And the one feedback we've got is, like, it's great that you're releasing all these features, but upgrades can be difficult, especially for, like, our large customers. And that is something that is going to be added in,

5:09 I believe, Teleport 10, maybe 11. But there's a team actively working on sort of automatic upgrades, which is an interesting, you know, computer science problem in itself. Yeah. Definitely. Well, I think I've been using Teleport and Cluster now for, like, eighteen months. In fact, starting with Teleport five, and we're now at Teleport nine. And it's always auto installed the latest version, but at no point has it ever broken my automation. Like, we always just seem to get a new version of Teleport, and it just works. Like, I mean, half their backwards compatibility breaks, and I've just been very lucky?

5:43 Or, like I think it's mostly some of our customers who may be on four or five. Alright. Okay. You know, also because we have a whole, you know, large suite of customers from, like, open source community to sort of, like, Fortune five hundreds. You know, we've still supports like, we supported Center West 6 longer than Center West 6 was sort of supported by Red Hat. And so we generally try to, like, meet our customers where they are. And you might see sometimes in our releases, we do do back patches to I think you're you're going

6:18 back to, like, six for both performance and security. And you can see that in our, like, semantic versioning we describe, like, at what point will we, like, not back port security fixes. But if it's a bad one, we will sort of go about our way to back port one. I think we backported a fix into, like, four dot something recently. So we try and support all of our customers as much as we can. That's awesome. Cool. Alright. Well, we're here today to talk about the latest release, Teleport nine, which dropped yesterday. Do you wanna give us the the highlights

6:41 Teleport 9 Feature Highlights Introduction

6:49 of the release so that we can dig into a few of them? I can pop the blog up if you wanna start on Machine ID, or do wanna start with just a high level No. Let's start on Machine ID. Okay. Go for it. So machine ID is a new addition to Teleport nine, and we sort of described the problem in this blog post. But, you know, up until Teleport nine, like Teleport nine, Teleport was mainly focused on sort of humans accessing infrastructure and being really good at that. But we would have people who would set up all

6:58 Feature: Machine ID - Problem & Concept

7:22 of their infrastructure with Teleport, you know, maybe not even have, like, open SSH. And then we want to also use the same short lived certificates for, like, Ansible runs for Jenkins. Basically, everything that also has some form, like, SSH certificates to some degree. And you could do this with Teleport. You could do, like, TCL auth exports, asserts for a short period of time. But people would just end out making these, like, five year SSH certificates, which is basically the same as like a public private key to some degree. Yep. And, you know, in Teleport, we have a CA

7:58 rotation that you can run. So if you did a CA rotation, that long lived certificate would break. And then all of that, like, other work that people like, when we had teams kinda build this in house, and so machine ID is a sort of a new addition to sort of Teleport that makes it very easy for you to issue sort of certificates and identity for machines and sort of provide that same tooling for, infrastructure tools. I imagine for people that were doing this prior to machine ID, especially when they're using one of the, you know, OpenID or back

8:33 ends and not local users, which I assume is probably a large majority of Teleport customers. But they would probably have to break out to local users for machines. Is that a fair assumption? They could. We also added impersonation. And so you could create a u like a non login user that you could impersonate, and that was sort of our recommended approach to this. But, yeah, some people would create like a local user. Because you ultimately don't want someone to be able to like physically log in as the Jenkins bot person. True. That makes a lot of sense. So

9:09 machine ID is is gonna solve this challenge for people that wanna be able to get their CI systems access to the infrastructure. Yeah. So, you know, we'll focus a lot on, like, infrastructure tooling. It's just probably our strongest thing in Teleport nine. But in nine point one, we extend the same short lived certificates for database access, Kubernetes, and applications. And as you may know, in our current configuration of Teleport for Kubernetes, you actually have to issue a very long join token because the it doesn't persist the connection. So we plan to use machine ID to issue the cube

9:52 configs and always keep your cube config and the connection between, like, Teleport and your Kubernetes cluster always updated and secure. Alright. Awesome. Oh, we we just got a question from Evanash. He's just joined and is curious if the concept of machine ID has been explained yet. So we're we're talking about it right now, Evanash. But I'll patterface that and think or write down if I got wrong. Yeah. That's a good idea. I wouldn't be able explain it twice unless I've just completely fluffed it and misunderstood that away. But, you know, the what Teleport has done really, really well today

10:26 is when humans want access to infrastructure because let's face it, humans are the security problem anyway, that they go through Teleport and authenticate as a human, and then you get federated access to servers and databases. The challenge there has been that if you want to give Jenkins, GitHub actions, or any automation access to any of these resources, which are being commoditized. I don't even know if that's the right word, commoditized or secured via Teleport. People would have to do some weird things there. Now in the past, I've just used local users and breaking out of that entire control flow

11:02 and saying this works, which is usually bad practice. Ben mentioned that there's also impersonation and Teleport, but that is then using user authentications to pretend to be another role within the Teleport ecosystem, which works well. But this new concept, machine ID, allows machines and automation processes to become first class citizens within the Teleport authentication workflow and give them access to the same resources in a secure fashion. Does that work? I think that's a great summary. Awesome. I was really terrified I was gonna make an answer to that. But we're So do you wanna tell us a little

11:40 bit about I'm curious of how machine ID works from setting up and getting started. And I we will have demos, but maybe we could talk about at a high level. How does it work into the flow that we have now? And how does it work under the covers if we're willing to kinda go into a little bit of detail there as well? Yes. So at a high level, you know, I think where where to begin for the high level? So we have a T Bot service. And so when you install Teleport, now there's like a fourth binary called T Bot.

11:57 Machine ID - How it Works (TBot, Join Methods, Roles)

12:10 And T Bot is the service that you will issue on, let's say, your Ansible control node. And you sort of enroll Tbots in the same way in which you'd enroll Teleport nodes, either through, like, an ephemeral join token or IAM token join method. I would actually recommend well, we've, like, focused most of our sort of UX around the AWS token join method, which means you don't need to share any secrets. You just give it a basically, as long as the machine has the a role identity, it can automatically join the cluster. And then you scope your access in the

12:47 same way in which you'd use Teleport roles. So you just use another role, and the role says, you can access these like, this machine can access these hosts, say, like, Jenkins hyphen star, and they can do the like, these commands. And then if you're in your application for it has some, like, handy other features. So for SSH configs, it, like, automatically creates an SSH config for you. So I'll do a Ansible example, and you'll see it just more or less works sort of out of the box. You don't need to make any of the other changes.

13:21 And let's say if you're like, when we add database support, you just sort of change your setup to use the three certificates for TLS connections as opposed to, let's say, like, the password for your Postgres database. Okay. So I didn't know that I hooked in to so that's just you can use AWS kind of workload identity to be able to speak to Teleport. Yeah. Yeah. So it's I think we added it in, like, seven, but there's a IAM join method. And so you just give you create a token that says, this token can join with this role.

13:51 Iem Join

14:01 And as long as it has the identity document from this account, you can automatically join the account. And does this does this change anything for the operator side of Teleport? Do they do I need to do anything different when deploying Teleport? Or is this just something that I get out of the box by default for free with any Teleport, like an app upgrade or an app install? Yeah. You just get it for free. You might have to change, like, the token type. But as long as you're in Amazon so, like, what people were doing is, like, let's

14:34 say it's like the big strapping your trust problem. And so Teleport has, like, a range of different tokens. So I think get the sort of least secure, you just do, like, a hard coded string, which is probably our least secure version. Then you can do a issue like t cuddle, like, nodes join or cube join. Then that's like a short lived join token. And some people would publish those tokens to, like, secret store. And then when you bootstrap the host, it would, like, go to secret store, download the join token, and then it would join.

15:08 In this way, you don't need to do any of that other kind like, machinery. You can just use the IAM join method for your notes, for example, or for T Bot in this case. Okay. What about for people that are not on AWS? Is this something that you think will be rolled out to, like, Google's version of workload identity or Azure's or whatever people on bare metal? Do they have any options to be able to look into this machine ID concept? We'll probably look into the other cloud providers. You know, GCP and Azure have the same,

15:39 like, identity documents available on the hosts. But for bare metal, you sort of have to build your own trust of what the machines are. That doesn't sound scary at all. Right? I mean, I guess in theory, you know, that, like, the pro like, historically, like, you go to the rack, you see that machine, you'd like, okay. I trust this machine. But in the cloud, you don't know necessarily that that new machine that fired up is, like, your machine or not. Yeah. That makes sense. And there's, like, another interesting project like, SOPs was a project from

16:12 Azita Yeah. For encrypting your, like, ENV. And they use the same sort of method for encrypting their tokens using a sort of a KMS key based upon, like, the identity documents. Okay. Cool. But this is like a whole rabbit hole of actually, I a webinar on, like, cool tokens, TLS, and Teleport, which is sort of deep dive into all of the different ways in which you can set up and configure tokens. And it's a fun topic, but it's it's also, like, quite day two operations. But day two happens to be all the other days after you set up Teleport, so

16:43 it is important. Okay. What are some of the the primary use cases then for people to start rolling out or using machine ID? I think infrastructure tooling. I have a good demo for Ansible and then also for Jenkins. Just these tools that you've had around in which you've had to provide them SSH certificates or access for a period of time, but you haven't rotate like, you you might have created a public private key and you haven't really rotated it that often. Yeah. And I think, you know, we've seen more supply chain attacks that have gone through CICD

16:46 Machine ID Use Cases (Ansible, Jenkins, Automation)

17:21 build servers, and I think it's sort of the area in which people are paying more attention to. So that's probably where we would start. I think the next one will be adding support for databases and sort of applications and Kubernetes clusters, but, that support is in 9.1. Alright. Awesome. Another use case is if you wanna interact with the Teleport API, you can use T bot to get a certificate that will talk to the Teleport API as well. Okay. So, I mean, I don't know what your Ansible demo is, but let me guess. And then you could you can walk us

17:53 through it. So when I think of Ansible, I think about a human typically running an Ansible playbook, which has a inventory fail, but allows it to go and speak to one or more machines. I'm assuming your demo is gonna be, well, what if I wanna automate running an Ansible playbook and I have to machine do that without the human component there? I'm assuming Ansible is gonna request some sort of ephemeral SSH token to go onto all the machines to run the playbook, Or is that just way off the mark? It's pretty close. I think the thing that's

18:25 different is it's more or less the same. And Teleport or, like, machine ID does everything for you. So you don't need to worry about certificates and access, SSH config, actually generates it all for you. So maybe just a demo would be the easiest way to kinda, like, show you. Yeah. Let's take a look. Sounds good. Okay. Share my screen. Alright. Your screen is now shared. I can see a terminal, and I can see GitHub login for Teleport. Yeah. We have this sort of diagram, which I'll kinda come back to it. But this is sort of what I'll walk

18:55 Machine ID with Ansible Demo

19:10 through. So we have Teleport deployed. We have a Ansible control node, and we have a group of sort of EC two servers that we've added that it can connect to. And like I said, this machine ID is joined using the IAM join method. And they actually these credentials for the SSH config are rotated every twenty minutes Okay. Which is sort of a cool feature that sort of happens out of the box. And there's other security features like you can lock hosts, SSO in. So much multifactor. Yep. Every time. Oh, and my password. I have a blog post called why SSO

20:03 sucks, and you can see why. This is my daily experience. Okay. So, you know, here's Teleport. You know, people are familiar. It's just, like, a list of all of the servers and hosts. You don't necessarily have to like this machine ID service has, like, Teleport and machine ID just for this sort of demo purposes to make it easy to access. And I have a Ansible playbook. So let's just so the my playbook, very simple. It just all hosts, the user, it just pings host name. So Let me just run it. Oh, no. Okay. So some of these ones haven't connected. And,

20:29 Ansible Playbook

20:56 actually, there's a reason for this is this is because my host file it means, like, it's probably this host has been cycled through. Yeah. And I no longer have access to this host. But these other ones are running. I should be coming to active sessions. You can see it's sort of like all these sessions are firing up. It's sort of going. And in our audit log, you can see that we have command execution from my Ansible bot. I think what's also what's kinda interesting is you see a like, an SCP upload of, like, how it

21:40 sort of Ansible sort of works behind the scenes. And so I guess this is sort of the auditing aspects of machine ID, but you see it more or less worked, but until I guess it didn't work for these ones for our. And so for bigger me my Ansible config. See, my Ansible config is pretty standard. I the host key is on. I have inventory of host in my dot host file. SSH connection is true. And then my argument is I pick the information from this machine ID SSH config. And we have this command that will, like,

22:06 Machine ID & Ansible Configuration Details

22:25 populate this for you. So if I You can see I have you know, this is sort of my s s like, this looks very familiar for people who have a, like, dot SSH. Yeah. And in inside of here, we have sort of the the key, the SSH certificates, SSH cert, and then we also create this SSH config for you. And this SSH config sort of does the plumbing to know, like, okay. Under the hood, it's connecting to these hosts with these identity files. It's running these commands. It's going over port thirty twenty three, and you don't need to edit this at

23:00 Ssh Config

23:20 all. So that's the SSH config. Let me just show you the hosts file. And then you see my host is just a list of all of my nodes, which are the same nodes here based on host name plus the cluster name of my cluster, which is Teleport nine. And I think some of these were disconnected. That's why the host name wasn't able to connect. But that's sort of the configuration and sort of setup. I'll just kinda, like, pause there for Yeah. I wanna make sure I your thoughts. I understand what's happening here. Right? So this

24:06 this is a AWS EC two virtual machine just running some Linux operating system. Is there a Teleport agent running on this machine? There is a Teleport agent which you know, I'm in a Teleport session, but there is also a Keybot service. Right. Okay. And so this is sort of very familiar for people who are used to sort of Teleport. You know, we have a, like, T bot start. We have a config that I can show you. It's using the IEM join method. And then you can see the renewal interval is every twenty minutes. And so in the background,

24:47 this is connecting and talking to Teleport. In each twenty minutes, it just issues new certificates, which in the background, AMS will use. Okay. If this TPO service is offline for more than twenty minutes, will it fail to rejoin? Is that what the certificate rotation means? Yeah. Alright. Okay. So as long as it's online and happy, it's gonna continue to rotate the service. Then there is the so Azure was using an SSH config. So what's the command that I need to run to be able to generate that for? There is let's go to our docs. So there is t bot init.

25:36 And init has some, yeah, also useful commands for, like, ACLs and writing permissions for which uses the owner and when to initiate it. So I think we have examples here with file permissions or with ACLs, Let's say for the Jenkins example, use if you're running it as, like, a root user, but you can also want to make it to the owner of Jenkins Jenkins, we have these instructions here, and that'll sort of configure your files correctly. And then once you run t t bot start, this will just go about and configure everything for you. I think the one other addition which actually

26:17 is probably missing here is it's based upon your t bot config. And currently, this kind so configs is SSH client. You can get TLS certificates. And as we add databases and other additions, you can just sort of edit the YAML file to get those other certificates for you. Right. Okay. I think I understand that. So I'm thinking about then if I were setting this up for myself, I spent up an EC two instance, I installed Teleport. What does that T port? Does T bot need configured besides the I'm stuff that we put this, like does that need to add join token?

26:59 Does it need to add certificate? Yes. It needs okay. So we have the first thing you need to do is add a, like, a user. And so this is, like, token based. The IAM method, you create, like, a new token, and this is kinda what I was saying. You restrict it to AWS on for access and give it a name. And then when you enroll the token, you sort of do t boss add, and then that's sort of the same flow as, t cuddle, Teleport sort of add a node. And then it's the same flow of starting

27:37 machine ID. And, yeah, then there's, like, other sort of debug commands here for making sure that you have access. Cool. Awesome. I like that. Is there anything else for tea bot? And I think I well, let's just run my we have the let's run the in the playbook again. And, actually, so you noticed in my current demo, I just have hardcoded host file. And we have instructions here on sort of how to generate the host file in our Ansible guide. In this case, we use, like, TSH to list, and then we just add this to a

28:31 host file. This should be something in which, like, we would like to make a small program that will just talk to the Teleport API to generate this host file for you, but this is sort of just a example. And this is why the host file was sort of out of date with Teleport because I've, you know, been firing up my cluster up and down. So this would be, like, another area in which you might wanna automate. The T bot daemon could just, like, load an eBPF program to do this dynamically rather than actually modifying the host file. I think that would be

29:00 pretty pretty cool. Yeah. Then that's also, the same inventory of Teleport. You know, you'd wanna also, like, you know, create different like, use our labels based for RBAC. And, actually, if you come to RBAC, they do create these, like, automatically generated rows, which will show you what labels you can get access to, which under the hood, which roles does it impersonate, and what resources does it have access to. And so it has access to the certificate authority, but it can't read any secrets. And there has been lots of work done around the security of these certificates and tokens.

29:41 And I think that's sort of one core focus that we've worked with our customers is just that if someone was to get access to a certificate, one, you have the ability to lock these bots pretty easily. You can make quotation easier. And there's sort of some other stuff behind the scenes we've done to really sort of secure it. So definitely, it's relatively easy for Ansible, but I think the most value comes from people who are very sort of conscious and concerned of these different sort of robots. Ants like, Ansible can, like, touch all of your systems. And Yeah.

30:16 If someone gets access to those, like, public private keys, you could be in some trouble. Yeah. Definitely. Like, I mean, I've lost count of the amount of times in my career that I've generated that, you know, SSH key gen. And then that just that's just been a key I've used forever for the lifetime of a project to do any automation whatsoever. And then Yeah. So we also have, like, the audit log of certificates issued, and I think we sort of ran through like, we've executed a command, like, what happened during these sort of sessions. It's just sort of useful telemetry that

30:38 Machine ID Auditing

30:54 you wouldn't necessarily otherwise get of because all the connections go through Teleport. Yeah. And that's super important because we get all of that auditing stuff that we we just need for all of these processes. Like, if we ever wanna understand if a bot has gone a bit rogue on us for whatever reason, then we have all that information for the Teleport Autologue. I think that's a really exciting addition and I think it's gonna be really important. And I'm gonna just say like the the TLDR for the people watching this, that if you have any automation that needs access to

31:25 a server or a database, then machine ID is probably the way that you wanna start building out that automation in the future. Yeah. And it's, you know, it's kinda like a primitive. You build upon it based upon how you want to configure your services. You know, you could run Tiva on the machine, or you could, like, publish certificates and pull them through a secret manager if you wanted to. You have a range of options. Does that mean I can have all of my Kubernetes components authenticate via automatically self rotating every twenty minutes x five zero nine certificates? Is that something that

31:51 Machine ID Future: Databases, Kubernetes, Apps Support (9.1)

32:00 you see and maybe not today if it's not possible or in the future? Yeah. I think that will be coming in, I think, Teleport nine one. I think the first today is just SSH, but we'll be adding database and Kubernetes support. Awesome. Very, very cool. Alright. Anything else on machine ID that you want to I think that's it. I think the blog post goes into detail. Also, if you're interested, check out our documentation. Jenkins is another tool. We have an example here of using Jenkins for your Jenkins workers, which is another example of a service that you might have around which you

32:47 haven't necessarily considered hardening how you do deal with, like, SSH. And once you sort of configure it and set it up, you know, this is an example in the Jenkins pipeline. You can you know, if there is something in your pipeline, you can just use Teleport for it. Yeah. I'm gonna have to clear that. I mean, I don't use Jenkins as much as I I used to. That's for sure. But so You're a lucky man. Yeah. I don't think everybody has that privilege. There's a bit of a mechanical buzz there. Can you if you got, like, a cable for your

33:23 mic or something, just pull it out and push it back in there. Okay. Is it still there? Alright. I think we're good. Is it still there? Oh, yeah. It's still busting. Alright. How was that? You've angered it. I made it worse? Oh, no. Hold on. Okay. Is that any better? Much better. That is clean. We're good. Okay. Perfect. Okay. Should we move on to the next features? Are there any questions? Yeah. If you are watching and you've got yeah. I'll pop this back over to face mode. Hold on. If you're watching and you're curious or have any questions about machine ID, drop

34:38 them into the comments, and Ben and I will tackle them. But there is a lot more in Teleport nine that we can take a look at. So what do you wanna tackle next? I did have a list. Okay. We've let's talk about Redis. Alright. I know you've been dying to talk about Redis, so let's talk about Redis. I mean, Salvatore is no longer working on Redis, which I'm very sad about, but many small Redis comps with Salvatore was always always fun. Do you wanna share my screen again? Yeah. Sure. Let's go. So we are talking about Redis and Teleport nine,

34:56 Feature: New Database Support (Redis, MariaDB, MS SQL)

35:26 I'm assuming Redis has now been added as a supported database so that people can use the Teleport proxy to connect and secure the access to it. Correct. Yep. Awesome. So Redis is, like, one of those interesting products. And I actually, like, hosted a RedisConf years ago, and Salvatore was always, like you know, he's quite an opinionated programmer. He's like, you shouldn't really put Redis on the Internet. It wasn't designed for that. And meaning that, you know, like, Redis should really be in its own, like, private subnet. You're like, you don't necessarily wanna, like, publicly expose

36:03 Redis, which, you know, I guess eventually people will do things with your tool that you may not necessarily expect. Oh, awesome. Have, like, basic, yeah, password or but in Redis six, they added support for TLS certificates for authentication. And just in general, like, database support on Teleport, you know, they're always, like, kinda like where your crown jewels are, but also is most likely to just be passing around a shared password, maybe for a team or a group. And it runs into that same traditional problem of if a team member leaves, like, did you rotate the password? Like, pro probably not. It's

36:42 like that one ops person who seeded the first RDS database, like, there's still a password likely on, like, his notepad somewhere that will get root access. And, you know, Teleport, you know, we started with the large, like, SQL and NoSQL databases, and then we're just sort of expanding our suite of tools. So in Teleport nine, we added Redis, Maribadb, which is, you know, MySQL variant and Microsoft SQL Server, which I haven't configured in this one because I don't wanna necessarily go through the pain of configuring Microsoft SQL Server. Alright. So you didn't just add one database

37:24 to the Teleport line. You've added Redis, Microsoft SQL Server, and MariaDB. Yep. That's cool. I know a lot of people that prefer MariaDB to buy SQL ever since Oracle had their acquisition. So it's good that the people that that could have done that route can also take advantage of these tools. And I guess some people like Microsoft SQL Server. I've never used it. I have no I couldn't tell you a single damn thing about Microsoft SQL Server, but it's cool that people can secure it. All I knew was when I looked at our docs, it required going to, like, the

37:59 UI. Or maybe not. Yeah. If if I have to hook up active directory to speak to my database, I am already broken. This is not gonna happen. Oh, yeah. When this popped up, I was like, oh, I don't know if this is for me. But it is we do have people and I'll probably configure this at some point when I when I get the time for it. But these are all of our databases, you know, we support, and we're just sort of slowly expanding them. The team is cranking like crazy. So if there's anyone that you'll

38:30 have a particular interest in, you know, you can ping me. I know we've already had requests for Redis, like the AWS version of ElastiCache, which is in process as well. And then also just a note on Redis, there's, like, two different instructions for if you're using Redis cluster or standalone Redis. The instructions don't differ too much. But once connected, you'll get a sort of, like, a list of your databases. And everything for connecting and accessing your database is actually done on the terminal. You know, we have some instructions here to sort of to tell you how to, like, log in

39:04 Database Access Demo (Redis)

39:08 and get get credentials. But we assume most database people have their own tooling either, you know, it's like the Redis CLI or it is like GUI for Postgres. And we also have instructions on how to configure those GUI. So let's log in. And then I'm using my SSH flow again. I didn't have to log in again. That's good. I think you're okay. Maybe? I think it's okay. There we are. So we're in. And, you know, you know, same sort of flow, like, for servers, we have TSHLS, TSH DBLS. One thing you do need to do is

40:06 Log In with the Database User

40:07 you need to log in with the database user, and this is almost like your principle in the world of servers. So if you would log in as like Ubuntu, it needs to be sort of baked into your certificate. You do the same thing for logging into the database. So I have a I actually think this user's different. I think it's SRE team is my name of my user. And then I can just do TSH DB connect Redis. And what's really interesting here, so this is connected. But if I do, like, ping, not authenticated. Redis does require you to also add a

40:52 password auth in addition to the TLS certificates, just the way in which it works. Don't know about that. I think most of the ones, like, there's always, a little quirk. Let's say if you configure so you have access to the database, but you also need to map the user and the database together. And so for people who are used to, like, the Kubernetes and server flow, just be aware of that extra mapping of a database user. It normally differs per database just to sort of put in your documentation. Cool. Can I ask you the hard question then?

41:31 Machine ID for Application Access / Microservices

41:31 So with the introduction of machine ID Teleport now supported more and more databases, I'm starting to have this really weird idea that even in my application code, instead of configuring it to speak directly to my database with a username and password or, you know, forever certificates that I'm never gonna rotate because I'm a terrible SRE. Does Teleport fit into that workflow or would that just be really silly? And by that, I mean, if my back ends application used Teleport and machine ID for every connection, every request. Is that a good thing? Should I be should I be experimenting with that?

42:20 We like, that is if you look at the demo video for machine ID, that's sort of what we led with. It's talking like a microservice, talking to your infrastructure. Now I guess the question is, is it a good thing? We believe so, especially for, let's say, your development or, like, let's say, you will need to do some local development against a staging database. It's like a perfect use case. For your production DB, I'm not a % sure on, like, our performance. You know, if everything has to go through Teleport, what's the performance overhead? I asked

42:53 What's the Performance Overhead

42:55 the team. They said it's kinda, like, minimal because once they sort of connected, it kinda goes through, but it is like another something else in the middle. But I guess Teleport is not handling log. Connection pulling or anything like that. I'm assuming that every request is is spot up, accepted, responded, and then shut back down. I'm not a % sure on the connection pooling and the performance overhead, but we can I think this is an area that we'll, like, keep exploring, and this is definitely an area in which we wanna, like, get you know, we wanna sort

43:31 of remove all passwords? And passwords, we also see as, like, API keys for your databases or whatever sort of secrets you have. Yeah. And so you Sorry. You can do it for, like, your microservice. So instead of adding it instead of using, like, hard coding a Postgres or, like, Redis database, you just use the certificates that you obtain, and then you can sort of go about your connections. And one, you know, like, who's accessing which database, and you have the full sort of audit log. Yeah. I've kinda got this grand vision though that in my Kubernetes

44:11 cluster is that every single backend service that needs to speak to any other service could just have a sidecar teleport thing that does the machine ID and then works out if I'm allowed to speak to Postgres or I'm allowed to speak to Kafka or I'm allowed to speak to Redis. And it just, like to the point where I'd almost have, like, zero configuration anymore to speak to a database. And I think that would be pretty cool. Yeah. And I've it it seems like what you're showing me is kinda where it's heading, and I think that's really exciting.

44:46 Yep. It's definitely where we're heading. And, hopefully, we'll have more demos over the next coming months. Maybe this is a good follow-up livestream. Yeah. That seems like one of those questions to you. Like, that's not a road map, and I can't really talk about that yet. So just shut up, David. No. It is not. It is in our video. I don't have a working example. No. No. Sorry. I don't need to see. I just I think, you know, especially if, you know, I know that the workload identity works with AWS, but I'm like, could they pick up the

45:15 service account and verify that? And then they know the pod with, like, the the the downward API. All of this could be baked in to some sort of agent. And then all of my requests to other databases and even other services to another point could be handled in this kind of secure audited fashion. And I've basically, what I'm saying is Teleport, that's product has to be a service mesh. I think that's where you're going now. So we need we need we need t mesh. Can we get t mesh? We actually talk about why machine ID isn't a service mesh in our

45:43 Machine ID vs Service Mesh

45:48 blog post? I used to start paying more attention. Let me see. Go here. And what do we say? We have a full so we talk about like PKI and machine ID. And then we'll talk about service measures. Oh, yeah. So like how we kind of like fit in. This is like the official word. Think our answers are still a bit vague. So we say we're complementary. We sort of work together with service meshes. But like all these tools, you know, it's always like building on top of other tools and adding on top of it. Well, yeah.

46:26 Well And if you look at like Sorry. Go ahead. Oh, if you look at, like, Kong or or Envoy, like, they have, like, some basic certificate authority kinda, like, baked into them, but you can provide your own certificates. And so maybe this is sort of a avenue for least for the, like, CA management and RBAC is sort of how we support the sort of service mesh solutions. Well, yeah, I I was kind of about to touch on that as well. Like, all of these things pretty much, I think nine times out of 10, they run their own

46:57 end cluster CA as they handle automatic MTLS across services. But I think they only work mostly for HTTP traffic. I don't think it works with the database level layer, or at least it doesn't have that, I guess it's L7 awareness. I gotta have to try to remember my OSI layers, but I think L7 is application though, right? It would be cool, yeah. I could see a future, I guess, where Selium and STO, Linkerd, etcetera, maybe start to rely on Teleport as a CA and be able to pass those around, bake them in more levels of an

47:33 awareness and application handling, be like, oh, you know, this pod is now trying to speak to Postgres. We're gonna give it the right certificate to go and authenticate against that. I don't know. This is just grand ideas. I'm I'm it just makes me excited to see what Teleport does in the next six months and the next two releases because I'm pretty sure you guys have got a lot in your heads and a lot of people with great ideas too. Yeah. Definitely, we'll be exploring more avenues. So, like, we just closed the loop on Redis. Oh, yeah. That's right. We were doing a

48:02 Redis demo. I got And then you can see we have oh, you can see I typed Redis info wrong. You can say, like, who executed which query and which database. And I think this is, like, the core, you know, Teleport's bread and butter. It's like, you have these shared logins for accessing services. It's the SRE team, and you know it's, like, tied to my identity as an error in accessing this host, doing whatever, like, debugging commands, which is that easy sort of compliance check. Cool. Alright. What is next on my list? Okay. So another one that's pretty exciting, but I tried

48:39 Feature: Moderated Sessions (Four Eyes Principle)

48:41 it before this and it didn't work. So is our access requests, and we can try it now. But I had this book post today, which is why the four eyes principle is critical for access. It's a bit of a long ramble about basically how, like, the majority of our Teleport has gone from is there's different technology solutions. So we have like our session recording, but there's ways around our session recording because there's problems with restricted shells. And so we talked about why we created enhanced EPPF session enhanced session recording, which is great, but it also doesn't necessarily

49:23 stop everything. So, like, you can't really like, you can turn off SCP, but there's still ways in which you can, like, exfiltrate data. And we there's another good blog post here about restricted shells. And the long winded conclusion is, like, ultimately, you sort of need more individuals and humans to be on a session to either moderate it or see what's happening. She Coinbase back in the day, I guess they don't do this now. They don't now they're globally remote, they had, like, a SSH room with a drop cam in which Coinbase would do all of their SSH activity, and that

49:56 was the only way in which you get access to certain hosts. And and to some degree, you know, with shared sessions, which I guess you use for your clustered, you anyone can kinda join them or there's, like, some limited, But you can now set, like, our back to require certain roles to join a session for them to join. So there's an example Kubernetes role here in which you need to be a sec ops role and an audit role to join a session. And I can try the demo gods here. I mean, this Lisa was my demo user.

50:31 Moderated Sessions Demo & Audit Log

50:41 Also a great programmer at Teleport. Log in. And then this is, you know, like the standard flow for thing, keep configs. I have a okay. So they should I think I just filed a bug this morning. Like, there should be a message here saying, like, this session is sort of hanging right now. And if I come to the audit log, you can see that Lisa has made a request to access a Kubernetes cluster for, like, a kubectl exec and is waiting for someone to approve it. And until someone else joins this session, it would just sort of hang.

51:34 And that other party or multiple parties have the ability to also terminate the session as well, and this is sort of our Kubernetes moderated sessions. Cool. But I it's a bit difficult to demo because I also need to hang up another Yeah. To have a second login. Join. And yeah. And I think I can't, like, exit this either. I think there's, another bug, though. More of the bugs should be gone for moderated sessions. Awesome. Yeah. That would be a cool feature as well. I've been asked while you're on the audit page. I've been asked what is the EI

52:05 What Is the Ei Field in the Auto Log

52:07 field of the audit log? Field, but maybe, you know. I actually don't know what the e I field is. That's a great question. I think it's always zero. I can get back to you. Yeah. Always zero. Unfortunately, we don't know. We'll we'll work it out. We'll leave a comment on the video, and we'll tweet something about it. But, yeah, I'm curious about that. Alright. Then last up. Wait. There's more? There's more. There's always more, Dave. You should know. It's fun for the Seven for Redis. Just throwing that out there. So e I for Redis was seven.

52:52 For Redis? Sorry. You you can't get over your demo. I can track that though. So one of the other additions, which actually I think is pretty cool, we didn't really talk about this much, me edit my role, is we added prior MFA. So we've added this feature that lets you add other let me just check that this this is right. I could just I just see it over and see what happens. I think it doesn't okay. Require MFA is not right. Okay. So added to the per session MFA, and you can set required MFA sessions.

53:42 Require Mfa Sessions

53:44 And so each time you start a session, you are required to present a hardware token or a software token. And we added this support for Windows access and what is it? Require. Require session MFA. And so this lets you add an additional second factor. So in my account settings, I have a YubiKey, and you can add multiple hardware tokens. Yeah. And this sort of prevents like, let's say someone gets access to your GitHub account. If you acquire MFA, you know, they would also need access to, like, a hardware token in addition to whatever to get access. Yep.

54:29 I think I need to log out back in again. Useful for production databases, I would imagine. Yes. And there's actually one open issue that we're making, like, per like, it's currently per road, and we'll plan to make, like, per label to just give you a bit more flexibility. Okay. But we added this let's say if I come into this domain controller. Okay. Without the clipboard. And it's gonna ask me to verify my identity, which means I need to tap my token, and it logs me into my domain controller. You can see that clipboard sharing is enabled. It's recording.

54:48 Desktop Access Demo (Windows RDP) & Session Recording

55:11 I don't know if you spend much time in main controllers. I try not to. But no. You try not to. But, you know, this was like, desktop access was adding in Teleport eight, but we've sort of rounded it out so the recording is a big addition along with our clipboard. So let me just close this. Let me just disconnect. And now we have the same session recording and playback for desktop sessions as well. Very cool. So you can know what's happening, which is pretty interesting. Like, the most basic level of RDP is like a tile of PNGs.

55:59 And actually, I have another, like, livestream with the developer. This is pretty interesting sort of background. Like, we built it from scratch, our sort of RDP protocol support. And so if people who are interested especially in, like, we built it with Rust. It's like an interesting sort of tech deep dive and sorts of interesting tech problems with adding desktop support. 60 PNGs per second just being saved. Is that what it is? Yeah. I think it's like cut, like, tiled. And you actually you can't stop and play it due to like the stream of PNGs. We're also looking like different video formats. So

56:39 we're sort of, I guess, trying to get feedback about where we should sort of take this session player next. But for people who sort of upgrading their, like, Windows desktop access, lots of exciting sort of updates and improvements for them. Sweet. Awesome. Well, I'm glad I'm Alright. Working on session recording for desktop sessions, and other people get to do that because so while you were also saying that, I looked up the e I thing. Are you curious? I'm curious. Yeah. It's event index. So the zero is every time you open a new session, and then every subsequent command,

57:07 Q&A: Audit Log 'EI' Field (Answer)

57:18 e I will be incremented by one so that you have an order for the commands executed, which is why register seven because you did the ping pong earlier, and then you did an off, and then you did a and so forth. So that's it. Easy with you. Alright. That's it. Yeah. It's good idea. Event index. So I know there's a lot here in this version. If anyone has any questions for, like, machine ID or they wanna try it out, I know I'll be hitting you up, David, for when we add our database support. Sort of Kubernetes services

57:56 will be an exciting area for exploration. Yeah. Definitely. It's something I'm gonna have to I'm gonna start playing with machine ID now and and see how I can incorporate it into my my production infrastructure. I have a lot of automation that works. So and I am very bad for long lived hard coded tokens. So I'm gonna see if I can remove a lot of this from my workflow. I'll definitely I'm I'm sure I'm gonna be in touch on Slack, like, help. Yeah. If anyone wants to join, here's our link to our Slack room. And, yeah, we'll it's sort of growing, and

58:31 we're always sort of happy to help. And then if there's any bugs, you know, we're like an open call, open source company. You can sort of come in here and sort of see what's happening. And then the last thing is we talked about, like, roadmap and supports. We have our upcoming releases page sort of describes, you know, Kubernetes and database access support for machine ID. We have a Teleport Terminal, which is gonna be an exciting addition in 09/2002. More to come on that. What's the Teleport Terminal? My interest is peak. Tell me. Me. Tell me. It's one for another

58:44 Upcoming Features & Roadmap (Teleport Terminal, Databases)

59:04 webinar. Keep that for a different session. But, yeah, you can see we're starting Cassandra and Snowflake support and going GA. So lots of stuff sort of happening here. So, yeah, just we're sort of available and here to chat. Alright. Well, we'll give everybody a couple of minutes. So if you have any questions, I'll drop them into the comments now. I see we've got one from you, Evan Ash, so we'll tackle that first. But if anyone else has any questions, let us know, and we'll tackle that. Asks, does Teleport provide access controls to streaming services like RabbitMQ

59:40 as well? So are do you plan on supporting Rabbit, Kafka, Redpanda, anything like that? I'm not too sure, actually. I'll have to talk to Roman. I kinda forget what the auth protocol is for Rabbit and Kafka. It's been a while since I've been on the sharp end of it. I think Kafka by default is nothing. But you can configure it for x five zero nine. So this it may work well. RabbitMQ, I'm not that familiar with anymore. I haven't touched it in in many years. So I'm not sure what the authentication is like there. Yeah. It's definitely something that we can look

1:00:20 into. Like, open a GitHub issue, we can sort of dive into it. I guess the idea being if it's x five zero nine, then Teleport can support it. And it's just a matter of how many people want it and upvoting all these issues. Right? Yeah. Pretty much. Alright. So, I've been asked, go check for RabbitMQ or Kafka issue, whatever one you're looking for. If it's not there, start it. Thank you for your question. Oh, well. We'll we'll give everyone just one more minute. How's your how's your day going, Ben? Is it morning or evening for you? It's the

1:00:55 morning here. Just getting started. It's the accent that throws me off. I'm always like, he's right next to me, but he's not. I know. Just getting started. Done my blog post for the day. I've done my livestream, so might take a little break. Just important to step away from your computer. Always. Well, not from me. I'm looking outside. It's absolutely heaving down the rain. So I might just end up staying at my office and not leaving. It's gonna be 24 degrees today out here. So Where are you again? I'm in Oakland. Oakland. There we go. I

1:01:28 Yeah. You're always gonna get a good weather over there. Right? Yeah. Do you ever miss the British weather? No. I don't miss the British weather. Alright. I don't think we're Oh, by the way, think last thing, are you going to KubeCon? I am going to KubeCon. Are you going to KubeCon? I won't be going to KubeCon, but we'll have a big team there. So if the last six people on the stream will be going to KubeCon, I know you have some exciting t shirts too, and we will have lots of swag. So definitely swing by the booth for

1:01:41 Community & KubeCon

1:02:01 a chat or a demo with some of our team. Awesome. Well, maybe I'll see you at KubeCon. It's Detroit in North America, right, at the in October? Yeah. I think I'll likely be in Detroit. Cool. Sweet. Alright. Well, Ben, you for joining me. It's always a pleasure. I'm looking forward to doing more experiments and demos with you, especially as we roll out more Teleport features. And I I'm definitely gonna hit you up for Teleport Terminal and see what that is. Plus the fun things to play with. But thank you again. Have a great day, and I'll

1:02:13 Conclusion & Thanks

1:02:31 speak to you soon. Bye.