1:30 Introductions

1:30 Hello and welcome to today's episode. This is Rawkode live. I'm your host Rawkode. Today, we're gonna be taking a look at Falco, a security run time for Kubernetes with the magic of EBPF. I don't really know what that means, but fortunately I have two wonderful guests who do. Leo DiDonato and Lorenzo Fattana, two former colleagues of mine at my days from influx data, now working as engineers at Sysdig and maintainers of the Falco project. Before we bring them in, I wanna thank my employer, Equinix Medal. They provide the time for me to work on the show and

2:03 produce content to hopefully educate and help us all learn this very vast and complicated cloud native landscape. If you want to try Equinix Metal yourself, you can use the code Rawkode live. This will get you $50 of compute time. That can get you roughly a hundred hours of a sixty six ninety six giga ram machine with 28 cores or you can splurge it all in five hours with hundreds of gigs of ram, 400 gig of ram and three terabytes of disk. So you know, take that take that $50 and use it wisely or unwisely. That is up to you.

2:37 Guest Introductions (Lorenzo & Leo)

2:37 Now, I am going to bring in my wonderful guest for today. Lorenzo. Hey. Nice to see you. There we go. How are you both doing? So far so good. We are really into some new things for Falco at the moment, but everything's fine. Aside from the fact that everyone is buying into our houses here in Italy again as everyone else in the in the world. Alright. Well, why don't you I enjoyed your music, honestly. Yeah. Yeah. Yeah. Like, I don't know. I keep meaning to make a joke about it. Like, the intro and the outro are quite

3:18 quite energetic, and then there's me, which is, like, way down here in the energy kind of space. So the better polarity, but, you know, I I like it. It's good. So why don't we start with Lorenzo? Do you wanna just introduce yourself? And then we'll move over to Leo, and then we'll talk about Falco. Yeah. Well, I'm Lorenzo Fontana. My nickname is unpronounceable, so I'm not going to pronounce it. Just read it. And I'm working at Sysdig, and Sysdig is sponsorship. Mine and Rawkode works on Falco to be the maintainers of Falco and to, you know, help Falco becoming

3:55 some more, you know, stable and adding features. And also, you know, with, hopefully, having Git graduated in CNCF after reaching the incubation last year. I'm also the author of the Hooks of SurveyMedi with with David Calavera for Riley, if you want to check out that book. And I'm doing a lot of work recently, with my friend Leo and with others, and that's it. Leo, to you. Okay. So it's my name is Leonardo DiDonato. As Lorenzo said, I'm an open source an open source software engineer at Sysdig, where, basically, our daily jobs is to code Falco, maintain it, and all of it. So,

4:40 yes, we spend all the day on GitHub, which is something that we love. And basic basically, you can find me in the web verse with nickname that I put here, Ledido, over Twitter, over GitHub. My daily duties are very similar to Lorenzo's one. So, basically, we're we're in charge of coding Falco, which is a c plus plus project, but also to maintain other parts of Falco that that makes possible to detect security threats at runtime, namely Libsys, and the drivers of Falco that at at the moment, have, like, three drivers, a kernel module and a BPF probe one and user space

5:26 one based on Big Trace, Ackery, Wizardry. Great stuff. Yeah. I mean, I understand some of the words. So, I mean, we're making progress here. Can we We we don't even understand them words. Yeah. I mean, you said c plus plus, and I was like, oh, dear. Like, that takes me way back. I haven't written a line of c plus plus in a long time. Although I do keep up with the changes to the language and it's looking really more impressive these days with some of the newer changes. So Yeah. Let's talk about Falco then. What are the primary use cases for Falco?

5:57 What is Falco? Use Cases

5:59 Why should people have Falco running on their cluster? Go. You wanna go or I I go. You can complement. Leo keeps me sane usually. This is one of also of Leo's duty, I mean, keeping me alive. So Falco is, you know, kind of simple, but also, you know, somewhat complicated. Basically, you use it to be alerted when something bad happens in your cluster. This is a very basic explanation, very high level explanation. So let's say someone spawns a shell or someone, know, creates a privileged pod or someone creates a config map in Kubernetes with, you know,

6:46 some, credential that should not be there, like, you know, AWS credential or, Equinix credential, these kind of things. Someone she modified, they should not be she mod, or someone gets root access to something, or someone manages to, you know, get a root shell somewhere where it should not be allowed to. This is all done via a thing that we call Falco rules, and they are just expressed in YAML because, you know, everything is YAML. And so you just write some YAML, and out of that YAML, you get some nice alert messages for when something happens. The value of Falco

7:29 is not just, you know, the code that, you know, processes the YAML and gives you the alerts, but the community also puts a lot of effort into creating a same default rule set for, you know, the vast majority of the people out there. So you can create your own rules. You can make rules, you know, adapt to your organization and your needs. But by default, we ship a default rule set, which should be, you know, good enough for most of the use cases. That's what I had to do. Can I add a bit? Yeah. Please. Please, sir. Basically,

8:02 Falco's Place in the Security Landscape (Detection)

8:02 just like an introduction, more more generic. If we look a way to better position Falco in the security landscape I mean, if we look at the security landscape, we could simplify it into, basically, two parts, prevention and detection. Both of the parts have one thing in common, which is policies, which is another cool name that we could give to Falco rules. And the difference is that while prevention tools use those policies to change the behavior of processes by preventing Cisco's to happen or stopping processing to to execute and things like that. On the other side on the other end,

8:47 the policies detection tools use policies to monitor the behavior and notify a a user when the behavior of a process steps outside of the policy, which means that Falco, differently from other tools like Secom, Secom BPF, app app armor is poses the Falco position is on the detection side of the security landscape, which does not mean that you should only use Falco or you should only use prevention. Basically, the best approaches in the security is to use a debt defense in that, which means stacking different tools with different strategies, some for prevention and some for detection together to

9:28 achieve satisfactory level of security, which, by the way, will not all will not be perfect in any case. So this is where we can position Falco to make clear maybe to the to the audience that different from other tools like AppArmor, Secop, and things like that, it positions on the detection side, especially for cloud native environment like the Kubernetes based ones, which is a a field that's where security, I mean, should yeah. Should be solved yet. There there's only Falco and nothing more to as far as I know. Alright. Yeah. I mean We call it container

10:10 runtime security. Yep. Yeah. That makes sense to me then. Like, I I like the, you know, defense and death. We should be using our Palmer sec comp, all these things to restrict what our containers can do. But, inevitably, things can and will go wrong, and Falco is what we use inside of our clusters or or, I guess, it doesn't doesn't need to be in a cluster. Can it just run on on Linux? Like Yeah. It's going to you can also run on theoretically, that theoretically, that's the preferred way to run Falco on the host. The only cluster.

10:39 So we are on Falco on our host. It's gonna use e b p f and some other drivers. It's gonna alert to us when things happen that we want to be aware of. And then Yeah. From that, we can then react to that. We can then decide how we want to handle that information. Sometimes it may just be, notifications, like someone pseudo or whatever. I don't know what the capabilities are, but we'll look into that shortly. But I get it. That makes sense. Alright. Let's tackle. We got a couple of comments, so I'll just run through those quickly.

11:07 Oh, there's someone. So Robin says, you've been enjoying the streams. Thank you very much, Robin. They're gonna keep coming because I love to sound in my own voice. Alison is saying, looking forward to learn more about Falco. Me too. Definitely one of those things that seems scary and I'm hoping that you two are gonna make this easy for me to understand. And at Kakaran, bloody YAML, I think we are all we all have that same sentiment. Yeah. We all have that sentiment, but I still prefer YAML to second policies. So I have a few And

11:42 also processing YAML in Go is, I think, processing YAML in c plus plus, different league. It's a whole new level. I mean, as much as we rant about YAML, I think we all do in the cloud native space because there's so much of it. It's really, I mean, besides the edge cases, it's really easy for people to pick up and learn like, you know, there are a lot of benefits as well. We shouldn't maybe be too quick to judge it. But anyway, let's get started. This is gonna be interesting because I had some homework to do upfront

12:08 Demo Setup & Falco Prerequisites (Kernel/OS)

12:13 and I'm hoping I've not messed that up. So if anyone wants to learn more about Falco after today's session, it is open source, you can go to github.com/falcosecurity/falco. Today's servers are provided by my employer Equinix metal. I have spun up using the cluster API, which is just amazing. My own Kubernetes cluster, and I have some backup resources here that I think I'm gonna need when it comes to doing some of the Falco stuff. That was my homework that I failed to do. Regardless, we do have a running Kubernetes cluster, which should still be running. There's always that moment of fear. But yes,

12:40 Linux requirements for Falco

12:49 we have a nine one nineteen two Kubernetes cluster. We are running Ubuntu 20. Now, there were some prerequisites from you both regarding to what I needed in this cluster. Can we maybe just discuss why we why we're worried about the operating system version being Ubuntu? Why we're worried about the Kubernetes version? What are some of those prereqs? So very quickly, Falco is very tied to the kernel. Right? So we are trying to make the Falco experience very you know, to be very good on, you know, most of the kernels, most of the distributions. It's very easy for us to be very

13:35 working very well on newer kernels because you have a couple of options to run Falco. I mean, the, you know, real options. We have, you know, the new user space option which which is, you know, still under heavy development, which should be solving the problems that, you know, I'm talking about. But the option that you have are are essentially the kernel module and the eBPF pro. Right? So the eBPF program is solely on very new kernels, as well on very new kernels. And the kernel module runs from kernel, like, two dot six to, you know, newer kernels.

14:11 This means that you have to compile a kinda module or compile a BPF probe because both, you know, rely on a stable, API from the kernel. So they cannot be compiled once run everywhere yet. Right? So you just have to compile for kind of where you are. The Falco community tries to package a Falco in a way that you you don't have to worry about that. And the way that we package that is, you know, the usual RPM and dev based packages. So if you install from those, it should be installed smoothly, and you shall just should

14:49 just work. If you install from a container, like, QCTL run QCTL apply something where, you know, a Falco container is running. Like, you use your our own chart or you use our YAML definitions. It will use a Falco docker image, Falco container image, that will need to figure out the kernel and compile the kernel module, the BPF pro for you. Right? Okay. If it can't, we have a web server where we ask thousands of compiled kernel module in VP of probes for thousands of different kernel combinations. So you are trying. You can go to downdownload.falco.org

15:34 to to count all the the the page drivers. And this is very I mean, just quick driver and other should QA is the latest. So this is very this is very useful, but what happens is that for common operating systems, we pay more attention to it. Like, for Ubuntu 20, for US Fedoras, for Centos, for very widely used of our thesis. We did a survey, and it turned out that, you know, people use Ubuntu, Fedora, Centos Amazon Linux. Amazon Linux. A lot of Amazon Linux is probably our, you know, most used, image, and the others. So we try to keep

16:21 up with that. We have a lot of Yam every every one of them is a YAML file. So, yeah, more YAMLs for the comments. So if you want to compile if you want us to provide a precompile for your specific combination, you open up a request with a YAML file, and our build system keeps that takes it and compile the Falco driver for that one. The current module or the BPF probe or bus? Yeah. The current module. Depends on if it can. And you put it on this downloads repository. So we are trying to, you know, make

16:56 sure. But for this demo because, you know, the demo gods are not always good. We just said, oh, work to 20 should work. Maybe it doesn't, but sure. We'll see. So to summarize that, it it wasn't actually a prerequisite. It was just to simplify things for today's session. If if you want to use other operating systems, it's always an option. You just may have to compel the drivers of yourself. That's it. I okay. Yeah. I think I get that. That makes sense. So let's That's the last resort we want you to have. But hopefully I'm sure things things are on their side.

17:30 Installing Falco

17:35 Yeah. Yeah. We'll be fine. Okay. So I should I pull up the docs? That would probably be the first step. Right? Exactly. Yes. Alright. So we have a Kubernetes cluster. I have access to all of the machines. So I'm assuming you mentioned earlier that we're gonna apt get install Falco. It's kinda what I'm seeing here too. So am I just gonna go ahead and add this repository. Right? Yeah. We have two. As I said, we have two ways of installing Falco. Maybe, Leo, you want to give a reason why we are suggesting apt? Well, this way, we're

17:39 Installing Falco on the Host (Apt)

18:17 this way, we'll install Falco on Diosk. I don't I I'm it's not clear to me if we want to install Falco on Diosk, which is the recommended way or if we want to install it with something like Alcharts. I will go with everything because the experience is way cleaner in my opinion. And also I agree. Architecturally, that's the place where the Falco should be in the host. Yeah. Otherwise Sorry. Sorry. It's very what what do they want to say? Maybe I'm just saying it for him. Is that Falco is a very foundational component in an infrastructure

18:56 because it actually, you know, makes sure that nothing happens in the host. Right? We are also doing steps to make sure that Falco boots as fast as possible in a system, like, even maybe, if possible, like, exactly when the kernel boots at some point because Falco needs to see everything. I mean, if someone put a rootkit in your unit 30, you're not gonna have Kubernetes booted up in your unit 30. Right? So, for example, so we are, you know, encouraging users to always deploy Falco at the same level they deploy, you know, their other, machine component that they use for, like, the

19:36 queue the cubelet or the CNI plugin. So it's at same level so that they so that Falco can take over and, you know, understand everything that's happening. You can still deploy Falco in Kubernetes. Right now, it doesn't have, you know, any differences in features. But in the future, we want to make sure that Falco, you know, takes over more components. So we are, you know, suggesting that. Also, if you install Falco in Kubernetes, that means that if someone access your Kubernetes cluster, you know, unauthorizedly, they can stop Falco, like, immediately from every node that you have.

20:12 And that's not good because Falco is there for Or worse, they can leave it running but removing rules. So you will not notice. You will go, okay. Everything is fine. Falco is not for triggering any rule, any alert, and that's because they disabled the the rules at all. So that's You know, worse. I Yeah. Yeah. That's that's makes sense. You know, Kubernetes is great. Containers are great for most workloads. But if we're integrating and talking the kernel, we're providing security concerns, like, running this on the host seems to me like it should be the right way forward. So

20:45 yeah. Let's And also, not many people want to run a privileges container in a cluster, and Falco would be one because it does no other way. So so It's broadcast. So that should Yeah. Yeah. Yeah. I know this is gonna be a pain for people to read, but all I'm doing is copy and pasting the command. So we've added our Falco repository. We need the Linux headers because we're compiling a kernel module or installing a kernel module, I would imagine. And then we install Falco. And that should be it. You're really mastering this science, David.

21:08 Verifying Falco Installation and Initial Logs

21:26 Yeah. It's it's always start off strong and it goes down fast. Don't worry. Like so that's it. Okay. That's just other distributions. So does the a bunch of package do the driver installation, or do I still need to run through these steps here? It should be already done. You can check that out with LS mod pipe grep Falco. Alright. So mod probe Falco? Oh, wait. You probably didn't do a p t install Falco. I did. Oh, yes. Oh, yes. It's it's there. Okay. Okay. Okay. I just had to mod probe it, and all of the machines

22:05 now have Falco returning on Alice mods. But I think that's a win. If you just see the CTL start Falco, it will it will have done the mod probe for you. Yeah. Just we didn't start it. Of course. Test on the all the things. I always forget. Okay. So that Let's verify let's verify. Okay. PSOX. Okay? If you run Oh my gosh. It's working. Yeah. If you run journal CTL dash f f u, Falco, we should see something. I feel like you just told me to go fuck myself. Yeah. Fuck. Yeah. No. Sorry. No. That's so that's what

22:52 That's another way to see it. Okay. And you will see that Falcon already showing you something. I see something. Yep. Okay. Yep. I think it we're good. It's running. Okay? We is that do you wanna run anything else? Is there anything else that's gonna give you a bit more confidence? Or we We can do like, before we dive into Kubernetes or we see anything else, I will love for us to analyze what Falco's saying. Looks like you're running Calico. Yep. And, you know, Calico is totally, you know, a good workload here because you installed it.

23:28 And but Falco is complaining because it's a privileged container and has been started by the image Calico node v three one five three. It looks like we don't have an exclusion for Calico. Right? We try to do as much exclusions as we are able to, but we don't, you know, find we don't have exclusions for all the, you know, software that should be excluded. So if someone is listening and they want to, you know, do a contribution, just go to our rules file in the Falco repository under the rules folder and make an exclusion for Calico. It would

24:07 be very good as a first contribution. And if you want to try something just to see, you know, what Falco can do, go to the TMP folder and create a file. Alright. Like, touch or read to file in t m p. Alright. Can I ask a question first then? Yeah. Sure. So that this is Why are we here? So this is visible to people. Do I have any more commands I need to broadcast, or can I close these other tabs? Okay. So we don't have to do anything, but we probably want to configure the Kubelet,

24:42 the Kube API servers of the machines Alright. Later. I'll just grab the IP address of this note then and get a new tab. Yeah. Maybe. If you want. If you are able. Yeah. That's fine. I'll keep the I'll keep the splits here because I we may have to do other stuff and the rules fail itself may have to show up. Okay. So let's jump on this box. Now what we were seeing there was so that yeah. This is the line here. So Falco is running. As noticed, we have a privileged container and Yeah. Okay. And what as Falco is saying here, we're

25:18 starting an internal web server because we, by default, start the web server, which exposes a cube API server webhook endpoint for the audit logs, and you can configure a cube API server to send hooks to that. But before we dig we dive into that, I will love you to, you know, try to make Falco angry. Yeah. Let's create a Lorenzo file here. Okay. And then g mod g plus s Lorenzo. Save save this file. Also, empty it's no. Change mode to on the file. Yeah. Just change mode the file. Set the suite d group ID g g

25:45 Triggering a Syscall Rule Manually (chmod)

26:08 plus s. Plus s. Plus s, Lorenzo. Okay. Now if you go to the Falco outputs, you should see another triggered. Uh-huh. Look. Now this set to your set to the beat is set via c mod, and it tells you everything that he knows. That the file name was the MP Lorenzo, the flags that have been set, the user that did the operation and other things like that. The the exact command line, she mod g plus s Lorenzo. And in this case, container ID is sourced because we are running this operation on the host, not in a container. But if you do

26:56 the same, like starting a container, an Alpine one, for example, and the second exponer shell in the container and you do the same thing, Falco detects it. In that case, it will give you the container ID of the container where you did that operation that malicious operation. Exactly. And also shows you before telling you that, tells you, hey. Someone started a container with root shell, and they are doing comments, and the comment is this one. Yeah. Let's try. Maybe it's better if we try it. So Okay. So what's the docker run run IT Alpine. Docker. Alpine

27:29 Triggering a Syscall Rule in a Container (touch)

27:40 3.9. Do do dash do dash IT. Yeah. Yeah. Something like Oh, this connection is very fast. Okay. Cool. So, like, you can you can edit a file in the bin in the bin directory, or you could you could redo the TMP, Laurens, a thing. Or you can Maybe maybe try to create a file in the BIM directory with whatever you want and just save it and then go out. Just touch just touch the file without anything. And if you do APK like you were doing, it will also see, hey. Someone spawned the package manager. Yep. Okay.

28:29 Like So just touching that file, and then you want me to drop out the container and look at the Falco logs again. Yep. Yep. Yeah. Love the comments. It's read. I love the comment I just said. I love how Leo is sat here like I wanna type. Yeah. It's always like that. Okay. So Falco internal Cisco event drop. So it so what what does this one mean? What does it mean by the one system call was dropped in last second? That in some cases, this is something that we are really been trying to solve, and we improved that a

29:07 lot. Basically, as Lorenzo say we're saying ten minutes before, we are using Ciscos, which which are the main interface that we have to the Linux kernel. And the problem is that Cisco's are a lot. They change very often, so it's very hard to keep Falco track. But, also, they happens at a rate that sometimes is very, very difficult to keep up. So in some cases, for various reasons that can depends on the the throughput of Cisco, the amount of Cisco's that are happening on the machine, or, at the same time, issues with the page the memory pages and things like that,

29:47 we can experience and detect some drops. So we can experience that we have not received some input events on Falco, so we could not know what happened in that only in that exact instant. But this basically is dropping, like, what what? One system calls in the last second. In a second, can also happen one or two millions of Cisco's. I mean so okay. We will fix it. We will try to avoid this to bring this number down to zero, but this is what what the backlog tells us. What we are really interested here, we about the

30:02 Understanding Dropped Events

30:29 the experiment that we were doing, yes, is that line. File below a non binary directory open for writing, touch by doctor, file being by doctor. So it detects it and Yeah. You can reconstruct the whole story. Right? If you go notice, privilege container started as first event after web server. And then notice at the so with the GDB, which is what we did before. And then notice a shell was spawned in a container. And this is was the a s h shell that you spawned, and it tells you the image and the container ID. Right?

31:00 Falco default rules

31:05 Then, you know, that's the drop. But then it immediately keeps up and, you know, shows that you just, you know, created the file. So it gives you a kind of, you know, history of what happened so that you can reconstruct the whole attack payload and see what happened during the session of the attacker. Yep. Also, in this case, David, as you as we were mentioning before, we have the container ID and the image name differently from the past experiment that was telling us container ID host because it was run on the host. Exactly. Okay. So

31:45 Recap Syscall Detection

31:46 the one thing that I'm I'm kinda thinking about now, like, haven't configured any rules. So can you is this a default rule then? If we create something on a known binary directory, it's gonna trigger on it. What what are some of the default rules? Can we look at Good question. Yeah. If you go to h c Falco. Yes. I love it when stuff's just intuitive. It's like, I just I I just thought it's definitely gonna be there. So Yeah. Just it's DG path. And so you have Falco YAML where all the default Cisco rules are. You have Falco

32:00 Exploring Default Falco Rules (YAML, Macros, Lists)

32:24 you have Kate Savit rules YAML where all the rules for the Kubernetes, we didn't use it yet. And your Falco rules local, where you can add your own rules without the package system to override them, or you can create, you know, multiple files in rules default. A couple of options. Rules dot d. Yep. Rules dot d. Yes. We are also working on a system to enable people to cruise from the outside, like, from distributed system, like, you know, it's CD or Kubernetes, and also, like, reload them Yeah. Which is something that we go in the next features. The next That is

33:08 that is what we were working before entering this stream. Yeah. Alright. So let me see what we have here. Okay. So this is the Falco configuration, but the the default rules are Yeah. Sorry. This one. If I if I could type. So rules. YAML. You can type YAML. Yeah. Okay. So we got Wait a minute. Wait a wait a minute. Since since we are here, Falco rules is YAML, but, basically, we built a language, let's call a language, on top of the off top of the I was on YAML features. For example, we you can define Falco rules using

34:00 macros. So you can also define macros to later reuse them in different rules. For example, here we have some macros that are specifically being created to detect open Ciscos, open read, open write. You can do the same, and we have for exec v, another Cisco that's usually used to run executables. You can also create lists that you can use to exclude or include a list of topics. For example, I will exclude Calico, this other and this other this other CNI plug in because I know that they are okay to run-in my Kubernetes cluster. So I will create a list with the

34:40 all the name of the CNI plug ins in ages and exclude them in a in a in that rule that we were talking about before. Soon, we are also implementing a mechanism for exceptions in rules, but the Falco default rule set tries to cover the most common threats, the most common use cases. And since it's checked in the GitHub repo, everyone, and this is very common, can contribute to it and help us to keep it on track with new new tools, new methodologies, new things, and so keeping the rules updated day by day. Anyway, if you scroll.

35:22 If you want to have, like, an example of what you did before, you can search for privileged, and you will see the rule that privilege it. Yeah. Search for privileged container started. Yes. Exactly. Launch privilege of container. This is what triggered when you started Calico in your in your cluster. Okay. So Yeah. Here, there's container started. There's a macro sorry. Go. No. I mean, you're I I was gonna just guess what I think it means, but I'm more than happy for you just to walk us through that. So if you wanna run through the conditions, that's that's all good.

35:25 Analyzing a Specific Default Rule (Privileged Container)

36:07 Yeah. The is what I we were saying before. Basically, the to provide this rule to become, like, a a flow of if a a flow of conditions, basically, we used some macro here. Container started and container and container privilege is true, which is a field. In this case, this is not a macro, and not Falco privileged containers and not user privileged containers, which are true lists, which means that you can pull you can put something in the user privileged containers list to exclude that thing to trigger this this rule. At the same time, you can go and look at the container

36:52 start at the macro or at the container macro. If you search for macro, colon, space, container, you will reach the column, micro double column. I don't remember how to call it English. No. One. Sorry. Column space container. Space. Yeah. There we go. Nope. Really they already want to type. Yeah. I okay. Yeah. So as you can imagine, this this this model is very simple. If container ID is not us, as we've seen with the two examples, that means we are in con we we are in a container and not in in The Us. And so that evaluates to true.

37:38 At the same time, container started checks the event type. It's exactly. And that basically, the we have the container image name complete. Yeah. Okay. I think we have a question, which I believe it's interesting. Cool rules come from OPA tool? Yep. Interesting question. It's a very interesting question. And the question is interesting because the work that I mentioned before that Liran and I are doing, you know, like, today is explicitly to allow external implementations to load rules into Falco. What I mean, that we will have a default implementation of our rule loader that we go, like, on,

37:56 Future of Rules: External Loading (OPA)

38:30 you know, files, Kubernetes, you know, whatever we find the default means. But, you know, either the Falco project or anyone can create a Falco rule loader that watches something and load rules in Falco. OPA is a very, you know, good use cases use case for that because, you know, it's, you know, what you express in OPA, and, you know, it's very, you know, how to say it's very expressive, and it's also very generic. So you can express, you know, a condition that resembles the Falco rules and then create the Falco rules and load them in Falco.

39:10 All these kind of use cases are exactly why we are developing that. And I believe that at some point, something like this will will happens once we have this, this thing. If you believe that this is important for you, I will love to see you in our community calls, to define maybe some more details. Because what does, you know, what does that mean? At some point, it's just, you know, what do the user expect, and I'm not a user of Falco. So I will just, you know, love to talk with some users about that. Okay. Just for anyone who's not familiar, Opa

39:47 is open policy agent. And if you want an introduction to that, I do have an episode scheduled for the November 18. So feel free to tune in for that one too. That's a question. Yeah. That was a really good question. Okay. Let's let's move on then. So we've got our Falco rules. Do we get all these default rules by default? They monitor things, the containers being run as privileged. They monitor for binaries and no one, know, privileged direct, not privileged, but where we keep binaries. So we have plask stuff, SSH stuff, like, so it seems

40:19 like you're just you're casting a wide net here with regards to what we think could be potentially dangerous and then giving people alerts on that information. And if I scroll to the bottom, we had 3,000 lines of YAML. So Do do you want to know a very funny story about Falco rules? I would love to. At at some point, we introduced the rules so that we can detect whether or not your cluster is mining some Bitcoins. Right? At that point, to detect if we were mining Bitcoins, the only way that we had was I mean, because, you know, a Bitcoin

40:59 miner what does a Bitcoin miner do? Right? I mean, they are all different. A very common thing that they do is that they connect to a miner pool to, you know, exchange keys and, you know, say, yeah. Yeah. I miner this. I miner that. Do I need tokens? And so what we did, we created a rule that connects to minor pools to figure out their IP address so that we can filter out the IP address for rules. At some point, people start opening issues of Falco is connected to minor pools. It's a minor, and Falco was not really a minor. But

41:37 that was a good idea, honestly, because, know, you can make some millions. We were probably on an island somewhere. Yeah. Yeah. Well, I guess That would be very cool. Yeah. If you're working on security software that helps people there on that stuff. It's also the best backdoor, though, where you could just go and make all the crypto you want. So why not? Okay. So We don't do that. No. Of course not. We're We're we're good people. So we have Falco running on all of these machines. We can see right away that we're getting output to our system d log that tells

42:12 us when bad things are happening. Is the next step now to hook up the Kubernetes audit log, or is there something else that you you wanna show off just from this setup that we have? Before we dive into that, I would love to just say something to the audience just for, you know, completeness. Because you installed Falco in, you know, with APT in all your machines. Right? And, you know, in a real production environment, what you will do is to, you know, put the Falco logs somewhere or be able to, you know, at at least see them in a way that

42:22 Falco Output Options (Sidekick, gRPC)

42:45 it's not SSH in the machine. So Falco has some constructs to do that that you can use. Falco can write logs to either a file or to the CS log or to To another program. It can be like you're doing to other program. It can use a it can it has a gRPC server where it can connect where it can connect with the g r p with a go or rust or a Python client that are already there. So you can, you know, integrate that into, you know, your stream log stream pipeline. Or, you know, we have a software in the

43:17 Falco security organization which is called Falco Sidekick that, you know, for example, can take, the Falco logs and send them to Slack or send them to AWS SNS or these kind things. So in a production environment, you will have also that piece that, you know, strings along somewhere so that you can analyze them, for example. Right? But that's, you know, not something that we do, like, in this session. Yeah. That makes sense. Very good for that. Okay. Cool. Oh, let's go with the audit. Yep. Okay. What what does that mean? Yeah. Oh, so we said that we have

43:50 Manually sending Kubernetes events to Falco web-hook receiver

43:57 a Falco web server. Right? The first thing that you can try to do is to actually send event ourself. Like, let's pretend that you are Kubernetes. Right? And we want to trigger an event to Falco. I'm sending the comment to you because we don't want to figure it to figure that out. Give me a second. Yeah. Sure. Okay. So I like the sound of your switches. I don't know if it can I send Jason in this private chat? Yeah. Sure. It'll work. It should work. Okay. This is an event. You should just save it to

44:04 Simulating a Kubernetes Audit Event

44:46 you just copy the command and save it to a file. Are we sure it has not been trunk truncated, Laura? Oh, maybe yes. It has. I I can I can send a guest? Maybe Yeah. Sure. Yeah. Share a guest. I I knew. I knew. I I should have done this before. Things dot txt and then this. Alright. Alright. Here's my guest. Great. So just a guest, it tests two comments in a file name, things update txt. And the first one creates just do them one by one because we need to the first one is creates a file sample

45:38 event dot JSON. So what happens when the Kubernetes when Kubernetes sends audit log is that it it just takes an endpoint with some JSON. Right? And, like, surprising, it's not the YAML. So it send this event, and the event here should trigger Falco a Falco rule described in Kate's audit rule YAML, and you shall see it in the Falco log. This is just to confirm that, you know, Falco is configured correctly, and it's working to accept Falco Kubernetes audit event should be, like that by default. But we just, you know, want to see an event before we configure

46:23 of Kubernetes itself to send them so that we don't think, hey. It's it's, it's Falco that's not accepting event or something like that. So then you can do the cool command sent to you. Okay. So this is just gonna send that to Yes. The Falco web server? Yeah. This, yeah, opens the sample event event JSON file and just sends it to the case, add it, and says, okay. And you can now yeah. We have to improve it. And you can now return our CTL, and it says, warning request by anonymous users. Allowed a verb. Create

47:03 API need spaces. So this is how an audit log that triggers a rule looks like. And you could open the rule file for the create Savvy logs and see the actual The re this rule one, request by anonymous user. Exactly. Okay. So we'll It's very cool because that when you have Falco ops, you can also at some point, you cannot start ignoring them. No. It's the other it's the other file, the file named gates underscore auditor. Exactly. Yes? Exactly. So the rule the rule sees it's a Kubernetes event, and the username is system anonymous. And it their authentication was allowed. And, you

47:55 know, anonymous authentication should not be allowed in your clusters. And this is not manages to do that. Yeah. And this is to match the SYS benchmarks. There's a CIS benchmark one one one that we I mean, every time a c I CIS benchmark comes out or a courtesy CV comes out, we try to update the rules files both for Kubernetes audit and both for the Ciscos to make sure that we are there from that behavior. Alright. Awesome. If we find the email for the CV. So I'm just processing in my head. So we actually without

48:40 Recap: Kubernetes Audit Integration (Webhook)

48:40 enabling any of the kubelet flags, we we just showed that the Kubernetes stuff works. Right? Like, we Yeah. We created a Kubernetes JSON, and then we just send it to Falco directly. And Yeah. This event, of course, didn't really happen in Kubernetes. It's just that we simulate it. We can now go and edit the the Kubernetes API server flags and configure it with the, the with the audit log path, audit policy file, and audit webhook config file. And in the Falco evolution repository, we have an audit policy that you can pass. But that is basically how you configure Kubernetes

49:00 Adding Kubernetes Auditing to Falco

49:27 for, you know, sending events on endpoint. The for Falco, the dynamic webhook feature that existed in Kubernetes, like, eight up until 01/18 was very useful. But, you know, it got dropped, so everyone who needs to configure their Kubernetes cluster needs to do this manual step, which is something that you are working on, you're removing, if it's even feasible for us. But every cluster is different. You might not have, you know, cluster. You might have a cluster from a distribution that doesn't do the same things in that way. So it's just, you know, just configuring the other log. So where is your API

50:07 server flags, file? Yep. Kubernetes No worries. One step at time. Server. Container d. I did look this up, and I have drawn a blank. Let's see. Oh, before before you, you know, start changing it on every node, we should just also download the audit policy file. Yeah. Go to github.com/falcosecurity/evolutionexamplefolderevolution.Evolution. Yeah. Okay. Yeah. It's like a bug. I think I've looked at this. Yeah. Go go yeah. One app. Yes. Auditing policy. Yeah. More. Okay. Yeah. We have a good extension to do this here, but we are trying to do it without looking at it. Okay. So you want me to copy all

51:16 of this? Yeah. Just download this file in all your notes. Yeah. Do get this file or do get as you prefer. In a place where you feel like you can leave it there. Like, for example, I don't know, ATC Falco over Okay. So broadcast w get Broadcast. Yes. I'm broadcasting some we are broadcasting a broadcast. Oh my gosh. Etcetera. I'll just start here. Yeah. Yep. I should have auto policy. Good. Very good. You don't have to configure it because it's actually, you know, set it up to to the default things, which should be good. And now we need the web config

52:12 for to pass to Kubernetes, which should should also be there, but they can drop it in the guest. This one here? Yeah. That one. Exactly. Yeah. We have to change Falco service cluster IP to local 127001. Yep. Okay. This is just something I'm going to apply to the cluster. Right? No. This is something that you still pass it to the Kubernetes API server. To the to the flags. We need to pass to the Kubernetes API server two files, the audit policy file and the audit web book config. And What? Also the audit log file. Yeah. And also the other log file if

52:56 you want to save the logs of file too. Okay. That's great. Yep. That was good. So what's next? We have to add the the API server flags. Right? Exactly. Edit API server flags. Alright. So let me find the status. Let let's over this. Go ahead and discuss our stats again after we do this. It it will. It looks like it's being controlled by the Can you edit them via the cluster API thing that you did, or you can't? I'm not sure. I could, but we'd have to wait for that to roll the changes out. Yeah. Just my curiosity. I don't want to

53:35 do it. I mean, I'm scared already. But Okay. So Kubernetes manifest, cube API server. Alright. Okay. We got what we need here. So That's cool. So the flags are But we wanted to do that on all of them. So let me Kubernetes manifests cube API server. Okay. Yeah. So the flags that you have to add are are developer I think that's I've got something. Let me let me copy them in the chat, probably. Oh, yeah. Sure. Go for it. I think they are in documentation too, but, I mean, I hope they are. Well, the first one should be this one,

54:27 dash epic server dash shark equal audit policy file. And then the other is about the workbook config file. Yeah. Just change audit policy file with the if you copy the net c file. I don't remember where you copied it. Yeah. I'm I'm writing the comments in the private chat. You've already done it for me. Perfect. Thank you, Leo. You're welcome. Those file there, or they are in Etsy Falco? I don't remember. Alright. Etsy. They are in Etsy, not Etsy Falco. Okay. So we're adding two Kube API server arguments. One of them is audit policy file, which goes

55:07 to the YAML file we got from the Sysdig repository, the evolution repository. No. I I need I wait. Wait. Wait. This is wrong. We have to remove Kube API server arg and just leave dash dash eight audit policy file. Right. Yes. Yes. Yeah. Of course. So dash dash. Yeah. Exactly. Yeah. This was for k three s k three s. Okay. So we're at an audit policy, audit web config, and those go to the two YAML files. Done. Exactly. And then restart our System CTL. Demo load. So these are running in the cluster because they're static manifest. So if I just

55:52 run get pods dash a Oh, nice. You really broke it? I broke it. We have broke it. Alright. Alright. So the static manifest that this change would have been detected by the kubelet as then restarted them. So let's jump over to one of them and find out what is going on here. So I Very cool. I know that we will have broken it. Yeah. We have no no API. Doesn't work. No. Let's just confirm the two paths. So we said we had a webhook.YAML. Okay. Probably Yeah. We're passing There's no webhook config. It's .com. It's webhookYAML.

55:59 Troubleshooting Kube API Server Config

56:34 It's only webhookYAML. I will rename the file, honestly. Easy fix. So k. And then by magic, this is gonna come back any second there. It's gonna come back. Are you sure? I don't know. Don't know. It's still Kubernetes. And also Falco. So Let's restart our Kubelet. Alright. So something's still failing. So okay. So let's just confirm. We have webhook.catyamo. And we have Yes. Auditpolicy.yamo with a dash, which is auditpolicy.yaml. Okay. Those are just Are both yaml or y m l? Good question. Too many tabs. Too many tabs. One more. Yeah. .Ya yeah. They're okay. And do we have any

57:42 logs from the API server, like, anywhere? Yes. So varlog containers. Let's get our to the Qtube API server. Which one is the most recent? This one here. Okay. So auditpolicy.YAML does not No such file of that. So the name is not that. Alright. Audit. It's the same user. Okay. So, yes. Here's the challenge. Right? This is a static manifest. So we need to move that file, don't we? Maybe. We do. So let's look at the mode points. Let's just open. Oh, yes. Because this is in the container. It's not the right. Oh, right. Very good. I I love this.

58:52 I hate it. But, anyway, so let's so we need share the Oh, I need to do this in everything. Right? So Oh, it's very very good. Now so if the Kubernetes is not on there, it will not be able to just talk to Falco via +1 270001 also. Shall we switch over to the backup cluster? Or do you wanna or shall we stick with this? As you prefer. Fine. Do do. You are David. I'm just worried. So Okay. So let me explain the problem then to the people that are watching. What's happening here is our API server is running inside

59:35 of a container and not on a host. The challenge with that is these two files exist on the host file system. Now what we can do is modify the main points at the bottom. And if we do that, the API server will start. But then Falco still has to speak. Does Falco speak to the API server or the Kubelet? API server, but we can I think that we can figure it out? Right. Let's do it. So let's just I'm gonna move both of these files to somewhere that is a bit more friendly. So let's call this our

1:00:04 Falco two. That's horrible naming. No. Yeah. I'll do that. API server. And let's move our web hook and our audit fail into the API server. Let's modify our static manifest now and add that as a mount point. So cube API server, jump down the bottom. Let's add a new volume, host path. Mhmm. Path slash etcetera API server and type directory or create a name API server config. Now we can go to volume mounts. Mount path, etcetera, API server. Read only true. And we need the name which I called API. API server config. I I love how you

1:01:02 remember all the Alright. So that is gonna make that available. Now we've one more change to make in this file, which is those arguments. So instead of here is etcetera, API server, API server. Oh, wait. Right? I also think that we need to change the The IP address in yeah. So it's advertising on the host IP. So we can just change. So we should be okay. And we can fix that anyway if we need to. What happened on these machines? We'll just ignore it. No. That's a static manifest which has now been reapplied. I'm gonna feel brave.

1:01:47 Because you can get notes. Yeah. Yes. Party. Party online long. It's less notes than before, but should be It's okay. No. It's I was joking. So we got our API server running with our Falco and audit integrations. We can celebrate, but that's only step one. So next, can we confirm that Falco can speak to the API server? Yeah. Let's open the Falco logs in one of the the in one of the nodes, and then we can try to do something evil and see what Falco thinks about that. Yeah. I see Kubernetes stuff. Yeah. So It's working. It seems to be

1:02:13 Confirming Kube Audit Events in Falco Logs

1:02:32 working. It seems. Let's now try. Yes. It's working. Yeah. I'm joking. We knew it was gonna work. Right? We knew. It's fine. I sent you a call, man. You can try to create a config map with the credential for, like, an AWS his an AWS access key. Okay. So I For example Well, yeah. Over to here. So this is me speaking to our cluster. We're creating a conflict map. We're calling AWS creds, and we're giving it something that looks like a secret. Right? Remove the backslash. Yeah. After AWS threads. Yeah. Oh, oh, yep. Yeah. Yes. So this should alert

1:02:41 Triggering a Kube Audit Rule (Secret in ConfigMap)

1:03:19 all the Falco instances at some point. It depends on which instance are you locking the Ah, Okay. So if I can't Try try to get logs again. No. Or or maybe just went into sea into the sea of the logs. So where do you want me to look? Sorry. We we want to see or or maybe it's not in this rule set that things I want to trigger. Okay. So we can check that as well. Right? So Yeah. If we do Kubernetes rules Yes. I'm assuming if I search for secret AWS. Yes. Oh, AWS rule.

1:04:16 Okay. So we have a macro at least. I'm saying private credentials. Yes. And we do have a rule that uses the macro as a condition here. I believe that there's some other rule that it's because the way events are triggered is that if the an event triggers a rule first, you're not going to see the other. So probably the way you create things is triggering the rules first. Like, maybe it's triggering someone some privileged user, the other the the other rule that you're seeing, right, if you show me the logs. Yes? Okay. So we wanna look at the

1:05:02 Falco logs again. Right? Yep. Yes. The cluster binding cluster binding cluster. Yeah. That's a lot of stuff. There's a there's a lot. Maybe it's even before these ones. Yeah. I think I want to see the you did journal CTL x a f. Let's try You, Falco. Okay. Let's have a look at it this way. I verified below. There's a lot. There's a lot. Let's see if I warning pods sensitive file open. Warning pod file. Yeah. There's a lot of stuff. Let me turn off the pager. Maybe grep for Grep for com AWS. Oh, AWS. No. It didn't show up

1:06:04 for some reason. Maybe that. I don't know. So can I just understand what's going on here? Right? Because currently, I am on the control plane node. Should all of the Falco's be alerted to this secret creation? That's a very good question. It should not be like that because you don't have the API server on on all nodes, but you have to manifest on all nodes. So we have to go to the active master then. Right? Exactly. I thought you were in the active master. Well, when we restarted the API server, I have no idea which one picked it up,

1:06:40 so we need to look. Okay. Cool. Okay. So if I take a look here, let's see if I can see the IP address. Because they're only the masters that they create that they send the events. Right? So this one actually potentially is the master, but what I will do is open the other two. And If you still get notes dash all wide should should be useful. Okay. Get note. Get notes all wide. Let's just grab these IPs. Or maybe it doesn't work. Right? Just It does work. I know it works. We just need to find the correct API server.

1:07:30 Now that I know how it's working, I'm I'm confident we can get the logs. Okay? You can insert your password with your fingerprint. Yeah. SE key is fantastic. Amazing. Or or if you're on Linux, there's a similar one too. But I have to use Mac as my new role, which I'm hoping changes soon. There we go. Yeah. I knew it worked. I mean, I didn't dream it. I mean Right. We tried this, like, ten hours ago, so nothing changed in Falco in the past ten ten hours. So Alright. That was One good one good improvement

1:07:51 Audit Log Entry Found

1:08:07 to Falco that we could do is we don't print out the credential logs. Yeah. I mean, otherwise, the this is very useful. So But this is a rule change. It's not a software change. Like, the person that made the rule explicitly decide to print out the credential. Yep. So it's not so maybe we can we can make a longer change that if we if we find the credential, we just hide it, but it's a bit more complicated than that. Okay. So that was really cool. I'm gonna try and summarize this. Right? So we modified the API server on our control plane with

1:08:48 two flags, audit policy and audit webhook. We injected some pre canned configs that we got from the Falco repository, that's evolution repository here. Yep. So we just copy and pasted that. We don't really do anything else except change the change the Falco IP just to be local host. And by that, we get all of the Kubernetes events going to Falco where the rules kicked in, detected a privacy and logged that. Now we had a better struggle finding that, but normally in a production scenario, that would be getting centralized anyway. I'd see that in Elasticsearch. I'd see that in Splunk. I'd see that

1:09:20 wherever I'm storing all my logs and that would be visible front and center. That's awesome. So, yeah, in a normal production scenario or in a development because, you know, development machine only have one Kubernetes node. In a production scenario, you have the centralized locking. So when this kind of when you do this kind of demos, you'll never remember that kind of details. But Yeah. I should have spun up a single master control plane. I I probably went a little bit over the top with my HD setup. But A little just a little bit. It's very cool to see how you

1:09:51 over engineered this demo. Yeah. You should see how many cores I've got on this thing and how much RAM as well for a cluster I'm spinning up for an hour. But that's the joy of bare metal. Like, it's great how much power I can throw at this. Right? Anyways, David, since we are since you mentioned the evolution repository, I think it's could be useful for the audience and people interested in Falco to know that this repo evolution is a simple repo where we basically try to keep all the things the getting started guides for users. For example, you can find here

1:10:05 Falco Evolution Repository (Examples)

1:10:28 examples manifest for deploying your first Falco in your Kubernetes. So there were also m charts here, then later we will bring the in the charts repository. Various examples, third party integrations, things like that can be very useful for people that just want that loves to approach a new a new project just by trying it without having to study all the the moving pieces of Falco. Without a PhD in a for. Alright. So I'll add that to the show notes. This this repository is definitely worth checking it then for anyone that is gonna start playing with Falco. Excellent.

1:11:11 Is there anything else you wanna show off, cover, after that before we finish up for this afternoon? Is there anything that we've not covered that makes sense? Oh, yeah. That's the thing. Go to Falcosecurity/ p dig. Yep. And that's also in BPF if we want to. Because we run Falcon now with the current model, not with BPF probe. Okay. Yeah. Maybe we don't want to run to restart Falco to run with BPF for now. So sorry. What what what should I go to here? I I just want maybe to talk for ten seconds about our user space efforts

1:11:39 User Space Driver Efforts (Pdig/Ptrace)

1:11:47 for with the pdig repo. Search for pdig. Oh, pdig. Thank you. Yeah. P dig. P dig. Yes. So, just a very underrepresented repository in our community. But For now. For now. Yes. So we we told about the fact that we, you know, either run with a kind of module or a BPF probe. Right? And that's a very good thing, you know, to have, like, because, you know, you can start kind of modules that you can use in a BPF probe. But that doesn't work in every environment. Yeah. For example, words are very good on bare metal,

1:12:28 but, like, it doesn't work on sandbox environments. For example, if you put that, you know, on Fargate environment or something like that, I just the only one that came in mind. Or you put, like, on a device or or on something like that. So you need something that doesn't rely on you know? The possibility for to assess the the kernel. Yeah. And so we started working on an interface, which is the user space API, which is not very well documented yet because it's just a ring buffer in memory that's shared between processes. But that one allows us to send events

1:13:12 input events to Falco, and one implementation of that is PDG. So PDG just uses p trace to send events to Falco. It's not as performant as Falco, as the Falco kind of module. It's not as performant as the eBPF probe, but it's a way to show that Falco can accept different kind of events coming from different sources that are not just, you know, those two that we are deploying production right now. And, also, this open, you know, a world of possibility for different kind of devices, like, you know, you want to play Falco on Mac OS or

1:13:47 Windows, something like that, you have an interface. And you can to different kind of inputs. Maybe maybe you could our our goal here is to create in the future, we are working on a an interface so that the community can create, for example, another inputs for Falco that this time is not Kubernetes audit logs, but some some other kind of audit logs without naming names. I don't know. But just, or other other kind of events, whatever. And so, in that in that way, user then could write will be able to write rules to to detect

1:14:31 even some events and trigger alerts on some events depending on this new input. I I found this I found this useful when I was trying to make Falco work on Arch 64. I was using Equinix for starting a very beefy machine because, you know, I had some I I didn't really have a Raspberry Pi. I didn't want to wait for it to compile Falco because it already takes a lot of time. Like, you compile open SSL and other things. So, it takes a lot of time. And you don't want to wait. So I just started that machine, but then I didn't have

1:15:10 a driver because our kind of modular VPF Pro are not made for r 64. They are made for x 86. We ported that to r 64 now. But when I want to just, you know, see, hey. This thing works, I was kind of I can just use PD because it's not, I can just use the user space interface because it's not architecture dependent. And, you know, that will work about very well. For so also for, you know, porting to new architectures. Nice. That's it. This is a very cool improvement that we did over the last year.

1:15:44 I mean, thanks to the whole team and overall to Lawrence, which actually made the code happen. Awesome. That's cool. That's good that people have that choice then depending on the environment that they're running. They still got an option to instrument and understand their applications using Falco or that's awesome. Nice one. So we have a couple more comments. Our mutual friend, says, Ciao. Hi. And we got a question from, is there a GUI? I mean, I'm happy to give you what I think the answer is, and then you two can correct me if you were to.

1:16:12 GUI Options (Commercial Products)

1:16:21 Yeah. Please. I'm gonna assume there there's no GUI. From what I understand of what you've you've explained to me today about Falco is that it's a detection system that events, and we should be storing those events somewhere where we have the ability to query them and notify on them and alert on them. And that may be something like Elasticsearch, and FoxDB, or even just Sesslog. But from there would be the interface. Is that is that correct? Exactly. Yeah. The the answer is correct. There's a GUI which is commercial, which is developed by Sysdig, but it's not, you know, Falco. It's, you

1:16:56 know, a product of Sysdig, which is called Sysdig Secure, which also has a query for Falco. Yeah. But it's just you know, they're it's one of their additional values on top of Falco. I want to I want to be clear with the audience here. Since they paid us a salary, I don't know why, honestly, but but Falco is a CNCF incubated project that plans to graduate next year, and that's being donated to the CNCF. Maybe we missed a bit of history of Falco, so let me just be brief about it. But Falco's graduated was sent back in

1:17:19 Falco's CNCF Status and Community

1:17:33 02/2018, was initially started at Falco in 02/2016 as an open source project. But after that, it has been donated to CNCF. Basically, we are CNCF maintainers, which means that since the engineering in any way can interfere with the plans that we have with Falco, which is, by the way, something that I love because it means that I can do whatever the hell I want. So, yes, there's a commercial GUI. The probably, are also other commercial GUIs and that will be out there. But this is not a problem for the Falco maintainers, for Falco community Exactly. Because

1:18:14 our goal is to make Falco consumable by the cloud native community, and this is the reason that we are working on APIs for rules that that watches rules that can changes, APIs for user space inputs, and things like that to make it more inter integrated with the the whole cloud native ecosystem. That's the goal. Sweet. So to summarize that again, you should be responsible for storing the events wherever you want and visualize them that way. But if you wanna support further development of Sysdig and their time to work on this continually and you're happy to pay, you should look

1:18:50 Conclusion and Thank You

1:18:53 into Sysdig Secure, which does it all for you. That sounds awesome. And, of course Yeah. Or or you can pay someone else that does the same thing, but it just just because Susie made a product out of it. Alright. Well, I just wanna say thank you to both of you. This has been a very great learning experience for me as even though we had a couple of small challenges, we overcome it. We've seen Falco running on our bare Linux system and side of our Kubernetes cluster. We saw those events. They're gonna help me provide secure platforms for my Kubernetes

1:19:25 stuff. So that's awesome. Thank you again for joining me. Thanks, David. Any closing thoughts before I before we say goodbye? Yeah. I'll be all set up, David, and your music. Me too. I also have another another thought. Don't we have five minutes to try VPS driver? I'm joking. I don't think we do. I'm I'm just joking. We can always have another session. Don't worry, Luca. We we can come back anytime you want and look at more stuff here. We can have a session on debugging why the PPI driver doesn't work on Keras 5.9. With Clang 10 or things like that. Okay.

1:20:00 Let's do it. Do you have GDP on your Mac installed and everything? We'll figure it out. Not today. Alright. Thanks again. Have an absolute great day, and I'll speak to you both. Thanks for inviting us. Thanks for asking us. Cheers. Bye bye. Ciao. Bye bye.