0:00 Introduction and Overview

0:00 Hi there. Welcome back to the Rawkode Academy. I'm your host, David Flanagan, also known as Rawkode. Today, we continue the complete guide to Kubescape. I wanna say thank you to the team at ARMOR for sponsoring my time to put this course together. Kubescape is a phenomenal tool that helps us improve our security posture and our Kubernetes cluster. Do I think it's hyperbole when I say every cluster should have Kubescape installed? No. So let's take a look at how we continuously monitor our security posture. When you run Kubescape, you get the ARMOR cloud dashboard. You can go and see

0:35 Kubescape Output Formats

0:43 the report and results of all your scans. But we're developers and operators and we like Grafana and dashboards and metrics. So how do we get our Kubescape control failures into a format that we can work with with our automation? Well, as we've seen in a previous video, the Kubescape CLI accepts a format parameter. We took a look at a JSON. There's also HTML, pretty print, and JUnit. The last option is Prometheus metrics. So first, we're just gonna take a very quick look at the Prometheus metrics output. Then I'll show you how we do it in production.

1:23 Production Monitoring Setup Overview

1:34 We run an operator and our cluster that uses Prometheus service monitors, speaks to Kubescape, gets a result of our scans, writes them to Prometheus, and to finish it off, there's even a Grafana dashboard to take a look and understand what's going on. So this will be a quick video, but there's a lot to love. So let's take a look. So the first thing that we want to do is run a Kubescape scan dot. This will scan our local directory, find Kubernetes manifests, and give us a report. As you can see, we've got resource limits that need specified

2:02 Kubescape CLI Prometheus Output

2:19 and oh, some host page IPC and network privileges. Well, that doesn't sound too great, does it? And of course not. We take a look at the YAML. I have enabled these three things. Why? Just to give us something to visualize on a dashboard as we improve our security posture. So let's see the Prometheus metrics output from the CLI. We can run the scan again. This time we set format equals Prometheus. And as you can see, we get a whole bunch of Prometheus style metrics written to the terminal. You could script them into Prometheus somehow, but there is a better way.

3:16 Setting Up Cluster Components

3:16 So the first thing I'm going to do is run this just target called helm. This is gonna set up all the repositories that we need for today's session. Next, let's list the targets. We have one for helm that we just run. We have installed Grafana, installed Kubescape, installed Kubescape Prometheus, and installed Prometheus. So let's do Prometheus first, All by Grafana. All by Kubescape. All by Kubescape. Prometheus. And then by the power of video magic, we will have a complete setup. Now before I click my fingers, the just fail is here and available via the repository

4:10 on the Rawkode Academy. The link is in the description. Go check it out if you wanna see what's happening. Alright. Let's run kubectl get pods all. We have Grafana. We've got Kubescape, Kubescape Prometheus, and Prometheus. So what's happening right now is that the Kubescape operator is running in our cluster. It's continually scanning and and monitoring our resources. Let's run kubescape control minus n kubescape Prometheus get service monitors. And we can see that we have a service monitor and a namespace. So, let's browse to Prometheus. We can run kubectl minus n, Prometheus port forward and we're gonna go to the

4:57 Exploring Metrics in Prometheus UI

5:04 service, kubescape Prometheus stack. Prometheus on port nineteen ninety. So we pop up on our browser and we have the Prometheus UI. We can zoom in on this and we can start to type Kubescape as a prefix. From here, we can select our cluster risk store and hit return. And we'll see that we have a score of 12. So let's take a look at one more metric. Kubescape framework can't failed. Now we have a breakdown of the number of failures per framework. If I zoom in on this more, you'll see all control 36, ARMOR best 22, DevOps best nine.

5:51 Visualization with Grafana

6:00 Now this is a fantastic resource of information, but it's not readily consumable in this format. Right? This is why we have Grafana to help us visualize our dashboards. Again, as I said in an earlier video, it's through visualization that we build understanding. So let's go check a look at Grafana. We can run kubectl namespace Grafana get secrets. Why? Well, because Grafana helm chart generates a random password and we'll need that to log in to the instance. We grab the Grafana secret, a YAML, and we run a base 64 decode of this value here. And now we have our secure

6:24 Accessing the Grafana UI

6:50 password. Now we can run kubescape control namespace Grafana port forward for service Grafana. And we'll expose this on port 3,000, although the service does listen on port 80. Now if we go to 3,000, username is admin and the password is the one we got from the secret. And now we have a Grafana dashboard. So we need to go to settings and data sources where we can add our Prometheus source. Now we're just used in regular Kubernetes service discovery to hook this up. That means that we use the service name, which was kube stack Prometheus. The namespace,

7:19 Connecting Grafana to Prometheus

7:40 which is Prometheus.sec.cluster.local, and the port is nineteen ninety. We click save and test, and it tells us as an error. Your Prometheus, stack Prometheus, and that is working. Alright. Let's go to the explore tab and take a look. Use the Kubescape prefix, and you can see that we have access to the cluster account based on resources and controls. We have a cluster risk score, which we can run. And we'll see as remain pretty static. That makes sense. Bring back our Kubescape prefix, and then we have a control account resource and a framework account resource, but not at

8:00 Exploring Metrics in Grafana

8:30 the cluster level. So probably broken down by namespace, and, of course, our risk score here. Let's take a look at Kubescape framework count control failed. This should give us a breakdown of the different controls by a namespace that have failed. Assuming if we go to name, we can filter by the framework. So we can see ARMOR best like so. Let's try one more. This time, Kubescape control resource failed. And now we can filter this by name, and we're going to find the host PID IPC privilege. Like so. Now we can see that this is slowly

9:29 claimed as I've deployed some example resources to the cluster. So let's pop back to our YAML, and we'll remove two. Like so. Let's look at the last five minutes and run the query. Alright. So after a little bit of time, we see that our resource failure dropped from three to two. Now Grafana is a great tool. Prometheus is a great tool. I've been able to query individual metrics and view them as a graph is also great. Sometimes you just need a dashboard. We need to see multiple queries, multiple metrics, and how they change together over time.

10:29 Importing the Grafana Dashboard

10:30 So let's run the download dashboard, just target. From there, we can cut the dashboard and copy it to our buffer. We can then go to the Grafana homepage, import dashboard where we could paste in the JSON. From here, we click load and we import. Let's change the time to five minutes. Now there is a bit of a small bug with Grafana and that our dashboard, no matter how many times we click refresh, won't seem to load. However, I found if you just go to edit and multiply the query ever so slightly and click run, it seems to jog it back into touch.

11:15 However, we do need to do this for all the panels. Like so. So while we've only changed one resource, we've not really affected our cluster risk score, and that is kind of expected. And you can monitor your controls over time. As we can see from this, we have a lot of controls, and we can see there has been a change, but it's difficult to pinpoint which. Now we know which control was affected, so we can scroll down the alphabetical list until we find host. Host. Host. Host. Lots of controls. There we go. It click host path to IPC privileged.

11:49 Exploring the Imported Dashboard

12:27 And now we can see that drop on the resource over time. On the right, we can see our framework risk score over time, which again much like the cluster score is unlikely unlikely to change by fixing one resource. And then we have a good overview of our two pie charts. We can see the resource status and control status, but this time not over time, more as a an overall picture of our security posture. So we could see actually that 74% of our resources are passing their controls. So we actually don't have too much to do.

13:03 Conclusion and Next Steps

13:04 However, when we go over to the control side, we can see and I'm not sure why it's green, the 43% of our controls have a resource that are in violation, which is why we see so much activity on the control risk score. So now we have all the information that we need to do two things. One, monitor and observe our security posture. You might be in good shape now, but as developers are deploying to your cluster, these things change over time. So it's great to have Kubescape in your cluster with Prometheus and Grafana. You can build

13:42 alerting rules just like you would for all the other parts of your infrastructure. And number two, you have a way to track and celebrate progress. Make sure you're identifying the controls that are important to your organization. Even build a custom framework. Go check out that video next. And that's it. I hope you've enjoyed this tutorial on Kubescape with Prometheus and Grafana. I'll see you for the next video soon. Have a great day.