Alex Pinto on the intersection of threat hunting and automation

In this episode, I talk with Alex Pinto, chief data scientist at Niddel. We discuss the role of threat hunting in security, the necessity for well-defined process and documentation in threat hunting and other activities, and the potential for automating threat hunting using supervised machine learning.

Here are some highlights:

Threat hunting’s role in improved detection

At the end of the day, threat hunting is proactively searching for malicious activity that your existing security tools and processes missed. In a way, it’s an evolution of the more traditional security monitoring and log analysis that organizations currently use. Experienced workers in security operation center environments or with managed security services providers might say, ‘Well, this is what I’ve been doing all this time, so maybe I was threat hunting all along.’ The idea behind threat hunting is that you’re not entirely confident the tools and processes in place are identifying every single problem you might have. So, you decide to scrutinize your environment and available data, and hopefully grow your detection capability based on what you learn.

There are some definitions, which I’m not entirely in agreement with, that say that, ‘It’s only threat hunting when it’s a human activity. So, the definition of threat hunting is when humans are looking for things that the automation missed.’ I personally think that’s very self-serving. I think this human-centric qualifier is a little bit beside the point. We should always be striving to automate the work that we’re doing as much as we can.

Gauging success by measuring dwell time

It’s still very challenging to manage productivity and success metrics for threat hunting. This is an activity where it’s easy to spin your wheels and never find anything. There’s a great metric called dwell time, which admittedly can be hard to measure. Dwell time measures the average time for the incident response team to find something as opposed to when the machine was originally infected or compromised. How long did it take for the alert to be generated or for the issue to be found via hunting? We’ve all heard vendor pitches saying something along the lines of, ‘Companies take more than 100 days to find specific malware in their environments.’ You should be measuring dwell time within your own environment. If you start to engage in threat hunting and you see this number decrease, you’re finding issues sooner, and that means the threat hunting is working.

The environments where I’ve seen the most success with threat hunting utilized their incident response (IR) team for the task or built a threat hunting offshoot from their IR team. These team members were already very comfortable with handling incidents within the organization. They already understood the environment well, knew what to look for, and where they should be looking. IR teams may be able to spend some of their time proactively looking for things and formulating hypotheses of where there could be a blind spot or perhaps poorly configured tools, and then researching those potential problem areas. Documentation is key. By documenting everything, you build organizational knowledge and allow for consistency and measurement of success.

The potential for automating threat hunting

There’s a lot of different factors you can consider in deciding whether something is malicious. The hard part is the actual decision-making process. What really matters is the ability of a human analyst to be able to make a decision whether an activity is malicious or not and how to proceed. Using human analysts to review every scenario doesn’t scale, especially given the complexity and number of factors they have to explore in order to make a decision.

I’ve been exploring when and how we can automate that decision-making process, specifically in the case of threat hunting. For people who have some familiarity with machine learning, it appears threat hunting would fit well with a supervised machine learning model. You have vast amounts of data, and you have to make a call whether to classify something as good or bad. In any model that you’re training, you should use previous experience to classify benign activities to reduce noise. When we automate as much of this process as possible, we improve efficiency, the use of our team’s time, and consistency. Of course, It’s important to also consider the difficulties in pursuing this automation, and how we can try to circumvent those difficulties.