Blog Series - Using ChatGPT for cyber defence part 3

Part 3 - Helping SOC analysts with Mitre ATT&CK, TTPs and threat hunting ideas?

Getting to grips with how to best use Mitre ATT&CK within a SOC environment is a key challenge for most SOCs. Trying to cover all the TTPs is typically out of reach and analysts need to focus on the most productive ones. Creating sensible threat hunting strategies and hypothesis is another key task that can be bottomless, so can we use ChatGPT to help?

Here are some initial ideas and sample responses:

Let’s start by trying to focus on the most common TTPs…

Pic1

The response here is just a list of TTPs, and doesn’t really help us focus on anything specific, so we focused on Execution phase attacks:

Pic2

This response is more helpful as it provides some examples of Execution attacks, but we’re not convinced these are the most common. We then tried a different approach:

Pic3

This response is helpful but it’s not really the best advice, i.e., using deception technology is not a common way of detecting Execution attacks and we don’t want IOCs, we want TTPs so overall this isn’t very useful.

There’s a pattern here, the responses are useful but we’re not achieving our goal of focusing down on the most common techniques, at least not in any level of detail.

We try to be more specific with a focus on number six (in the list above), the 6th most common attack.

Pic4

This is a useful starting point. It needs checking and improving but it’s a reasonable start to documenting a hypothesis and then writing threat hunting queries.

Let’s ask for some more info so we can create a better use case:

Pic5

Again, reasonably useful but needs checking.

Let’s try a different approach:

Pic6

The above needs checking but is useful info.

Let’s see if we can get some actual IOCs:

Pic7

Not much luck with that but that’s as expected, so let’s keep focusing on PowerShell:

Pic8

That’s a decent starting point and provides examples.

We then ask for example reports that contain examples which should be essential reading to help wth our threat hunting:

Pic9

This is a good list to get started with. Can we now get some help with the actual threat hunting queries?

Let’s ask for sample kusto queries based on these DFIR reports:

Pic10

Not a bad set of threat hunting queries to get started with. The queries look correctly formed but as we have found many times, it doesn’t mean that they are correct or that they will actually run. Always check them and always test them – don’t just trust them and copy & paste!

Pic11

Let’s ask for some queries we can use with Microsoft Defender…

These queries are not correct as they contain errors, but they are good ideas to get us started and can be easily modified and tested.

One of the challenges for threat hunting and SOC detections is ensuring you are collecting the right data in the first place. For a final step, we explore whether we could use it to help with data collection. In this case, we ask it to generate a sysmon config that would ensure that the PowerShell commands are logged using sysmon. We haven’t had a chance to explore this further but on first pass, it’s a start!

Pic12

In this next example, we want to know what we need to do to ensure Windows Event ID 4104 is logged as there’s no point hunting and using it if we aren’t logging and collecting it.

Pic13

Good answer! By default it isn’t logged so if we were looking to threat hunt in Sentinel across our Windows estate using the queies from earlier, unless we have specifically setup PowerShell logging as suggested, we won’t have the data we need.

Conclusion

It’s true that we could have done most of this with traditional search engines but we think that would have taken longer. If you can write a good question, you can get useful information back. However, as we have seen in our other blogs on this topic, the technical aspects are often basic and vague, and sometimes incorrect. Always treat these responses as ‘maybes’ and you will be on the right path. What it is good at is helping with document ideas you have and suggesting ideas for new ones. We found it helpful to point us in the right direction and remind us of things we may have overlooked or forgotten about and present new ideas.

Rob Demain, CEO

Written on February 28, 2023