Threat Hunting

Threat Hunting on Endpoint Data with Sysmon

A practical guide to setting up and hunting with Sysmon, Elasticsearch, and Kibana

Founder / CEO

January 28, 2019

12 min read

How To Hunt on Sysmon Data

If you need a primer on Endpoint data, what it is and why it matters, check out this post. Otherwise, read on. In this post I'm going to specifically tackle the topic of Threat Hunting on Endpoint Data. I'll use Sysmon as a source of endpoint data in the examples, mainly because it's free and fairly easy to get started with.

I'll go into detail about Sysmon setup and configs in a future post, but you can still give it a try with a basic config and do some really effective hunting.

Basic Setup

To get started hunting, you need some data to hunt on (Sysmon) and, ideally, you need a place to store it that allows for fast and efficient search. For this I'd recommend Elasticsearch and Kibana. You can also use Splunk, but I'd prefer to steer you toward free tools just because they lower the bar for getting started.

The other piece you'll need is a way to send the Sysmon events to Elasticsearch. For that I'd recommend Winlogbeat, made by the same company as Elasticsearch, creatively named Elastic.

Elasticsearch Setup

Install Elasticsearch somewhere on your network that your Windows endpoints can reach. You could even do it right on your computer if you're just going to hunt on that. You can find the documentation here.

You'll also need to install Kibana, the front end to Elasticsearch. You can install it on a different box, or on the same one you installed Elasticsearch on. If you're only going to hunt on a handful of Windows endpoints or less, then installing Kibana and Elasticsearch on the same box is perfectly fine. You can find the documentation for Kibana here.

I'm not going to go into detail on getting these installed because there are just too many ways to do it and Elastic's documentation is pretty decent. If you are able to get these installed and working, then the rest of this how-to should be pretty easy.

Sysmon Setup

If you haven't already, download Sysmon.

Install it with the following command:

sysmon64.exe -i -accepteula –h md5,sha256 –n

Go ahead and install Sysmon on several Windows endpoints, if you have them. Hunting is a lot more fun and interesting when you're dealing with multiple endpoints.

To confirm that Sysmon is running, you can run the following command:

sc query sysmon64

Winlogbeat Setup

Install and configure winlogbeat on all the endpoints that you installed Sysmon on. Rather than regurgitating the documentation for Winlogbeat, I'll just point you to it here. One thing not listed in the documentation is the config line to pull Sysmon data from the Windows Event log. Here it is:

winlogbeat.event_logs:
  - name: Microsoft-Windows-Sysmon/Operational

To send the events to Elasticsearch, you simply need the following in the config:

output.elasticsearch:
  hosts:
    - localhost:9200 # Use the URL of your Elasticsearch server that you just set up

There are other things you can do with the Winlogbeat config, such as adding or removing fields, but for now this is all you'll need.

To confirm that Winlogbeat is running, you can run the following command:

sc query winlogbeat

Verify that the state is listed as running.

Confirm Setup

Once Sysmon and Kibana are successfully installed, configured and running, you should start to see events in Elasticsearch.

To confirm that Elasticsearch is getting data, point your browser at the url of your Elasticsearch installation (not Kibana) with /_cat/indices on the end of the URL, for example:

http://localhost:9200/_cat/indices

This will give you a text listing of the indices currently in Elasticsearch. If you don't see one, then your data isn't making it to Elasticsearch. This is typically because of a misconfiguration of Winlogbeat, or a networking problem that prevents traffic from your endpoints from reaching your Elasticsearch server.

Once you confirm that data is making it to Elasticsearch and you see an index listed at the _cat/indices URL, then you can point your browser at the Kibana URL to load up Kibana. Follow these initial setup steps:

Navigate to the management page
Click on index patterns
Click Create Index Pattern
Follow the prompts to create a new index pattern that matches the index name you saw in the index listing previously

Now go to the Discover tab in Kibana and you should see some events. If you don't, then you have some more troubleshooting to do. If you do see some events, then you're ready to go on to the next section.

Start Hunting

Ok, so now you've got data that you can see in Kibana. Nice work! Now the hunting begins.

Orient yourself to the data

The best way to start is to first get acquainted with the data (and Kibana). In Kibana, the data is hard to read unless you do some tweaking. To start, do a search in the Kibana search bar for:

event_id:1

This will filter the list of events to just Process Creation events, meaning events that were triggered when a program was launched. Click on the little arrow next to one of the events to expand it. Look through the fields in the event so you can start to understand what fields exist for a process create event. The primary field of interest is Image. Click on the little column looking icon by Image to add that field to your overall results table in Kibana.

Another interesting field is CommandLine. There you'll see the exact command line that was used to launch this process.

Quick note on field names: Some of the field names I'm referencing will be a subfield of event_data. If you can't find a field I'm referencing or a search isn't working, make sure you prepend event_data. to the field name when you reference it, for example: event_data.Image

Another set of fields that makes Sysmon really unique and powerful are the Parent fields: ParentImage, ParentCommandLine, etc. These are interesting because they give us easy access to the most basic behavior we can hunt for, which is programs launching other programs, or more technically parent-child relationships between programs.

Start Searching

You can search for any value in any field. Some of the fields you might have discovered previously are event_id, Image, CommandLine, ParentImage, ParentCommandLine. For network connect events (event_id 3) some of the interesting fields are DestinationIp, DestinationPort and DestinationHostname.

You can filter your results in Kibana by specifying a fieldname, followed by a colon, followed by a search term. Here are some examples:

Show me all Process Create events:

event_id:1

Show me all Network Connect events:

event_id:3

Show me all events that Google Chrome generated:

Image:*chrome.exe

Show me all programs launched from a command shell:

ParentImage:*cmd.exe AND event_id:1

Show me where Chrome launched a command shell:

Image:*cmd.exe AND ParentImage:*chrome.exe

Also, in case it isn't obvious, you can adjust the time range of your search in the upper-right hand corner of Kibana. I typically do the last 4 hours unless I have a specific time range I'm hunting on.

Next Steps

Ok, now you're hunting. You may not know what you're hunting for yet, but as you get more familiar with the data and develop a plan, you can start to focus your hunting efforts. Depending on your environment, below are some things you might want to consider adding to your hunting regimen.

Webservers Launching a Shell

Search for a command shell, like cmd.exe, with a webserver, like w3wp.exe, as a parent. This would be indicative of a webshell attack, often used as a mean of persistence by advanced attackers.

Svchost Launching a Shell

Search for a shell with a parent of svchost.exe. This is weird, and probably not good.

Unusual Parent of Svchost

Search for svchost.exe where services.exe is not the parent. You can see from our Insights data here that services.exe should be the parent of svchost.exe. Anything else is anomalous and worth investigating.

Powershell Encoded Commands

Search for powershell with a command line argument of -enc. PowerShell has a feature allowing for commands to be encoded with base64 encoding. While this is sometimes used for legitimate purposes, it can also be used by attackers to obfuscate their activities.

Logon Bypass

Search for a shell, like cmd.exe, being launched by winlogon.exe. This is indicative of a Sticky-Keys attack where an attacker is able to create persistence by modifying the registry to launch a command shell at the logon screen when they hit the shift key 5 times. This is bad.

Powershell Download

Search for powershell with the strings DownloadString or DownloadFile in the command line. This is similar to encoded command in that you might find some legitimate usage, but it is often used by attackers to pull down commands or files from a remote server to achieve their objective.

Ok, that's all for now. There are many more things that we hunt for, but that should get you started. I hope this has been helpful in giving you some understanding of the possibilities that open up when you have access to endpoint process behavior data like that which Sysmon provides.

There is so much more I could talk about both in terms of Threat Hunting as well as the best way to use Elasticsearch and Kibana. I'll save that for future posts, but please reach out via Twitter or any other means to let me know what you'd like to hear more about.

Happy Hunting!

Want to enhance your threat hunting?

Explore EchoTrail's process behavior data to improve your hunting techniques and detection capabilities.

Explore Process Insights