Threat Intel Ingest

InsightIDR
1

Hello.

How is everyone ingesting TI into IDR? I did find a article written by R7 but the PowerShell scripts are out of date and some of the TI feeds are no longer active.

2

I also have a TAXII feed that I’d like to ingest but not sure on how to go about it.

3

interested as well

4

Currently I have an ICON workflow that runs daily and pulls IOCs from Threatfox that I can correlated to managed community threats based on the related_malware.

So the workflow goes through all of my managed threats (hosted as global artifacts) that I’ve categorized as Software, Groups, and Techniques and then if any of these have a related_malware value it calls an API to Threatfox looking for IOCs based on that malware. The IOCs are then filtered to make sure they’re both new and high-confidence.

I manage my threats based on MITRE identifiers, if available, or just based on the Software, Group, or Technique it’s affiliated to. But I’m primarily doing it this way because Threatfox is free and we currently don’t pay for any feeds. This does require me to occasionally pull IOCs from other sources manually and import them through another custom workflow I have.

1 Like
5

We have ICON and have not used it to it’s full capabilities yet as it req time which is slim. Do you have any good starting points to assist in pulling IOCs from Threatfox?

6

I’m going to just start with a disclaimer that I don’t consider myself experienced in this at all…I’m learning ICON, Python, and everything in between as I go…but am getting a bit more confident after making workflows over a few years.

Threat Structure

So first I’d consider how you organize your threats. Like I said I organize mine by Group, Software, or Technique since I try to map my threats to the MITRE framework wherever possible. Admittedly this isn’t the most logical since sources like UrlHaus should still be allocated somewhere even if it can’t be attributed.

I create a global artifact table that consists of my threats. This is where I host my threat_key values. The table also includes a related_malware string value which is actually a comma-separated list that can be converted to an array, along with other values which may or may not be applicable to your situation. The related_malware name is the actual malware name provided by Malpedia which is used by Threatfox.

Workflow

This is very brief overview of the workflow structure. I don’t want to provide an export or share it because mine does involve a lot of proprietary snippets/artifacts that would just not work elsewhere.

  • Trigger: Workflow runs nightly using the Timers plugin.

  • Lookup Threat: Use Helpers to lookup threats that contain “.” in the related_malware field since all malware names include this symbol.

  • Threat Loop: Create a loop to go through the matches from your lookup.

  • Type Converter: Convert the related_malware field from your threat object into an array.

  • Malware Loop: Create another loop to go through your related_makware array. If you don’t have multiple then you can just call the first value of the array instead.

  • Query Threatfox: Run an HTTP request against Threatfox. The API call structure is one of the easiest I’ve used. Adjust your limit as you see fit.
    {"limit":100,"query":"malwareinfo","malware":"{{["Malware Loop\"].[$item]}}"}

Output

This is where you just need to decide on how you want to handle the IOC results. I have created a Snippet that converts Threatfox IOC outputs into my own observable JSON objects which I used across my own workflows. I’ve found it very helpful to have a couple of snippets:

  • A snippet for converting Threatfox IOCs into your own format and for validating them. For example, this snippet could check to make sure the confidence level exceeds your own personal threshold.
  • A snippet for updating InsightIDR threats (since it’s something you’d probably like to have ready to go later). I built this because I also track each threat’s IOC count and the snippet updates this count and other values each time.

The simplest approach here for you would probably be just to create another loop that just goes through the IOCs and separates them into type (ie. domains, hashes, urls, ip_addresses) so that you can then just import the string arrays in the InsightIDR Add Indicators action.

I apologize if this response was too long or rambled on. Depending on your knowledge of ICON structure if you’d like more details on any of this I’m happy to help. Unfortunately a lot of my workflows and automation structure does include custom snippets, global artifacts, and my own indicator/observable JSON format so it can’t easily be shared.

7

I have powershell scripts that runs daily and pull down an *I-SAC TAXII threat feeds and uploads to a community threat and another threat feed that pulls down IOCs daily from a S3 bucket and uploads to a community threat in Rapid7. I can share my PS scripts if you want. Please PM me.

1 Like
8

interested please share not sure the is a DM on here

9

Hi @mwhite I’m not sure how to PM you. I was also trying to import TAXII feeds from *I-SAC. Any help is greatly appreciated.

10

Send me an email to matthew.white@hawaiipacifichealth.org for examples.

11

Sent. Thank you.

12

I also just emailed. Thanks