Hunting for IOC's with ioc-parser
Threat intelligence became a hot topic for a while. The food of threat intelligence is based on IOC's (Indicators of Compromise) which contains technical information like:
- Files, path
- Hashes
- IP addresses
- Domains
- Users
Mixed with other sources of information or tools, they help in detecting malicious behaviors of programs or networks. They are plenty of sources to collect IOC's. Some are publicly available while others are compiled and maintained by organizations for their customers or restricted users. DShield is of course a good source of IP addresses but Lenny (another ISC handler) is maintaining a nice list of resource on his website(1). Usually, free services offer lists of IOC's in common format that are reusable in your own environment. But sometimes, you will find interesting information published online. Many security researchers analyze pieces of malware and publish the results on their blog. Big organizations like to publish nice PDF reports containing juicy information. In both case, IOC's can be present but how to extract them automatically?
ioc-parser(2) is a nice Python script which might be very helpful in this case. It parses an input file and generates a list of IOC's in another format. It supports the following input formats: Text files, PDF files or HTML (URLs). Results can be generated in CSV, JSON, YARA or NetFlow. The idea is simple, it searches for patterns based on regular expressions. Everything is configurable and your own regexp can be added.
Here is the list of IOC's extracted from an old PDF report about Duqu 2.0 written by Kasperky Lab:
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 2 Filename msi.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 2 Filename klif.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 2 Filename 12CTwoPENC.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 2 Filename KMART.dll
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 2 Filename portserv.sys
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 3 URL https://en.wikipedia.org/wiki/Duqu
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 3 URL http://www.kaspersky.com/about/news/virus/2011/Duqu_The_Step_Brother_of_Stuxnet
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 3 URL http://70.auschwitz.org/index.php?lang=en
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 3 Host 70.auschwitz.org
The_Mystery_of_Duqu_2_0_a_sophisticated_cyberespionage_actor_returns.pdf 3 CVE CVE-2015-2360
But you can access URLs directly and extract IOC's present in the HTML code of the latest MalwareMustDie blog article:
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 URL http://www.blogger.com/go/cookiechoices
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 210.92.18.118
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.25
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.13
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.15
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.10
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.162.175
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.14
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.100
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.103
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.100
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.102
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.113
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.105
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.118
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 101.199.109.151
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.162.174
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.162.178
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 15.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.160.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.8
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 8.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.162.176
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 176.162.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 14.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.101
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.176.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.102
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.104
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.9
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 9.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 174.162.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.115
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.38.187.116
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 101.199.109.144
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 101.199.108.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.29
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 29.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 178.162.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.92
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 92.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.90
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 90.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 106.120.167.86
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 86.167.120.106
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 210.92.0.0
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 IP 222.186.34.220
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 360.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host qi89.f3322.org
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host qurl.qh-lb.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host qup.qh-lb.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host sdupm.360.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host sdup.360.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host sdup.qh-lb.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host flux.sh
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 15.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host chinatelecom.com.cn
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 176.162.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 14.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 174.162.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 29.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 178.162.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 92.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 90.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host 86.167.120.106.static.bjtelecom.net
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host dshw.co.kr
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host f3322.org
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host astpbx.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host libworker.so
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Host www.blogger.com
http://blog.malwaremustdie.org/2015/09/mmd-0042-2015-hunting-mr-black-ids-via.html 1 Email ppyy@astpbx.com
And the same results generated in YARA format:
rule mmd_0042_2015_hunting_mr_black_ids_via
{
strings:
$URL1 = "http://www.blogger.com/go/cookiechoices"
$IP1 = "210.92.18.118"
$IP2 = "106.120.167.25"
$IP3 = "106.120.167.13"
$IP4 = "106.120.167.15"
$IP5 = "106.120.167.10"
$IP6 = "106.120.162.175"
$IP7 = "106.120.167.14"
$IP8 = "106.38.187.100"
$IP9 = "106.38.187.103"
$IP10 = "106.120.167.100"
$IP11 = "106.38.187.106"
$IP12 = "106.120.167.102"
$IP13 = "106.38.187.113"
$IP14 = "106.38.187.105"
$IP15 = "106.38.187.118"
$IP16 = "101.199.109.151"
$IP17 = "106.120.162.174"
$IP18 = "106.120.162.178"
$IP19 = "15.167.120.106"
$IP20 = "106.120.160.0"
$IP21 = "106.120.167.8"
$IP22 = "8.167.120.106"
$IP23 = "106.120.162.176"
$IP24 = "176.162.120.106"
$IP25 = "14.167.120.106"
$IP26 = "106.38.187.101"
$IP27 = "106.38.176.0"
$IP28 = "106.38.187.102"
$IP29 = "106.38.187.104"
$IP30 = "106.120.167.9"
$IP31 = "9.167.120.106"
$IP32 = "174.162.120.106"
$IP33 = "106.38.187.115"
$IP34 = "106.38.187.116"
$IP35 = "101.199.109.144"
$IP36 = "101.199.108.0"
$IP37 = "106.120.167.29"
$IP38 = "29.167.120.106"
$IP39 = "178.162.120.106"
$IP40 = "106.120.167.92"
$IP41 = "92.167.120.106"
$IP42 = "106.120.167.90"
$IP43 = "90.167.120.106"
$IP44 = "106.120.167.86"
$IP45 = "86.167.120.106"
$IP46 = "210.92.0.0"
$IP47 = "222.186.34.220"
$Host1 = "360.cn"
$Host2 = "qi89.f3322.org"
$Host3 = "qurl.qh-lb.com"
$Host4 = "qup.qh-lb.com"
$Host5 = "sdupm.360.cn"
$Host6 = "sdup.360.cn"
$Host7 = "sdup.qh-lb.com"
$Host8 = "flux.sh"
$Host9 = "15.167.120.106.static.bjtelecom.net"
$Host10 = "chinatelecom.com.cn"
$Host11 = "167.120.106.static.bjtelecom.net"
$Host12 = "176.162.120.106.static.bjtelecom.net"
$Host13 = "14.167.120.106.static.bjtelecom.net"
$Host14 = "174.162.120.106.static.bjtelecom.net"
$Host15 = "29.167.120.106.static.bjtelecom.net"
$Host16 = "178.162.120.106.static.bjtelecom.net"
$Host17 = "92.167.120.106.static.bjtelecom.net"
$Host18 = "90.167.120.106.static.bjtelecom.net"
$Host19 = "86.167.120.106.static.bjtelecom.net"
$Host20 = "dshw.co.kr"
$Host21 = "f3322.org"
$Host22 = "astpbx.com"
$Host23 = "libworker.so"
$Host24 = "www.blogger.com"
$Email1 = "ppyy@astpbx.com"
condition:
$URL1 or $IP1 or $IP2 or $IP3 or $IP4 or $IP5 or $IP6 or $IP7 or $IP8 or $IP9 or $IP10 or $IP11 or $IP12 or $IP13 or $IP14 or $IP15 or $IP16 or $IP17 or $IP18 or $IP19 or $IP20 or $IP21 or $IP22 or $IP23 or $IP24 or $IP25 or $IP26 or $IP27 or $IP28 or $IP29 or $IP30 or $IP31 or $IP32 or $IP33 or $IP34 or $IP35 or $IP36 or $IP37 or $IP38 or $IP39 or $IP40 or $IP41 or $IP42 or $IP43 or $IP44 or $IP45 or $IP46 or $IP47 or $Host1 or $Host2 or $Host3 or $Host4 or $Host5 or $Host6 or $Host7 or $Host8 or $Host9 or $Host10 or $Host11 or $Host12 or $Host13 or $Host14 or $Host15 or $Host16 or $Host17 or $Host18 or $Host19 or $Host20 or $Host21 or $Host22 or $Host23 or $Host24 or $Email1
}
This is a nice script to keep in your personal toolbox. Of course, be careful to not re-use the generated data "as is", there could be false positives or bad regular expression matches.
Happy IOC's hunting!
Xavier Mertens
ISC Handler - Freelance Security Consultant
rootshell.be
truesec.be
(1) https://zeltser.com/malicious-ip-blocklists/
(2) https://github.com/armbues/ioc_parser
Reverse-Engineering Malware: Malware Analysis Tools and Techniques | Frankfurt | Dec 9th - Dec 14th 2024 |
Comments
https://github.com/kbandla/APTnotes
and IOC exports available:
http://aptnotes.malwareconfig.com/ioc_export.csv
http://aptnotes.malwareconfig.com/ioc_export.json
http://aptnotes.malwareconfig.com/yara_export.yar
>26K IOCs ready to use :-)
Cheers,
Tom
Anonymous
Sep 7th 2015
9 years ago
Anonymous
Sep 7th 2015
9 years ago
Anonymous
Sep 7th 2015
9 years ago
Anonymous
Sep 7th 2015
9 years ago
For domain names & URLs, be ware that reference sites such as sans.org fireeye.com, etc... are also in the feed. So you might get lots of false positives, depending on what your users are browsing.
Anonymous
Sep 8th 2015
9 years ago
I need to get the IOC's out of FireEye (web) - any ideas? The IOC's in fireeye are called "OS changes" and it has its own format (none OpenIOC nor XML) - we can run a report and get a PDF out of it but that's it.
Please advice.
Anonymous
Sep 8th 2015
9 years ago
Anonymous
Sep 8th 2015
9 years ago
The other thing I've found out is that we can also create reports in XML - the alert details report, this way we can extract the IOCs out that xml report. I figured it should be easier than a PDF report.
Chris M.
Anonymous
Sep 9th 2015
9 years ago
The PDF export is not doable because strings are not presents "as is" in the document.
I generated a XML report for the last 3 months. ioc-parser extract lot of stuffs but there are also a lot of false positives.. (The XML containing not only IOC's)
Anonymous
Sep 11th 2015
9 years ago
Please share the link.
Thanks
Sudhanshu
Anonymous
Nov 13th 2017
7 years ago