Shulou(Shulou.com)05/31 Report--

How to build a threat intelligence database, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

I. Preface

Threat intelligence is a data-based action that predicts upcoming attacks on an organization. Predict (based on data) the coming attack. Threat intelligence using publicly available resources to predict potential threats can help you make better defensive decisions. In enterprises, governments and even countries, it is very important to have a threat intelligence database with high accuracy and large amount of data.

II. Brief introduction

Mainly aimed at beginners, just started to build their own threat intelligence database, through a simple and convenient python script to build their own malicious ip database. The IP of this malicious database comes from more authoritative threat intelligence abroad. Describe how to collect the blacklist ip, the direction can also be extended to malicious domain names, malicious certificates, you can use similar methods. The reason why we want to include foreign open source intelligence threat libraries is more because domestic network security companies will not share their own databases, but also use the same way of thinking to crawl foreign information.

Third, blacklist IP (ClearC, malware, spam, web crawler)

First, the code will download the blacklist IP list (c2 servers, spam, web crawlers, spyware and other malware) from threat sources on authoritative foreign platforms. And while collecting files containing blacklisted ip, we also need to aggregate and filter data to extract only important available data.

We need to write the following command to cron, which can be used to automatically download the blacklist IP every 5 minutes.

"

Crontab-e

"

Add the following three lines to the configuration file

"

* / 5 * cd & & wget http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist.txt

* / 5 * cd & & wget reputation.alienvault.com/reputation.data

* / 5 * cd & & wget https://myip.ms/files/blacklist/general/latest_blacklist.txt

In Home, creating a new py file is called merge.py

"

Filename: merge.py

Arthor:Mike

Date:2019.8.5

"

Open a file

C2=open ("c2-ipmasterlist.txt", "r")

Bl=open ("latest_blacklist.txt", "r")

Reputation=open ("reputation.data", "a")

Data downloaded separately by row

For line in c2:

Ip = line.split (',')

Reputation.write (ip [0] + ", c2\ n")

For line in bl:

Ip = line.split ()

Try:

Reputation.write (ip [0] + "\ n")

Except:

Pass

Before executing the code, you need to add another command to the cron file, which imports the blacklist IP list every 5 minutes with the following command:

Open the configuration file for cron

Crontab-e

Add merge.py files that run every 5 minutes to expand the amount of blacklist data.

Then restart the cmikewhorontab service:

Service cron restart

This screenshot shows the malicious ip combined with merge.py, the total file of the domain name.

Create a new py file again and check whether IP is included in the blacklist.

Import mmap

Open the IP collection file to check whether the IP in the traffic is in the blacklist

File = open ("reputation.data")

IP = '207.241.231.146'

S = mmap.mmap (file.fileno (), 0, access=mmap.ACCESS_READ)

If s.find (IP)! =-1:

Print "This" + IP+ "is blacklisted"

File.close () IV. Summary

This article briefly describes how to use open source threat intelligence to help us better protect our network from malicious attacks. Readers can expand their thinking and collect C2 databases of various information sources through crawler technology.

5. Hint

It is suggested that some students may not be able to start crontab because of different configuration files. Here is a more detailed tutorial illustration to help you configure the threat intelligence library.

Open crontab log

Crontab does not perform the open date by default, so after writing at the beginning, it cannot be executed. I want to check the log, but I can't find it. So open the log of the timing task to view it.

Modify the rsyslog service to delete # cron.* and # before # daedon.* in the / etc/rsyslog.d/50-default.conf file

Restart the rsyslog service with the following command:

Service rsyslog restart

Then restart the crontab service:

Service cron restart, after reading the above, have you mastered how to build a threat intelligence database? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.