Shulou(Shulou.com)05/31 Report--

This article mainly introduces what Batea is a tool, the article introduces in great detail, has a certain reference value, interested friends must read it!

Batea

Batea is a context-driven sorting framework for network devices based on anomaly detection branches of machine learning algorithms, and the main goal of Batea is to allow and help security teams to use nmap scan reports to automatically filter interested network assets in large networks.

Batea working mechanism

Batea works by constructing a digital representation (numpy) of all devices from a nmap report (XML), and then applying anomaly detection methods to discover interesting or valuable network assets. We can also extend its functionality by adding specific characters to the numerical representation of the network asset element.

The digital representation of network asset elements is constructed using features inspired by the expertise of the security community. The unmanaged anomaly detection method allows the tool to use the network asset context or the overall description of the network as the core building block of the sorting algorithm. The exact algorithm used here is the Isolation Forest algorithm.

Machine learning model is the core of Batea. The model is an algorithm trained across the entire data set to predict the score of the same (and other) data points (network devices). In addition, Batea allows model persistence. That is, we can reuse pre-trained models and export models trained on large data sets for further use.

Tool installation $git clone git@github.com:delvelabs/batea.git$ cd batea$ python3 setup.py sdist$ pip3 install-r requirements.txt$ pip3 install-e. Developers install the $git clone git@github.com:delvelabs/batea.git$ cd batea$ python3-m venv batea/$ source batea/bin/activate$ python3 setup.py sdist$ pip3 install-r requirements-dev.txt$ pip3 install-e. $pytest tool using # full information $sudo nmap-A 192.168.0.0 sV 16-oX output.xml # partial information $sudo nmap-O-sV 192.168.0.0 batea 16-oX output.xml $batea-v output.xml tool Use sample # simple use (output the top five assets in the default format) $batea nmap_report.xml# output the first three $batea-n 3 nmap_report.xml# export all assets $batea-A nmap_report.xml# use multiple input files $batea-A nmap_report1.xml nmap_report2.xml# use the wildcard $batea. / nmap*.xml$ batea-f csv. / assets*.csv# you can Batea is used on pre-training model and derived training model. # persistent training, output and dump model $batea-D mymodel.batea nmap_report.xml# using pre-training model $batea-L mymodel.batea nmap_report.xml# using preformatted CSV and XML files $batea-x nmap_report.xml-c portscan_data.csv# Verbose mode $batea-vv nmap_report.xml how to add new features

Batea works by assigning digital features to each host in a report (or a series of reports). Hosts here refer to python objects derived from nmap reports, which consist of the following list of attributes: [ipv4, hostname, os_info, ports], where ports is a list of port objects. Each port has the following attributes: [port, protocol, state, service, software, version, cpe, scripts]. All attribute values default to None.

Features is an object that inherits from the FeatureBase class and instantiates a specific _ transform method. This method always takes a list of all hosts as input and returns a lambda function that maps each host to a numeric numpy column (host order is conservative) and then appends that column to the matrix representation of the scan report. Features must output the correct value (floating point or integer), but not other values.

Most feature transformations are implemented using simple lambda functions, just be sure to default a numeric value for each host to achieve model compatibility.

Specific examples are as follows:

Class CustomInterestingPorts (FeatureBase): def _ init__ (self): super (). _ _ init__ (name= "some_custom_interesting_ports") def _ transform (self, hosts): "" This method takes a list of hosts and returns a function that counts the number of host ports member from a predefined list of "interesting" ports, defaulting to 0. Parameters-hosts: list The list of all hosts Returns-f: lambda function Counts the number of ports in the defined list. " Member_ports = [21, 22, 25, 8080, 8081, 1234] f = lambda host: len ([port for port in host.ports if port.port in member_ports]) return f

Next, we can use the NmapReport.add_feature method in batea/__init__.py to add new features to the report:

From .principures.basic _ features import CustomInterestingPorts def build_report (): report = NmapReport () # [...] Report.add_feature (CustomInterestingPorts ()) return report uses precomputed tabular data (CSV)

We can also use preprocessed data to train models or make predictions. Data must be indexed by (ipv4,port), and each row has a unique combination. Columns must use one of the following names, but not all. If a column is missing, the parser defaults to a null value.

'ipv4', 'hostname',' os_name', 'port',' state', 'protocol',' service', 'software_banner',' version', 'cpe',' other_info'

Example:

Ipv4,hostname,os_name,port,state,protocol,service,software_banner10.251.53.100,internal.delvesecurity.com,Linux,110,open,tcp,rpcbind, "program version port/proto service100000 2 111/tcp rpcbind100000 3 111/tcp rpcbind100000 2 4" 10.251.53.100 pension.delvesecurity.comecurity.com LinuxMagic111openpjobindMagne10.251.53.188 serious.delvesecurity.comjinuxpjinxx11, "X11Probe: CentOS" output value indicates that

We can also output numerical matrices and score columns instead of regular output, which is very useful for further data analysis and debugging.

The above $batea-oM network_matrix nmap_report.xml is all the contents of the article "what is Batea?". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.