Shulou(Shulou.com)06/03 Report--

Mining useful threat information and intelligence from a large number of security incidents is a hot topic in today's discussion, and it is also a difficulty. How can it be realized? A technology called correlation analysis is used here, which is also the most common event detection method in SIEM (Security Information Event Management Security Information and event Management) system, which is nothing new. It was proposed 20 years ago. Usually based on time series, association rules are used for comprehensive association analysis of security events from the same data source or from different data sources. The specific functions of association analysis are described below.

Generally speaking, a malicious Gong JI will leave traces in multiple security devices or applications (such as network firewalls, switches, Web application logs, SQL logs, audit logs, etc.). However, all this information is isolated and stored in different device logs, and faults can be quickly located if association analysis technology is used.

Why is relevance analysis so powerful? In fact, there are many complex association rules in the background. These detection rules can identify A t t a c k events. In fact, these rules are artificially summarized in a large number of practices to generate a language description of Gong JI events, and accurately classify them. The more accurate the classification, the higher the recognition rate.

Tips: in the following, "Gong Ji" and "* *" stand for the vocabulary of sense of memory, you know.

I. the core idea of relevance analysis

The core idea of correlation analysis technology is to establish a behavior baseline by training a certain kind of events, and events outside the baseline range are regarded as abnormal events to classify. This algorithm is more suitable for this paper to detect scenarios, usually most of the logs in the SIEM system are normal events, through the training modeling of normal events to detect anomalies or Gong JI events, which is theoretically called One-Class SVM (single-class support vector machine). More dimensional feature vectors are needed in the OSSIM system to accurately judge the Gong JI source and avoid low detection accuracy or over-fitting. The algorithm input is a security event with the same data structure after OSSIM normalization, and the output is a marked security event. Tags are divided into two categories: Negative and Gong JI (Positive). The output of the OSSIM association analysis engine is the tagged normal and Gong JI events output by the One-Class SVM classification algorithm.

Through in-depth analysis of the association rules of the SIEM system, the most common configuration fields such as: event_id,timestamp,plugin_id,plugin_sid,src_ip,src_port,des_ip,des_port,protocol and so on. In order to make the association analysis rules independent of sensor configuration, this paper extracts several important fields plugin_id_name,plu-gin_id_description,sid_name from the rule base of OSSIM, and divides all fields into keyword tags, then generates its keyword tag statistical model for each detection rule, and uses the rule statistical model to replace the original rules to detect Gong JI.

The core of association analysis is accomplished by one or a set of associated instructions. The following is illustrated by an example of SSH** cracking. As shown in figure 1, SSH** cracking (Brute Force) is a Gong JI behavior derived from the birth of UNIX. According to statistics, more than 50% of the user names are root. A low-reliability (low reliability) SSH server successfully logs in after 100 login attempts, while a high-reliability (high reliability) SSH server succeeds after 10000 logins, so the associated engine can use the number of logins within a certain period of time, and the different source addresses and the same destination IP addresses at that time, and which countries these IP addresses come from. Their credibility and other information to comprehensively determine the Gong JI and respond.

Figure 1

Below you will experience the composition of the associated instructions of the associated engine.

Second, the associated instruction configuration interface

The interface of associated instructions in OSSIM is opened through Configuration- > Threat Intelligence- > Directives. After studying the contents of the previous sections, we have more or less come into contact with the interface of associated instructions. Let's summarize it below, as shown in figure 2 below.

Figure 2 OSSIM associated instruction interface

Key function explanation:

New Directive: click this option to create a new directive from scratch.

Test Directives: click this button to check whether the current new / modified instruction is correct.

Restart Server: click this button and the ossim-server process will be restarted. If you change any of the associated instructions, you need to restart Server. It is wiser to press this button than to restart the entire USM server.

It should be noted that although the restart time is short, all associated data will be lost during the restart.

Figure 3 A complete instruction

● Sensor: a sensor that collects various event information.

● Protocol: protocols, including ANY, TCP, UDP, ICMP.

● Sticky different:

Remember that Linux Foundation refers to the sticky bits used to set file and directory properties? A new attribute, Sticky different (different sticky bit) field, has been added to the associated instruction since OSSIM 4.3.The Sticky different in the rule is None. It is used to set the properties of instruction rules.

For the value of Sticky different, please refer to / etc/ossim/server/directives.xsd file.

...... Slightly

When the newly collected event reaches the correlation engine, it will be associated with the existing event, if the event attributes (such as IP address and port number) are the same, but different sticky bits can be set in the attributes of an instruction, such as None, DST_PORT, SRC_IP, PLUGIN_ID, etc., to distinguish the different events received so that they can be related to each other. For example, in the port scan class Gong JI, if you set the target port to Sticky different, only events from different target ports are associated.

3. Check the results in the Web UI of OSSIM

Let's look at the actual example, as shown in figure 4.

Figure 4 STICKY DIFF

Open / etc/ossim/server/alienvault-dos.xml to view the associated instruction AvAttacks,Possible DDOS

● Username: the user name specified in the event

● Pass: the password specified in the event

● Userdata1~Userdata8: the data domain specified in the event.

Figure 5: view the properties of the association rules to be changed

Note that the number under the From, To, Data source and Event Type columns indicates that it can be modified, and the number under Action means that rules can be added, but the rules in the system default directive are not allowed to be modified.

Fourth, go deep into association rules

1. Basic operation

The predefined rules (/ etc/ossim/server/*.xml) are used for association analysis in OSSIM. So how do we do this? first, through the Web interface, we create a new Correlation Directives, two new rules ssh and test, and then we look at the details of the file content, the path is in the / etc/ossim/server directory, named user.xml file. Other default rules are defined by / etc/ossim/server/directives.xml, as shown in figure 6.

Figure 6 Custom instruction

When the policy under Directive is called by the system, the corresponding XML configuration files are first read according to the categories.xml configuration file. The functions of these files are as follows:

Policy file storage location: / etc/ossim/server/ and description

The policy of the class of Z. alienvault-attacks.xml AlienVault Gong JI

The class Gong JI that is cracked by Z. alienvault-bruteforce.xml AlienVault**

Angular alienvault-dos.xml Alienvault denial of service class

@ alienvault-malware.xml Alienvault malware (including rules for detecting various worms)

Various kinds of error classes (Miscellaneous) of @ alienvault-misc.xml Alienvault

^ alienvault-network.xml Alienvault network class (not available in the open source version)

The policy of Autonomous alienvault-policy.xml Alienvault

Zero alienvault-scan.xml Alienvault scan

Dialog user.xml Alienvault user customization

If the OSSIM association engine cannot read these policy profiles, no alarm will be generated in Analysis → Alarm. Let's take a closer look at an example of xml.

two。 Understand the rule tree

The association engine realizes the association analysis of security events through rules. Readers need to be able to understand the rules of OSSIM, in which a unique tree rule is adopted, and each node in the rule tree corresponds to an association rule (rules). During the matching, the correlation engine matches the uniform format alarms received within a certain period of time from the root node to the leaf node one by one. According to the matching results, the system can aggregate events and raise the alarm level again. Let's look at an example:

Figure 7 Custom instruction content

As can be seen from the above figure, each instruction in this association method based on event sequence is equivalent to a tree composed of rules, so it can be called the association method based on tree instruction (Directive). Its basic idea is to create a series of rules to represent the Gong JI scene according to the related event sequence, and XML is used to define the association sequence in the OSSIM system. When the association analysis engine starts, import all associations into each association rule sequence. Each association rule sequence consists of the following tags. Refine the actual xml above to get the following template:

.

_ priv- > matched, FALSE); g_return_val_if_fail (directive- > _ priv- > rule_root, FALSE); g_return_val_if_fail (directive- > _ priv- > rule_root- > data, FALSE); g_return_val_if_fail (directive- > _ priv- > rule_root- > data), FALSE); g_return_val_if_fail (event, FALSE); g_return_val_if_fail (SIM_IS_EVENT (event), FALSE) Rule = SIM_RULE (directive- > _ priv- > rule_root- > data); match = sim_rule_match_by_event (rule, event); return match;} / * this will check whether the event can match some data in backlog. Backlog is actually an instruction that contains event data. Each backlog entry is a tree that contains all the rules from a single instruction (which is equivalent to an instruction clone). Each of these rules (simrule) also contains data for events that match the rule. * / gbooleansim_directive_backlog_match_by_event (SimDirective * directive, SimEvent * event) {GNode * node = NULL; g_return_val_if_fail (directive, FALSE); g_return_val_if_fail (SIM_IS_DIRECTIVE (directive), FALSE) G_return_val_if_fail (! directive- > _ priv- > matched, FALSE); g_return_val_if_fail (directive- > _ priv- > rule_curr, FALSE); g_return_val_if_fail (directive- > _ priv- > rule_curr- > data, FALSE); g_return_val_if_fail (SIM_IS_RULE (directive- > _ priv- > rule_curr- > data), FALSE); g_return_val_if_fail (event, FALSE) G_return_val_if_fail (SIM_IS_EVENT (event), FALSE); node = directive- > _ priv- > rule_curr- > children; while (node) / / * * We must check the event against all the rule nodes in backlog, except the root node, because it checks in that sim_directive_match_by_event is called from sim_organizer_correlation. * * {SimRule * rule = (SimRule *) node- > data If (sim_rule_match_by_event (rule, event)) {g_log (G_LOG_DOMAIN, G_LOG_LEVEL_DEBUG, "sim_directive_backlog_match_by_event; sim_rule_match_by_event: True"); time_t time_last = time (NULL); directive- > _ priv- > rule_curr = node / / each time the event matches, the instruction goes down to the matching node. Events will be checked next time based on this level. / / FIXME: there may be a memory leak in the parent node. Directive- > _ priv- > time_last = time_last; directive- > _ priv- > time_out = sim_directive_get_rule_curr_time_out_max (directive); sim_rule_set_event_data (rule, event); / / here we assign the fields in the event to the rule, so every time we enter the rule, we can see the matching event. Sim_rule_set_time_last (rule, time_last); if (! G_NODE_IS_LEAF (node)) {GNode * children = node- > children; while (children) {SimRule * rule_child = (SimRule *) children- > data; sim_rule_set_time_last (rule_child, time_last) Sim_directive_set_rule_vars (directive, children); children = children- > next; g_log (G_LOG_DOMAIN, G_LOG_LEVEL_DEBUG, "sim_directive_backlog_match_by_event: There are childrens in% d directive", directive- > _ priv- > id) }} else {directive- > _ priv- > matched = TRUE; g_log (G_LOG_DOMAIN, G_LOG_LEVEL_DEBUG, "sim_directive_backlog_match_by_event: The directive% d has matched", directive- > _ priv- > id);} return TRUE } else {g_log (G_LOG_DOMAIN, G_LOG_LEVEL_DEBUG, "sim_directive_backlog_match_by_event: sim_rule_match_by_event: False");} node = node- > next;} return FALSE;} / * * check all node rules in the directive to see. * / gbooleansim_directive_backlog_match_by_not (SimDirective * directive) {GNode * node = NULL; GNode * children = NULL; g_return_val_if_fail (directive, FALSE); g_return_val_if_fail (SIM_IS_DIRECTIVE (directive), FALSE); g_return_val_if_fail (! directive- > _ priv- > matched, FALSE); g_return_val_if_fail (directive- > _ priv- > rule_curr, FALSE) G_return_val_if_fail (directive- > _ priv- > rule_curr- > data, FALSE); g_return_val_if_fail (SIM_IS_RULE (directive- > _ priv- > rule_curr- > data), FALSE); node = directive- > _ priv- > rule_curr- > children; while (node) {SimRule * rule = (SimRule *) node- > data / / if the rule times out & & if ((sim_rule_is_time_out (rule)) & & (sim_rule_get_not (rule)) & & (! sim_rule_is_not_invalid (rule)) {time_t time_last = time (NULL); directive- > _ priv- > rule_curr = node; directive- > _ priv- > time_last = time_last Directive- > _ priv- > time_out = sim_directive_get_rule_curr_time_out_max (directive); sim_rule_set_not_data (rule); if (! G_NODE_IS_LEAF (node)) / / this is not the last node, it has some child nodes. {children = node- > children; while (children) {SimRule * rule_child = (SimRule *) children- > data; sim_rule_set_time_last (rule_child, time_last); sim_directive_set_rule_vars (directive, children); children = children- > next }} else / / last node! {directive- > _ priv- > matched = TRUE;} return TRUE;} node = node- > next;} return FALSE;} / * * backlog&directives is almost the same: backlog is the place to store instructions and populate event data. * "node" is a child node function. We need to add src_ip, port, and so on to that node from the node that references its level. If the "node" parameter is the children2 in the root node-> child node 1-> child node 2, and we have 1:plugin-sid in children2, then we must add the plugin-sid from the root node to the children2. * / voidsim_directive_set_rule_vars (SimDirective * directive, GNode * node) {SimRule * rule; SimRule * rule_up; GNode * node_up; GList * vars; GInetAddr * ia; GInetAddr * sensor; gint port; gint sid; SimProtocolType protocol Gchar * aux = NULL; g_return_if_fail (directive); g_return_if_fail (SIM_IS_DIRECTIVE (directive)); g_return_if_fail (node); g_return_if_fail (g_node_depth (node) > 1); rule = (SimRule *) node- > data; vars = sim_rule_get_vars (rule)

2019 latest works

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.