Shulou(Shulou.com)06/01 Report--

The purpose of this article is to share with you what CEP means in Flink. The editor thinks it is very practical, so share it with you as a reference and follow the editor to have a look.

Complex event processing (CEP) not only regards different data as different events, but also establishes a set of event relation sequence library by analyzing the relationship between events. The use of filtering, aggregation, relevance, dependency, hierarchy and other technologies, and finally achieve the generation of high-level event relationships from simple relationships.

At present, complex event processing is mainly used in credit card fraud detection, user risk detection, equipment fault detection, attack behavior analysis and other fields.

Based on DataSet Api, Apache Flink provides the component stack of FlinkCEP, which is specially applied to the direction of complex event processing. Apache Flink also provides Flink ML and Gelly component stacks for machine learning. Let's take a look at Flink CEP today.

Basic concept

Event definition simple event

The characteristics of simple events are basically unique events, which can be simply observed through data. There are no event rules that can be determined by the need to view the relationship between multiple events.

For example, an alarm will be given when the occurrence of a certain index exceeds a certain threshold. It can be calculated directly by querying the number of times the event occurs within a certain time range. And output the results. Complex event

Complex events are formed based on the combination of simple events, and complex events are not the only events relative to simple events. It is composed of dealing with multiple events, complex events trigger some actions, notification, alarm, trigger other events and so on when a specific event flow occurs. Event relationship

Only complex events have the relationship between events, and the relationship between complex events includes many types. For example, there are temporal relations, aggregation relations, hierarchical relations, causality, abstract relations or dependency relations and so on. Time series relation

There will be a temporal relationship between events. For example, when An events occur, the state of B events changes with the change of An events. Polymerization relationship

The aggregate relationship of individual events, for example, when the number of An events occurs more than 30 times, the alarm information is triggered. Hierarchical relationship

For example, An event is the parent of B event, and the time from An event to B event is specific. Dependency relationship

For example, the generation of An event depends on the trigger of B event. Otherwise, the An event will not be triggered. It shows that the An event depends on the trigger of the B event. Causality, for example, the trigger of B event is because the state of An event has changed, then the change of An is the cause, and the trigger of B is the effect. There are consequences only if there is a cause! The purpose of event processing complex event processing is to make corresponding data processing strategies for real-time data through the corresponding rule engine. These strategies include inference, root cause, decision, prediction and so on. Event inference

It is mainly used for the constraint relationship between event states, which can infer the state of another part from the state of one part. Root cause of event

When you have the result and know the initial state, you can find out the cause of a specific action that caused the result. Event decision

Get the state of the result, know the initial state, and then determine the action to be performed. Event prediction

This situation knows the initial state and the action to occur, and predicts unknown results. Common scenarios in CEP

Flink CEP can take advantage of many scenarios, and it also has a wide range of use cases and experience accumulation in actual business scenarios. CEP is commonly used in network attack detection, risk control model, credit card fraud and so on.

Network attack detection

Network attack detection is based on real-time data flow. For example, sudden SQL scanning or sudden increase in network bandwidth caused by common dragging behavior can be determined by CEP.

For example, the common rule is to give early warning directly when the bandwidth increases suddenly and the amount of database resources suddenly increases. Network credit

Credit card fraud based on CEP or current Internet loans. The previous loan approval process is sky-high, the current Internet loans are hour-level or even minute-level loans. Based on this, users' credit scores are basically achieved through CEP. Such as real-time import of user-related information, age, ID number, sesame credit score and so on. The rule may be: age

Thank you for reading! This is the end of the article on "what does CEP in Flink mean?". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, you can share it for more people to see!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.