Shulou(Shulou.com)06/02 Report--

In the era of mobile Internet,"big data" is one of the key words. As a pioneer in the push technology service industry, push constantly carries out technological innovation to lead the push revolution. By mining user usage scenarios and combining geographical location information, it accurately hits various needs of different users. In the process of serving large customers, big data has been developed through the massive data accumulated by its own platform.

At present, the most important technology of 3.0 products-electronic fence technology: electronic fence is a mobile phone push solution that accurately captures user scenes and pushes valuable messages to users in real time. According to business requirements, customers set electronic fence areas and target user attributes on the map, and push the right content to the right people at the right time, in the right place and in the right scene through cold data portraits (combined with big data analysis to screen target users) and hot data delivery (triggered in real time when target users enter the electronic fence).

To build a big data application based on LBS, the general implementation process is: sorting basic data after information collection, data mining/machine learning, service building and data visualization.

Basic Process of Data Mining

Basic data processing mainly includes: data integration and some partial data processing.

Data integration, data mining or statistical analysis may involve data from different data sources that we need to integrate together. However, if there is only one data source, this step can be omitted.

Basic data processing, some data is missing (data attribute values are null values), some data is noisy (attribute values are wrong, or there are isolated point data), sometimes the same information is expressed in many different ways (there is inconsistency in coding or naming), based on various problems to carry out basic processing of data. Through basic data processing, we can ensure that the information in the village data warehouse is complete, correct and consistent in format.

Data transformation is mainly to use existing fields to obtain new fields. Generally speaking, data transformation mainly includes four types: data discretization (using binning and other methods), generation of derived variables, making variable distribution closer to normal distribution, and data standardization. If you discretize a continuous variable, you avoid introducing any distribution assumptions. This eliminates the need for a normal distribution.

Data mining only according to the database data, with appropriate data mining algorithm analysis, get useful information. Among them, the evaluation of model algorithm quality is a very important step. And data mining is a cyclical process.

Big data applications based on LBS need to solve many problems: basic data problems such as massive data streams (> 20W/s), complex calculations of data processing performance (positioning and statistics), accuracy, second-level real-time requirements, and real-time data. Solutions for this are distributed streaming computing frameworks, Spark Streaming, publish/subscribe models, Apache Kafka, Events, etc.

Finally, LBS applications based on big data can use distributed streaming computing frameworks to build data closed-loops to continuously optimize underlying data.

Current success stories include:

1. Hangzhou white horse lake animation festival security. Through massive sample data collection and analysis, the real-time flow thermal analysis map is used as the main service interface to monitor the flow direction and congestion degree near Baima Lake Animation Square in Binjiang District during the Animation Festival. This deployment establishes a real-time monitoring and early warning system for people flow in Baima Lake block, which issues early warning in real time when the flow of people exceeds a certain number, so as to timely remind relevant personnel to pay attention to the evacuation and guidance of peak flow of people in the corresponding area, prevent stampede events from the source and avoid potential safety hazards.

2. Real-time alarm when earthquake occurs, and through the analysis of large data on crowd thermal map, customize reasonable scheme for post-earthquake rescue work and provide effective help.

3. Cooperate with tourism bureau to combine tourism analysis hotspot map with real-time push for evacuation and guidance of peak traffic in scenic spots to avoid dangerous events.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.