Shulou(Shulou.com)06/02 Report--

Today, I will talk to you about the methods of data preprocessing, which may not be well understood by many people. in order to make you understand better, the editor has summarized the following contents for you. I hope you can get something according to this article.

The methods of data preprocessing are: 1, data cleaning, "cleaning" data by filling in missing values, smoothing noise data, identifying or deleting outliers and solving inconsistencies; 2, data integration, the process of establishing a data warehouse is actually data integration; 3, data transformation; 4, data reduction.

Data preprocessing (data preprocessing) refers to the processing of data before the main processing. For example, before the conversion or enhancement of most geophysical area observation data, the irregularly distributed survey network is transformed into regular network by interpolation to facilitate the operation of the computer. In addition, for some profile measurement data, such as seismic data preprocessing, there are vertical stack, rearrangement, adding head, editing, resampling, multi-channel editing and so on.

The method of data preprocessing

1. Data cleaning

Clean up the data by filling in missing values, smoothing noise data, identifying or deleting outliers, and resolving inconsistencies. The main goal is to achieve the following goals: format standardization, abnormal data removal, error correction, duplicate data removal.

2. Data integration

Data integration routines combine and store data from multiple data sources, and the process of establishing a data warehouse is actually data integration.

3. Data transformation

The data is transformed into a form suitable for data mining by means of smooth aggregation, data generalization, standardization and so on.

4. Data reduction

In data mining, the amount of data is often very large, and it takes a long time for mining and analysis on a small amount of data. Data reduction technology can be used to obtain the reduced representation of the data set, which is much smaller, but still close to maintaining the integrity of the original data, and the results are the same or almost the same as those before reduction.

Data preprocessing is a hot research aspect of data mining, after all, it is determined by the background of data preprocessing-almost all the data in the real world is dirty.

After reading the above, do you have any further understanding of the methods of data preprocessing? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.