Shulou(Shulou.com)06/03 Report--

Enterprises can use the data in hand, or mine the needs of users, or optimize products, or seize the market, or reduce operating costs, and so on. A good data analyst can bring huge potential profits for the enterprise. As the demand for data analysts increases year by year, the compensation of data analysts is also rising.

When we open the recruitment website and look at the attractive salaries of data analysts, we will find that data analysis posts can be divided into "data analyst" and "big data analyst". What's the difference between big data and data analysis?

CDA Institute of data Analysis crawled the recruitment information of the keyword "data analysis" on Zhaopin recruitment website in August 2018. after cleaning the data, it excluded more than 10, 000 low-related recruitment and false recruitment, and analyzed the remaining 5706 related recruitment information.

Through the word frequency analysis of the two fields of "job responsibilities" and "job requirements" in the recruitment information, we can know the difference between them.

1. Differences in post responsibilities

First of all, we generate a word cloud (involving 813 pieces of data) for the field "Job responsibility" of the data analysis post:

We can summarize the main responsibilities of the data analysis position:

1. Responsible for daily demand research, data analysis and business analysis. The daily tasks of this process may include the submission of daily, weekly, monthly, annual reports, etc.

2. According to the business requirements, formulate the relevant data collection strategy, design, establish and test the relevant data model, so as to extract the decision value from the data. This process may require the writing of reports on specific analysis requirements

3. Study the data mining model and participate in the construction, maintenance, deployment and evaluation of the data mining model.

When we generate a word cloud (involving 508 pieces of data) for the field "Job responsibilities" of big data's analytical position, we can clearly see the difference between the two types of data posts:

We can sum up the main responsibilities of big data's analytical post:

1. Participate in the design and development of big data platform to solve the challenges faced by massive data.

2. Proficient in Java programming, able to build the company's big data analysis platform based on Hadoop/Hive/Spark/Storm/HBase, etc.

3. Manage, optimize and maintain Hadoop, Spark and other clusters to ensure that the cluster scale is sustained and stable.

4. Be responsible for the function, performance and extension of HDFS/hive/HBase, solve and realize the business requirements.

two。 Differences in professional background requirements of practitioners

Through the field "job requirements" of the data analysis post (involving 999 pieces of data), the CDA Institute of data Analysis found that among the current recruitment needs, candidates with three professional backgrounds of mathematics, statistics and computer science had the highest demand, followed by economics majors.

When we analyze the word frequency of big data's professional background (involving 658 pieces of data), big data's analysis position requires the highest computer professional background, and the word frequency is almost twice as high as that of statistics. The demand for mathematics majors ranks second, and the demand for economics majors is very little.

3. Use analysis tools to distinguish

What is the difference between the software tools used in the work of those two types of positions?

CDA Institute of data Analysis first makes a word frequency analysis of the data analysis tools in "Job requirements" (involving 999 items):

We will find that nearly half of the positions require proficiency in SQL and EXCEL, and the third is the current "Deep-Fried Chicken" Python in the field of data science. It is followed by the traditional statistical software R, SPSS, SAS. Of course, there will be a small number of enterprises require to understand the big data platform architecture software Hadoop, Storm and so on.

When CDA Institute of data Analysis conducted word frequency analysis on big data's analysis position, it was found that the highest word frequency in the job requirements were big data platform tools such as Hadoop, Spark, Hive, HBase, Storm and so on.

More than 60% of big data's analysis posts are required to be proficient in Java programming, and nearly 1/3 positions are required to be familiar with Linux development environment.

In addition to the software tools that require the mastery of massive data processing, the word frequency of Python and SQL is also very high, with nearly 1/3 of the positions required to master. However, we can obviously see that the software used in big data's analysis work is biased towards the development and architecture of big data platform, and the amount of data to be processed is far more than that of ordinary data analysis posts.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.