Shulou(Shulou.com)06/03 Report--

After thinking about the idea of developing in the direction of big data, I can't help but have some questions. How should I get started? What skills should I learn? What is the learning route? All of the ideas that came into the business were the same as those of students who wanted to learn Java. The job is very hot, the employment salary is relatively high, and the prospects are very considerable. This is basically the reason for yearning for big data, but I don't know much about big data. If you want to learn, first you need to learn programming, second you need to master mathematics, statistics, and finally integrate applications, you can want to develop in the data direction, generally speaking, that's it. However, this alone does not help much. What is it specifically? Let's take a look at it together with the data teacher of Ke Da.

If you want to learn big data well, you'd better join a good learning environment, you can come to this Q group 251956502, so that it is more convenient for everyone to learn, and you can also communicate and share information together.

Now you need to ask yourself a few questions:

1. What are your interests in computers/software?

2. Computer major, interested in operating systems, hardware, networking, servers?

3. Are you a software major interested in software development, programming, and writing code?

4. Majored in mathematics and statistics, particularly interested in data and numbers.

5. What's your major?

Several Stages of Big Data Learning

Stage 1: Java Language Basics

Java development introduction, familiar with Eclipse development tools, Java language basics, Java flow control, Java strings, Java arrays and classes and objects, number processing classes and core technologies, I/O and reflection, multithreading, Swing programs and collection classes

Phase 2: HTML, CSS and Java

PC-side website layout, HTML5+CSS3 basics, WebApp page layout, native Java interaction development, Ajax asynchronous interaction, jQuery application

Java Web and Database

Database, Java Web Development Core, Java Web Development Insider

Stage 4: Linux Hadoop System

Linux architecture, Hadoop offline computing outline, distributed database Hbase, data warehouse Hive, data migration tool Sqoop, Flume distributed log framework

Stage 5: Actual combat (real projects of first-tier companies)

Data acquisition, data processing, data analysis, data presentation, data application

Stage 6: The Spark Ecosystem

Python programming language, Scala programming language, Spark big data processing, Spark-Streaming big data processing, Spark-Mlib machine learning, Spark-GraphX graph calculation, actual combat I: recommendation system based on Spark (real project of a first-tier company), actual combat II: Sina website (sina.com.cn)

Stage 7: Storm Ecosystem

Storm technical architecture system, Storm principle and foundation, message queue kafka, Redis tool, zookeeper detailed explanation, actual combat I: log alarm system project, actual combat II: guess you like recommended system actual combat

Stage 8: Big Data Analysis-AI(Artificial Intelligence)

Data Analyze Work Environment Preparation Data Analysis Fundamentals, Data Visualization, Python Machine Learning

1. Python machine learning 2. Image recognition neural network, natural language processing Social networks processing, actual combat project: outdoor equipment recognition analysis

[if ! supportLists]· [endif] At present, there are many training institutions or post training institutions on the market. In essence, they are all for your skills. Whether you consider it is appropriate for zero-based people to say, it is OK to reply clearly to you, but if it is a bachelor's degree or below, it is harder to learn big data development. There are many majors in big data, big data analysis, big data development, database development.

Generally speaking, the course of developing big data is 4 months of study, for example, 3 months of database development in a single field is enough, the requirement of undergraduate degree or above for big data development is relatively easy, and database junior college or above is enough.

From the enterprise side, big data talent can be roughly divided into three areas: product and market analysis, security and risk analysis, and business intelligence.

Product analysis is a relatively new field in which algorithms are used to test the effectiveness of new products. When it comes to security and risk analysis, data scientists know what data needs to be collected, how to perform rapid analysis, and ultimately analyze the information to effectively deter cybercriminals or catch cybercriminals. For job seekers who want to engage in big data work, how do they choose jobs based on their own conditions?

Here are ten popular jobs related to "big data":

I. ETL R & D

As the variety of data continues to increase, the demand for data integration professionals is growing. ETL developers work with different data sources and organizations, extracting data from different sources, transforming and importing it into a data warehouse to meet the needs of the enterprise. ETL R & D is mainly responsible for extracting data from scattered and heterogeneous data sources, such as relational data and flat data files, to the temporary middle layer for cleaning, transformation and integration, and finally loading them into data warehouse or data mart, which becomes the basis of online analytical processing and data mining. At present, ETL industry is relatively mature, and the work life cycle of related positions is relatively long, which is usually completed by internal employees and outsourcing contractors. One of the reasons ETL talent is so hot in the big data era: In the early stages of enterprise big data adoption, Hadoop was just ETL for the poor.

HDFS provides massive data storage, MapReduce provides data computation. As data sets continue to grow in size and traditional BI data processing costs are too high, the demand for Hadoop and related inexpensive data processing technologies such as Hive, HBase, MapReduce, Pig, etc. will continue to grow. Technicians with experience with Hadoop frameworks are the most sought after big data talent today.

III. Development of visualization tools

Analyzing large amounts of data is a big challenge, and new data visualization tools such as Spotifre, Qlikview, and Tableau can visualize data intuitively and efficiently. Visual development is to generate application software automatically by visual development tool through operating interface elements on graphical user interface provided by visual development tool. Easily connect all of your data across multiple resources and hierarchies, and the time-tested, fully extensible, feature-rich and comprehensive Visual Component Library provides developers with a fully functional and easy-to-use collection of components to build extremely rich user interfaces. Data visualization used to belong to the business intelligence developer category, but with the rise of Hadoop, data visualization has become a separate professional skill and job.

Fourth, information architecture development Big data has re-stimulated the upsurge of master data management.

Making the most of enterprise data and supporting decision-making requires very specialized skills. Information architects must understand how to define and document key elements to ensure that data is managed and utilized in the most effective manner. Key skills for an information architect include master data management, business knowledge, and data modeling.

V. Data Warehouse Research

A data warehouse is a strategic collection of all types of data that supports decision-making processes at all levels of an enterprise. It is a single data store created for analytical reporting and decision support purposes. Provide enterprises with the business intelligence needed to guide business process improvement and monitor time, cost, quality, and control. Data warehouse specialists are familiar with big data appliances from companies like Teradata, Neteeza, and Extranet. Data integration, management and performance optimization can be done on these all-in-one machines.

VI. OLAP development

With the development and application of database technology, the amount of data stored in database has transited from M byte and G byte in 1980s to T byte and P byte now. Meanwhile, the query requirements of users are becoming more and more complex, involving not only querying or manipulating one or several records in a relational table, but also analyzing and synthesizing the data of tens of millions of records in multiple tables. Online Analytical Processing (OLAP) system is responsible for solving this kind of mass data processing problem. OLAP online analytics developer responsible for extracting data from relational or non-relational data sources to build models, then creating user interfaces for data access, providing high-performance predefined query capabilities.

VII. Data science research

Also known as data architecture research, the data scientist is a new type of job that can transform an organization's data and technology into business value. As data science progresses, more and more practical work will be done directly on data, which will enable humans to understand data and thus understand nature and behavior. Therefore, data scientists should first have excellent communication skills and be able to interpret data analysis results to both IT and business leaders. In general, a data scientist is an analyst, artist, and requires a variety of cross-scientific and business skills. VIII. Data Prediction Analysis

Marketing departments often use predictive analytics to predict user behavior or target users. Some scenarios for predictive analytics developers look somewhat like data scientists, testing thresholds and predicting future performance through hypotheses based on historical enterprise data.

IX. Enterprise Data Management

To improve data quality, enterprises must consider data management, and need to establish a data steward position for this purpose. This position needs to be able to use various technical tools to gather a large amount of data around the enterprise, clean and standardize the data, and import the data into a data warehouse to become a usable version. Then, through reporting and analytics, the data is sliced, diced, and delivered to thousands of people. Those who act as stewards of data need to ensure the integrity, accuracy, uniqueness, authenticity and non-redundancy of market data.

X. Data security research

This position is mainly responsible for the management of large-scale servers, storage and data security within the enterprise, as well as the planning, design and implementation of network and information security projects. Data security researchers also need to have strong management experience, knowledge and ability in operation and maintenance management, and a deep understanding of traditional business of enterprises to ensure that enterprise data security is not leaked.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.