Shulou(Shulou.com)06/03 Report--

What is big data architect: around big data platform system-level R & D personnel, proficient in Hadoop, Spark, Storm and other mainstream big data platform core framework. In-depth understanding of how to write MapReduce jobs and job flow management to complete the calculation of data, and be able to use the general algorithms provided by Hadoop, proficient in the Hadoop ecosystem components such as Yarn,HBase, Hive, Pig and other important components, to achieve platform monitoring, auxiliary operation and maintenance system development. I am a big data programmer. I have built a big data resource sharing group 593188212 to share big data's learning materials and learning methods every day. Now share my professional knowledge.

one。 As enterprise architects, why do we need to build data structures?

The data structure mainly includes the following contents:

1) inconsistent data standards

2) data model management is chaotic.

3) the problem of deep performance cannot be solved.

4) the poor writing level of SQL statements leads to serious performance problems.

5) developers are aware of the implementation plan

6) lack of audit before launch

7) lack of relatively complex data processing capacity

8) poor data quality needs to implement data quality management.

Data is the wealth of customers, although it is worthless to us developers, it is priceless to customers, ensuring the completeness, security and reliability of data.

two。 Responsibilities and skills as a data architect

Create a data management system to integrate, centralize, protect and maintain data sources

Essential language: java,SQL,XML,HIVE,PIG,SPARK

Skills and strengths: data warehouse solutions, in-depth understanding of database architecture, extraction of thansformation, loading (ETL), spreadsheets and BI tools, data modeling, system development

three。 If you apply for a job as an architect, what exactly do you think the design architecture does?

1: confirm requirements .2: system decomposition. 3: technology selection 4: develop technical specifications

Architects need to communicate not only with developers, but also with project managers, requirements analysts, and even end users. Therefore, for architects, there are not only technical requirements, but also interpersonal communication requirements. If you learn this, can't you get it? how to get a little salary?

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.