Shulou(Shulou.com)06/03 Report--

Although Sqoop has been steadily used in the production environment for many years, some of its own defects have brought inconvenience to the actual operation. Sqoop2 has become the object of study, so what are the advantages of Sqoop2?

First of all, let's take a look at the use of Sqoop. Data will not be lost when using Sqoop, and Sqoop is powerful and can lead data to various connectors such as HDFS,Hbase,Hive, but Sqoop is a pure client architecture, which will bring a lot of problems:

(1) the client needs to connect to the relational database, which requires each client to configure a connector. The more clients, the more configurations for each client, and the operation is troublesome.

(2) the user's password is required, but in the process of performing the operation of Sqoop, the password will appear in clear text, so the operation is not safe.

(3) it is difficult to integrate command line excuses in external applications.

(4) closely combined with JDBC semantics, it is impossible to access NoSQL database.

And Sqoop2 is on the basis of the original Sqoop to add the server side, so we can uniformly configure the connector to the server side, can we uniformly maintain the configuration items connecting to the database, and can we uniformly save the user's password? Of course, Sqoop2 can solve these problems very well. Advantages of Sqoop2:

(1) clients only need to connect to the Sqoop server

(2) the administrator has configured the DB connection

(3) end users no longer need to deal with database authentication

(4) centralized audit tracking

(5) better resource management

(6) Sqoop service can be accessed through CLI,REST API and Web UI

But at present, Sqoop2 is still under development, and its direction is good, but some of its functions and features have not appeared in our practical application, such as some functions of Sqoop can not be used in Sqoop2, such as guiding data to Hbase can not be used, although Sqoop2 and Sqoop do not have the same features, but its implemented features are considered to be stable. So how do Sqoop and Sqoop2 choose to use it in practice? Sqoop2 is preferred, unless some features are missing before choosing to use Sqoop.

This is my understanding and understanding of sqoop2, but sqoop2 is not a mature product after all, and we look forward to sharing with all of us. Since I usually like to follow the Wechat official account "big data cn" and "big data Times Learning Center", there are a lot of classic introductions, which is not bad for absorbing all kinds of knowledge. In the usual self-improvement, we should also communicate more, which is also of great significance to improve our knowledge structure.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.