Shulou(Shulou.com)06/01 Report--

This article mainly explains "what sqoop export is". The explanation content in this article is simple and clear, easy to learn and understand. Please follow the idea of Xiaobian and go deep into it slowly to study and learn "what sqoop export is" together.

Sqoop export

The Sqoop export tool is used to import file collections from HDFS into relational databases. The target table must have been created beforehand in the database. Input files are read and parsed into recordsets based on user-specified delimiters. By default, Sqoop converts these operations into INSERT mode, injecting data records into the target database. In Update Mode, Sqoop generates update statements to replace records that already exist in the database. In Call Pattern, Sqoop generates a stored procedure call for each record.

1,"INSERT mode": Sqoop defaults to convert each record into INSERT statement, added to the database table, if your table has some constraints such as keyword uniqueness, use insert mode to be careful, avoid violating these constraints. If a record insertion fails, the import job will fail. This pattern is typically used to import data into a new, empty table. Sqoop uses INSERT mode by default

2."UPDATE mode": If you specify the--update-key parameter, Sqoop will modify the existing dataset in a database, and each record will be modified as an UPDATE statement. Record modification statements are determined by the column specified by--update-key. If an update statement does not have a corresponding record in the database, no error is reported and the export operation continues. In fact, this means that no new records are imported into the database, only the original data is updated.

II. Introduction of parameters

--update-key can also be followed by multiple keyword column names, which can be separated by commas. Sqoop will match multiple keywords before updating.

The--export-dir parameter is used in conjunction with the--table or--call parameter to specify the fileset directory on HDFS where data needs to be imported into MySQL.

--update-mode updateonly and allowinsert The default mode is updateonly. If you specify--update-mode as allowinsert, you can import data that does not exist in the target database into the database table. There is no data insertion, there is no data update.

III. Combination test and explanation

1. When update-key is specified and the relational database table has a primary key:

A, allowinsert mode, in order to update the contents of the target database table, and the original non-existent data is also imported into the database table;

B. In updateonly mode, the contents stored in the target database table are updated, and the data that does not exist originally is not imported into the database table;

2. When update-key is specified and there is no primary key in the relational database table:

A. In allowinsert mode, all data is added to the database table;

B. In updateonly mode, the contents stored in the target database table are updated, and the data that does not exist originally is not imported into the database table;

3. When update-key is not specified and the relational database table has a primary key:

A. In allowinsert mode, primary key conflict is reported, and there is no change in data;

B. In updateonly mode, primary key conflict is reported, and there is no change in data;

4. When update-key is not specified and there is no primary key in the relational database table:

A. In allowinsert mode, all data is added to the database table;

B. In updateonly mode, all data is appended to the database table.

Thank you for reading, the above is the content of "sqoop export is what", after the study of this article, I believe that everyone has a deeper understanding of what sqoop export is, and the specific use needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.