Shulou(Shulou.com)05/31 Report--

This article will explain in detail how to extract offline data from Datax in MySQL. The editor thinks it is very practical, so I share it with you as a reference. I hope you can get something after reading this article.

Datax download address: https://github.com/alibaba/DataX

Datax MySQL read: https://github.com/alibaba/DataX/blob/master/mysqlreader/doc/mysqlreader.md

Datax MySQL writes: https://github.com/alibaba/DataX/blob/master/mysqlwriter/doc/mysqlwriter.md

1. Download and install Datax software (jdk version 1.8 or above must be installed)

[root@localhost ~] # tar xvf jdk-8u65-linux-x64.tar.gz

[root@localhost ~] # mv jdk1.8.0_151 / usr/local/jdk1.8.0_151

[root@localhost ~] # vim / etc/profile

Export PATH=$PATH:/usr/local/jdk1.8.0_151/bin

[root@localhost ~] # vim / etc/ld.so.conf.d/mysql-x86_64.conf

/ usr/local/jdk1.8.0_151/lib

[root@localhost ~] # tar xvf datax.tar.gz

[root@localhost ~] # cd datax/job/

two。 Edit configuration file

[root@localhost job] # vim job.json

{

"job": {

"setting": {

"speed": {

"channel": 5

}

}

"content": [

{

"reader": {

"name": "mysqlreader"

"parameter": {

"username": "taf"

"password": "taf2015"

Column: ["*"]

"connection": [

{

"table": ["T2"]

"jdbcUrl": ["jdbc:mysql://172.16.8.160:3306/test?useUnicode=true&characterEncoding=utf8"]

}

]

}

}

"writer": {

"name": "mysqlwriter"

"parameter": {

"writeMode": "update"

"username": "wangying"

"password": "wangying"

Column: ["*"]

"connection": [

{

"jdbcUrl": "jdbc:mysql://172.16.8.93:3306/db_stktag"

"table": ["T2"]

}

]

}

}

}

]

}

}

[root@localhost job] #

The results of the tables on both sides of the 3.mysql must arrive, and the data must be extracted using python2.

[root@localhost job] # python2 / root/datax/bin/datax.py job.json

DataX (DATAX-OPENSOURCE-3.0), From Alibaba!

Copyright (C) 2010-2017, Alibaba Group. All Rights Reserved.

2019-01-29 11 VMInfo# operatingSystem class 1815 50.569 [main] INFO VMInfo-VMInfo# operatingSystem class = > sun.management.OperatingSystemImpl

2019-01-29 11 the machine info 18 18 15 [main] INFO Engine-the machine info = >

OsInfo:Oracle Corporation 1.8 25.151-b12

JvmInfo:Linux amd64 3.10.0-123.el7.x86_64

Cpu num:8

TotalPhysicalMemory:-0.00G

FreePhysicalMemory:-0.00G

MaxFileDescriptorCount:-1

CurrentOpenFileDescriptorCount:-1

. . .

. . .

Mission start time: 2019-01-29 11:18:50

End of mission: 2019-01-29 11:19:04

Total time spent on the task: 13s

Average task traffic: 1B/s

Record write speed: 0rec/s

Total number of readout records: 7

Total number of read and write failures: 0

[root@localhost job] #

This is the end of this article on "how to achieve offline data extraction in Datax in MySQL". I hope the above content can be of some help to you, so that you can learn more knowledge. if you think the article is good, please share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.