Shulou(Shulou.com)06/01 Report--

Detailed explanation of data Quick loading (dmfldr) in DM7 Database

An overview of 1.1 features

Dmfldr (DM Fast Loader) is a command line tool for fast data loading provided by DM7. By using dmfldr tools, users can load the text data sorted according to a certain format into the DM database in a simple, fast and efficient way, or write the data in the DM database into a text file according to a certain format.

1.2 Architecture

As shown in the figure, dmfldr actually includes a dmfldr function module in the DM7 database server in addition to the client tools, which work together to complete the functions of dmfldr.

1. When loading the data, the dmfldr client receives the commands and parameters submitted by the user, analyzes the control files and data files, packages and sends the data to the dmfldr module on the server side, and the server completes the real data loading work. And analyze the messages returned by the server, and if necessary, specify to generate log files and error data files according to user parameters.

2. When exporting data, the dmfldr client receives the commands and parameters submitted by the user, analyzes the control file, and converts the user requirements into corresponding messages sent to the server-side dmfldr module. The server parses and packages the data that needs to be exported and sends it to the dmfldr client, which writes the data to the specified data file and generates a log file according to the user parameters if necessary.

Two dmfldr starters 2.1 launch

After installing the DM7 database management system, you can find the dmfldr execution file in the "bin" subdirectory of the installation directory.

Start the command line window of the operating system, go to the directory where "dmfldr" is located, and if the environment variable is configured, you can start it anywhere, and you are ready to start the dmfldr tool.

The use of dmfldr must specify the necessary parameters, otherwise the tool will report an "invalid number of parameters" and exit. The format of the parameters specified for dmfldr is:

Dmfldr keyword=value [keyword=value...]

For example:

Dmfldr USERID=SYSDBA/SYSDBA CONTROL='c:\ fldr.ctl'

As shown in the example, USERID and CONTROL are the parameters that must be specified to start dmfldr, and USERID must be the first parameter and CONTROL must be the second parameter.

2.2 dmfldr syntax

The use of dmfldr is more flexible and there are many parameters. Users can use "dmfldr help" to view the simple information of each parameter.

Dmfldr help

Format: DMFLDR KEYWORD=value

Routine: DMFLDR SYSDBA/SYSDBA CONTROL='c:\ fldr.ctl'

USERID must be the first argument on the command line

CONTROL must be the second argument on the command line

String type parameters must be enclosed in quotation marks

Keyword description (default)

USERID username / password format: USER/PWD@SERVER:PORT#SSL_PATH@SSL_PWD

CONTROL control file, string type

LOG log file, string type (dfldr.log), which is not specified in the file location by default

BADFILE error data record file, string type (dfldr.bad)

SKIP initially ignores logical rows (0)

Number of rows that LOAD needs to load (ALL)

ROWS submission frequency (50000). DIRECT is valid for FALSE.

Whether the DIRECT is mounted in a quick way (TRUE)

Whether SET_IDENTITY inserts a self-incrementing column (FALSE)

Whether SORTED data is sorted by clustered index (FALSE)

INDEX_OPTION indexing options (1)

1 does not refresh the secondary index, and the data is sorted first according to the index, and then loaded.

Insert sorted data into the index

2 do not refresh the secondary index, and rebuild all secondary indexes after the data is loaded

Maximum number of big data errors allowed by ERRORS (100)

CHARACTER_CODE character encoding, string type (GBK, UTF-8, SINGLE_BYTE, EUC-KR)

MODE loading method. String type IN indicates loading, and OUT indicates loading. Default is IN.

OUTORA means to load ORACLE (IN)

Whether the CLIENT_LOB large field directory is local (FALSE)

LOB_DIRECTORY large field data file storage directory

LOB_FILE_NAME large field data file name, valid for export only (dmfldr.lob)

The size of the BUFFER_NODE_SIZE read-in file buffer (10), valid values range from 1mm 2048

The maximum number of rows processed by the READ_ROWS worker thread at a time (100000), with a maximum support of 2 ^ 26-10000

Whether SILENT loads data silently (FALSE)

Whether the NULL string is treated as NULL when NULL_MODE is loaded

Whether the unloaded time-space value is treated as a NULL string (FALSE)

A string that is treated as a NULL value when NULL_STR loads

The number of sending nodes when SEND_NODE_NUMBER is running (20). Valid value range is 16065535.

The number of threads used by TASK_THREAD_NUMBER to process user data. By default, the number of cores is the same as that of the processor. Valid values range from 1 to 128.

Number of BLDR_NUM server BLDR (64), valid value range 1mm 1024

Size of BDTA_SIZE bdta (5000), valid values range from 100to 10000

Whether COMPRESS_FLAG compresses bdta (FALSE)

MPP_CLIENT mpp environment, whether to distribute locally (TRUE)

BLOB_TYPE BLOB type actual type of field data value, string type (HEX_CHAR)

HEX indicates that the value is hexadecimal, and HEX_CHAR indicates that the value is of hexadecimal character type

Valid only on direct=FALSE

The directory where the OCI_DIRECTORY OCI dynamic library is located

DATA specifies the data file path

ENABLE_CLASS_TYPE allows users to import CLASS type data (FALSE)

Whether to swipe the disk immediately when FLUSH_FLAG is submitted (FALSE)

Whether IGNORE_BATCH_ERRORS ignores error data and continues to import (FALSE)

Whether SINGLE_HLDR_HP uses a single HLDR to load the HUGE horizontal partitioning table (FALSE)

HELP print help information

2.3 Control file syntax

The control file CONTROL is a parameter that must be specified to start dmfldr, which is used to specify the format of the data in the data file. When the data is loaded, dmfldr parses the data file according to the format specified by the control file; when exporting the data, dmfldr also generates the data file according to the column delimiter, row delimiter, and so on specified by the control file.

Other dmfldr parameter values can also be specified in the control file.

The syntax of the dmfldr control file is as follows:

[OPTIONS (

=

……

)]

LOAD [DATA]

INFILE [LIST] [] [

[]]

[BADFILE]

[APPEND | REPLACE | INSERT]

INTO TABLE [.]

[FIELDS]

[]

:: = parameter

:: = value

:: = file address

:: = include file list file address

:: = STR [X]

:: = {}

: = INTO TABLE [.]

[WHEN]

[FIELDS [TERMINATED BY] [X]]

[]

[]

:: = pattern name

:: = table name

:: = {AND}

< field_condition>

: = [(] [)]

:: = | (p1:p2)

:: = | |! =

:: = [X]''| BLANKS | WHITESPACE

: =''

: = ({,})

: = [FILLER] []

[]

:: = column name

:: = | NULL

:: = position (p1:p2) | position (p1)

:: = DATE FORMAT''

:: = TERMINATED [BY]

:: = WHITESPACE | [X]

: = [OPTIONALLY] ENCLOSE [BY] [X]

:: = CONSTANT ""

:: = "function name ()"

For the above control file syntax, it is important to note that:

When  1 and dmfldr deal with newline characters in data files, windows defaults to 0x0D0A (\ r\ n), and non-windows defaults to 0x0A (\ n). Users should adjust accordingly according to the newline characters in existing data files. The corresponding option is that if the specified value is a hexadecimal string value that requires the [X] option, the value no longer needs to start with 0x. If the [X] option is not specified, the value is the specified string

 2. With regard to column delimiters, the user should specify at least one of FIELDS or coldef_option. If both exist, the setting in coldef_option shall prevail. If the delimiter indicates the [X] option, it indicates that the delimiter is a string in hexadecimal format

 3. With regard to the LIST option, when INFILE uses the LIST option, it indicates that the actual data file path is stored in a file specified by INFILE, which can store multiple actual data file paths, using commas or newline splits

 4. With regard to the OPTIONS option, this option supports the specification of all parameters in the command line parameters except userid,control,help, with each parameter value pair separated by spaces or newlines. For the parameters that appear in option, which also appear in the specified execution parameters of dmfldr, dmfldr will select the values of the corresponding parameters in option to execute

 5. About col_def,FILLER means to skip the value of the specified column in the data file

 6. About property_option options

 A, position (p1:p2): this column value is from the 1st byte to the p2 byte of each row of data in the data file, including the boundary p1m p2

 B, position (p1): start from the first byte of each row of data in the data file to the next column score

The data between the delimiters is the column value, including the boundary p1

 C and position options are not valid for large field data. If you specify this option for large field types, an error will be reported

 D, NULL: the specified value is NULL, ignoring the value in the data file

The property_option parameter is only valid for import

 7. About the term_option option, which is used to specify the end flag of the specified column in the data file. The end of a column can be a WHITESPACE (space) or a user-defined string or hexadecimal string. When term_option is specified, the column does not need to be separated by FIELDS

 8. With regard to the enclosed_option option, this parameter specifies the closure character, which is optional and does not exist by default. If the closure character is set in both into_table_clause and coldef_option, the setting in coldef_option shall prevail. If the [X] option is specified before the closure character, the closure character is a string in hexadecimal format

The length of a  9, delimiter or enclosing character string cannot exceed 255byte

 10. With regard to the constant_option option, after you specify the constant keyword, there is no need to prepare data for the column in the data file. If specified, the column data will be loaded as the next field data, resulting in data confusion. The constant option is not valid for large field types

 11. With regard to the fun_option option, only the trim () function is currently supported.

Three dmfldr instances 3.1 normal files (text files) 3.1.1 create test tables

SQL > drop table test

The operation has been performed

Elapsed time: 29.165 milliseconds. Execution number: 1174.

SQL >

SQL > create table test (C1 int,c2 int,c3 date)

The operation has been performed

Elapsed time: 21.497 milliseconds. Execution number: 1175.

SQL >

3.1.2 create the source data file test.txt

[dmdba@dm3 ~] $cat / home/dmdba/test.txt

1 | 1 2019-09-22

2 | 2 2019-09-22

3 | 3 2019-09-22

[dmdba@dm3 ~] $

3.1.3 create a control file test.ctl

[dmdba@dm3 ~] $cat / home/dmdba/test.ctl

LOAD DATA

INFILE'/ home/dmdba/test.txt'

INTO TABLE test

FIELDS'|'

(

C1

C2 TERMINATED BY''

C3 DATE FORMAT 'yyyy-mm-dd'

)

[dmdba@dm3 ~] $

3.1.4 data loading using dmfldr

[dmdba@dm3 ~] $dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'log=\' / home/dmdba/dmfldr_test.log\'

Dmfldr V7.6.0.171-Build (2019.07.02-109059) ENT

Dmfldr:

Copyright (c) 2011, 2015, Dameng. All rights reserved.

Control file:

Rows loaded: all

Number of server rows per submission: 50000

Skip rows: 0

Number of errors allowed: 100

Whether to load directly: Yes

Whether to insert a self-incrementing column: No

Whether the data is sorted by clustered index: No

Character set: GBK

A total of 1 data files:

/ home/dmdba/test.txt

Error file: fldr.bad

Target table: TEST

Column name wrapper data type termination

C1 CHARACTER |

C2 CHARACTER WHT

C3 yyyy-mm-dd |

Number of row buffers: 2

Number of task threads: 1

3 lines of record submitted

Target table: TEST

3 lines loaded successfully.

Row 0 is discarded due to data format error.

Row 0 was not loaded due to a data error.

Total number of logical records skipped: 0

Total number of logical records read: 3

Total number of logical records rejected: 0

Usage time: 10.231 (ms)

The output from the log tells us that 3 lines were successfully imported.

3.1.5 query validation

SQL > select * from test

Line number C1 c2 c3

--

1 1 1 2019-09-22

22 2 2019-09-22

3 3 3 2019-09-22

Elapsed time: 10.354 milliseconds. Execution number: 1178.

SQL >

3.2 large field data processing 3.2.1 Export of large field data

When dmfldr works in the export mode, that is, MODE is OUT, the data file name corresponding to the large field generated by dmfldr is specified by LOB_FILE_NAME. If it is not specified by default, the file is stored in the directory specified by LOB_DIRECTORY, and if LOB_DIRECTORY is not specified, it is stored in the same directory of the specified export data file.

3.2.1.1 create a test table

SQL > drop table test

The operation has been performed

Elapsed time: 29.165 milliseconds. Execution number: 1174.

SQL >

SQL > create table test (C1 int,c2 blob,c3 clob)

The operation has been performed

Elapsed time: 21.497 milliseconds. Execution number: 1175.

SQL >

3.2.1.2 insert data

INSERT INTO TEST VALUES (0XAB121032DE)

INSERT INTO TEST VALUES (2pr 0XAB121032DEP abcdefg')

COMMIT

SQL > select * from test

Line number C1 c2 c3

--

1 1 0xAB121032DE abcdefg

2 2 0xAB121032DE abcdefg

Elapsed time: 0.489 milliseconds. Execution number: 1184.

SQL >

3.2.1.3 create a control file test.ctl

[dmdba@dm3 ~] $cat / home/dmdba/test.ctl

LOAD DATA

INFILE'/ home/dmdba/test.txt'

INTO TABLE test

FIELDS'|'

(

C1

C2

C3

)

[dmdba@dm3 ~] $

3.2.1.4 data export using dmfldr

[dmdba@dm3] $dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'lob_directory=\' / home/dmdba\ 'mode=\' out\ 'log=\' / home/dmdba/dmfldr_test.log\'

Dmfldr V7.6.0.171-Build (2019.07.02-109059) ENT

2 rows is load out

Export a total of 2 rows of data

Usage time: 199.145 (ms)

[dmdba@dm3 ~] $

3.2.1.5 Verification

[dmdba@dm3 ~] $cat test.txt

1 | dmfldr.lob:0:5 dmfldr.lob:5:7

2 | dmfldr.lob:12:5 dmfldr.lob:17:7

[dmdba@dm3 ~] $

[dmdba@dm3 ~] $ls

Dmfldr_test.log test.ctl Public Video document Music dmfldr.lob test.txt template Picture download Desktop

[dmdba@dm3 ~] $

In this example, LOB_DIRECTORY is specified instead of LOB_FILE_NAME, and the exported large-field data file is stored in the / opt/data directory specified by LOB_DIRECTORY, with the file name dmfldr.lob.

3.2.2 loading of large field data when DIRECT is TRUE

When MODE is IN and DIRECT is TRUE, if large field objects are involved in data loading, you need to specify a large field data file. If CLIENT_LOB is TRUE,LOB_DIRECTORY, you should specify the client local directory where the large field data files are located; if CLIENT_LOB is FALSE, the user must first transfer the relevant files to the main library of the DM server, and then use LOB_DIRECTORY to specify the storage directory.

The large-field data file is specified in the data file and can be in any format. In the data file, large fields are recorded in the data file in the form of "file name: start offset: length". When the specified file name is invalid, dmfldr reports an error and the load fails. For CLOB type fields, dmfldr will fail to load when there are incomplete characters within the specified offset and length range.

3.2.2.1 create a test table

SQL > drop table test

The operation has been performed

Elapsed time: 29.165 milliseconds. Execution number: 1174.

SQL >

SQL > create table test (C1 int,c2 blob,c3 clob)

The operation has been performed

Elapsed time: 21.497 milliseconds. Execution number: 1175.

SQL >

3.2.2.2 create the source data file test.txt

[dmdba@dm3 ~] $cat / home/dmdba/test.txt

1 | testblob.txt:0:1 | testclob.txt:0:1

2 | testblob.txt:1:2 | testclob.txt:1:2

3 | testblob.txt:2:3 | testclob.txt:2:3

[dmdba@dm3 ~] $

Testblob.txt and testclob.txt are text files with a length of more than 3 bytes and a storage path

Is / home/dmdba.

3.2.2.3 create a control file test.ctl

[dmdba@dm3 ~] $cat / home/dmdba/test.ctl

LOAD DATA

INFILE'/ home/dmdba/test.txt'

INTO TABLE test

FIELDS'|'

(

C1

C2

C3

)

[dmdba@dm3 ~] $

3.2.2.4 data loading using dmfldr

[dmdba@dm3 ~] $dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'lob_directory=\' / home/dmdba\ 'log=\' / home/dmdba/dmfldr_test.log\'

Dmfldr V7.6.0.171-Build (2019.07.02-109059) ENT

Dmfldr:

Copyright (c) 2011, 2015, Dameng. All rights reserved.

Control file:

Rows loaded: all

Number of server rows per submission: 50000

Skip rows: 0

Number of errors allowed: 100

Whether to load directly: Yes

Whether to insert a self-incrementing column: No

Whether the data is sorted by clustered index: No

Character set: GBK

A total of 1 data files:

/ home/dmdba/test.txt

Error file: fldr.bad

Target table: TEST

Column name wrapper data type termination

C1 CHARACTER |

C2 CHARACTER |

C3 CHARACTER |

Number of row buffers: 2

Number of task threads: 1

3 lines of record submitted

Target table: TEST

3 lines loaded successfully.

Row 0 is discarded due to data format error.

Row 0 was not loaded due to a data error.

Total number of logical records skipped: 0

Total number of logical records read: 3

Total number of logical records rejected: 3

Usage time: 37.195 (ms)

[dmdba@dm3 ~] $

The output from the log tells us that 3 lines were successfully imported.

3.2.2.5 query validation

SQL > select * from test

Line number C1 c2 c3

--

1 1 1 2019-09-22

22 2 2019-09-22

3 3 3 2019-09-22

Elapsed time: 10.354 milliseconds. Execution number: 1178.

SQL >

3.2.3 loading of large field data when DIRECT is FALSE

When MODE is IN and DIRECT is FALSE, the large field column data in the data file is the field content. The BLOB_TYPE parameter specifies that the contents of the BLOB column are hexadecimal or string:

When l BLOB_TYPE is HEX_CHAR, the BLOB column in the data file is treated as hexadecimal content

When l BLOB_TYPE is HEX, BLOB is listed as a string in the data file, which is converted to hexadecimal after import.

The BLOB_TYPE parameter is valid only when DIRECT is FALSE, and defaults to HEX_CHAR.

3.2.3.1 create a test table

SQL > drop table test

The operation has been performed

Elapsed time: 29.165 milliseconds. Execution number: 1174.

SQL >

SQL > create table test (C1 int,c2 blob,c3 clob)

The operation has been performed

Elapsed time: 21.497 milliseconds. Execution number: 1175.

SQL >

3.2.3.2 create the source data file test.txt

[dmdba@dm3 ~] $cat / home/dmdba/test.txt

1 | 0x12d3c8a7 | abcdefg

2 | 0x12a4cbac | hijlkmn

3 | 0x22d3c8b3 | adefhjd

[dmdba@dm3 ~] $

3.2.3.3 create a control file test.ctl

[dmdba@dm3 ~] $cat / home/dmdba/test.ctl

LOAD DATA

INFILE'/ home/dmdba/test.txt'

INTO TABLE test

FIELDS'|'

(

C1

C2

C3

)

[dmdba@dm3 ~] $

3.2.3.4 data loading using dmfldr

[dmdba@dm3] $dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'direct=false blob_type=\' hex_char\ 'log=\' / home/dmdba/dmfldr_test.log\'

Dmfldr V7.6.0.171-Build (2019.07.02-109059) ENT

Dmfldr:

Copyright (c) 2011, 2015, Dameng. All rights reserved.

Control file:

Rows loaded: all

Number of server rows per submission: 50000

Skip rows: 0

Number of errors allowed: 100

Whether to load directly: No

Whether to insert a self-incrementing column: No

Whether the data is sorted by clustered index: No

Character set: GBK

A total of 1 data files:

/ home/dmdba/test.txt

Error file: fldr.bad

Target table: TEST

Column name wrapper data type termination

C1 CHARACTER |

C2 CHARACTER |

C3 CHARACTER |

3 rows processed.

Target table: TEST

3 lines loaded successfully.

Row 0 is discarded due to data format error.

Row 0 was not loaded due to a data error.

Total number of logical records skipped: 0

Total number of logical records read: 3

Total number of logical records rejected: 0

Usage time: 1364.220 (ms)

The output from the log tells us that 3 lines were successfully imported.

3.2.3.5 query validation

SQL > select * from test

Line number C1 c2 c3

--

1 1 0x12D3C8A7 abcdefg

2 2 0x12A4CBAC hijlkmn

3 3 0x22D3C8B3 adefhjd

Elapsed time: 25.011 milliseconds. Execution number: 1198.

SQL >

For cases where blog_type is hex, readers can test it themselves, using the same method as above.

Four special applications 4.1 the first behavior column name of a text file

The SKIP parameter is used to set the number of logical lines and int values that skip the start of the data file. The default skip starting number of rows is 0.

Continue to use the previous test.txt file and modify the file as follows:

[dmdba@dm3 ~] $cat / home/dmdba/test.txt

Col1 col2 col3

1 | 1 2019-09-22

2 | 2 2019-09-22

3 | 3 2019-09-22

[dmdba@dm3 ~] $

For the import of such files, you need to set the first line of the skipped text file, which can be set in two ways:

1. Through the control file

Create the control file test.ctl:

[dmdba@dm3 ~] $cat / home/dmdba/test.ctl

OPTIONS

(

SKIP = 1

)

LOAD DATA

INFILE'/ home/dmdba/test.txt'

INTO TABLE test

FIELDS'|'

(

C1

C2 TERMINATED BY''

C3 DATE FORMAT 'yyyy-mm-dd'

)

[dmdba@dm3 ~] $

2. Specify the skip parameter directly during import

Dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'log=\ / home/dmdba/dmfldr_test.log\' skip=1

We choose the first way here:

Perform data import:

[dmdba@dm3 ~] $dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'log=\' / home/dmdba/dmfldr_test.log\'

Dmfldr V7.6.0.171-Build (2019.07.02-109059) ENT

Dmfldr:

Copyright (c) 2011, 2015, Dameng. All rights reserved.

Control file:

Rows loaded: all

Number of server rows per submission: 50000

Skip rows: 0

Number of errors allowed: 100

Whether to load directly: Yes

Whether to insert a self-incrementing column: No

Whether the data is sorted by clustered index: No

Character set: GBK

A total of 1 data files:

/ home/dmdba/test.txt

Error file: fldr.bad

Target table: TEST

Column name wrapper data type termination

C1 CHARACTER |

C2 CHARACTER WHT

C3 yyyy-mm-dd |

Number of row buffers: 2

Number of task threads: 1

3 lines of record submitted

Target table: TEST

3 lines loaded successfully.

Row 0 is discarded due to data format error.

Row 0 was not loaded due to a data error.

Total number of logical records skipped: 0

Total number of logical records read: 3

Total number of logical records rejected: 0

Usage time: 10.231 (ms)

The output from the log tells us that 3 lines were successfully imported.

Verify:

SQL > select * from test

Line number C1 c2 c3

--

1 1 1 2019-09-22

22 2 2019-09-22

3 3 3 2019-09-22

Elapsed time: 0.721 milliseconds. Execution number: 1203.

SQL >

4.2 the last line is not formatted correctly and cannot be imported

The data files used by dmfldr are all in text format, in which the column values are stored in the data file as strings. To load this data into a database table, you need to convert the string to the data type corresponding to each column of the database table. Dmfldr supports all column definition types supported by DM databases, including strings, numeric values, time, date, and time

Date interval, large field type, and so on.

If the encoding mode of the data file is different from that of the DM database server, dmfldr also needs to convert the character encoding. Dmfldr supports conversion between UTF8 and GBK encodings.

The data type and transcoding work is done by the dmfldr client, and if an error occurs in the process, dmfldr skips the line to continue the rest of the work and records the error line to the file specified by BADFILE.

4.2.1 create a test table

SQL > drop table test

The operation has been performed

Elapsed time: 29.165 milliseconds. Execution number: 1174.

SQL >

SQL > create table test (C1 int,c2 int,c3 date)

The operation has been performed

Elapsed time: 21.497 milliseconds. Execution number: 1175.

SQL >

4.2.2 create the source data file test.txt

[dmdba@dm3 ~] $cat / home/dmdba/test.txt

1 | 1 2019-09-22

2 | 2 2019-09-22

3 | 3 2019-09-22

4 | 4 yyyy-mm-dd

[dmdba@dm3 ~] $

4.2.3 create a control file test.ctl

[dmdba@dm3 ~] $cat / home/dmdba/test.ctl

LOAD DATA

INFILE'/ home/dmdba/test.txt'

INTO TABLE test

FIELDS'|'

(

C1

C2 TERMINATED BY''

C3 DATE FORMAT 'yyyy-mm-dd'

)

[dmdba@dm3 ~] $

4.2.4 data loading using dmfldr

[dmdba@dm3 ~] $dmfldr userid=sysdba/SYSDBA control=\'/ home/dmdba/test.ctl\ 'log=\' / home/dmdba/dmfldr_test.log\ 'badfile=\' / home/dmdba/test.bad\'

Dmfldr V7.6.0.171-Build (2019.07.02-109059) ENT

Dmfldr:

Copyright (c) 2011, 2015, Dameng. All rights reserved.

Control file:

Rows loaded: all

Number of server rows per submission: 50000

Skip rows: 1

Number of errors allowed: 100

Whether to load directly: Yes

Whether to insert a self-incrementing column: No

Whether the data is sorted by clustered index: No

Character set: GBK

A total of 1 data files:

/ home/dmdba/test.txt

Error file: / home/dmdba/test.bad

Target table: TEST

Column name wrapper data type termination

C1 CHARACTER |

C2 CHARACTER WHT

C3 yyyy-mm-dd |

Number of row buffers: 2

Number of task threads: 1

2 rows of records submitted

Target table: TEST

2 lines loaded successfully.

Row 0 is discarded due to data format error.

Row 1 was not loaded due to a data error.

Total number of logical records skipped: 1

Total number of logical records read: 3

Total number of logical records rejected: 1

Usage time: 18.103 (ms)

[dmdba@dm3 ~] $

From the log output, tell us to import 3 lines successfully and skip 1 line.

4.2.5 query validation

SQL > select * from test

Line number C1 c2 c3

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.