Shulou(Shulou.com)06/02 Report--

It is believed that many inexperienced people are at a loss about how to realize the bulk storage function of kettle files. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

In addition to being an ETL tool, kettle also has many practical functions in data processing. Here are a few practical small functions.

1. Files are stored in batches

This function can load all or part of the files in a directory into the database.

The first step: the get file name step gets the file name in a directory and uses a regular expression to specify the file name.

Specifies all file names that end in .png in the E:/opensource/kettle-trunk/ui/images directory. In this step, the following fields are created to describe all the acquired files:

FILENAME

SHORT_FILENAME

PATH

TYPE

EXISTS

ISHIDDEN

ISREADABLE

ISWRITEABLE

LASTMODIFIEDTIME

SIZE

EXTENSION

URI

ROOTURI

Step 2: use a javascript script to read the contents of the file. If you don't want to store the contents of the file, you don't have to do this step. Here is the content of the javascript script:

File = new Packages.java.io.File (filename.getString ())

FileInputStream = new Packages.java.io.FileInputStream (file)

Var content = Packages.org.pentaho.di.core.Const.createByteArray (file.length ())

FileInputStream.read (content, 0, file.length ())

FileInputStream.close ()

Don't forget to select the "get variable" button and use content as an output field of type Binary.

The third step: "Table output", the fields obtained in the first two steps are output to the table in the database. If there is no corresponding table in the database, you need to use the "SQL" button to get and execute the SQL statement that creates the table.

Through these three steps, the file name, file attributes and file contents can be saved to the database.

If the file is binary, the contents of the file are generally saved as BLOB, Binary, Image, and so on.

If the file is a character file, the contents of the file are generally saved as CLOB, varchar, Text, and so on.

Note: because this method reads all the contents of the file into memory at once, it can only deal with smaller files.

After reading the above, have you mastered how to realize the bulk storage function of kettle files? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.