Shulou(Shulou.com)06/01 Report--

In this issue, the editor will bring you about how to use BlockETL, the basic tool for block chain data analysis. The article is rich in content and analyzes and describes for you from a professional point of view. I hope you can get something after reading this article.

BlockETL software package is used for data extraction / conversion / loading (ETL) in Bitcoin blockchain data analysis. Block and transaction data can be extracted from the original Bitcoin blockchain file and loaded into the general SQL database to facilitate subsequent data analysis and processing. It is very suitable for graduation design or research projects related to blockchain data analysis.

1. Overview of software packages

The main features of the BlockETL package are as follows:

Read the original block file directly and extract it quickly.

Load the original block and transaction data into the SQL database to facilitate subsequent block chain data analysis

Multi-pipeline parallel processing is supported, and the number of concurrent pipelines can be controlled.

The structure design is clear, and the code is easy to expand.

The BlockETL package is developed by Java and supports multiple operating systems such as Windows, OSX and Linux. The current version 1.0.0. The main classes / interfaces and relationships are shown in the following figure:

For the directory organization of the BlockETL software package and the list of main code files, please see the official website description: http://sc.hubwiz.com/codebag/blocketl-java/

2. How to use the software package

Before you start using the package, install gradle 6, jdk 8, and the SQL database of your choice. Take MySQL as an example to illustrate the use process:

To install the mysql server, refer to the documentation: Install and Upgrading MySQL.

Install jdk 8, reference documentation: JDK 8 Installation

To install gradle 6, refer to the documentation: Gradle Installation.

2.1 initialize database / table structure

Go to the sql directory of the package and use the ddl.sql script to create the database and underlying table structure:

Mysql-u root-p < ddl.sql

The above command requires the password of the root user to be entered interactively, or you can use other usernames / passwords with sufficient permissions. Upon successful execution, the above command creates the database bitcoin_analysis and the underlying data table:

Blocks: bitcoin block data sheet

Transactions: Bitcoin transaction head data sheet

Tx_inputs: bitcoin transaction input data sheet

Tx_output: bitcoin transaction output data sheet

2.2 extraction process configuration

First, modify the block directory and database connection information in the demo/src/App.java file and save it according to your basic environment configuration:

String dataDir = "…" ; / / Block data file directory String dbUrl = "jdbc:..."; / / SQL database connection URL String dbUser = "…" , / / SQL database user name String dbPassword = "…" / / SQL database password String network = "mainnet"; / / main network: mainnet, test network: testnet2.3 data extraction and loading

Enter the demo directory of the package, start the demo application with the gradle run or gradle.bat run command, and you can see the progress of loading the data output on the screen:

In addition to outputting the log on the screen, the demo application of the package will also be recorded in the app.log file in the current directory, which can be used to troubleshoot.

To run unattended for a long time, you can use gradle distZip to package and then execute the compiled program.

2.4 data analysis

Once the block chain data is extracted and loaded into the SQL database, the data can be analyzed.

In order to improve the data loading speed, the index of the data table is not defined in the previous ddl.sql, so before the data analysis, we should first use index.sql to establish the data table index to speed up the execution of the statistical query. Enter the sql directory and execute the following command to complete the index construction, and enter the password as required:

Mysql-u root-p < index.sql

After the index is established, you can use SQL for data analysis.

This is how the blockchain data analysis basic tool BlockETL shared by the editor is used. If you happen to have similar doubts, you might as well refer to the above analysis to understand. If you want to know more about it, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.