Shulou(Shulou.com)06/01 Report--

This article mainly explains "what is the method of Apache IoTDB deployment and operation and maintenance". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Next, let the editor take you to learn "what is the method of Apache IoTDB deployment and operation and maintenance"?

Right out of the box

The idea of IoTDB is that the operation and maintenance of the system should be simple and should be started with one click and used immediately out of the box. Let's start with startup. You need to install jdk8 or jdk11, download the release version, and unzip the directory structure as follows:

Memory in conf/iotdb-env.sh, configure MAX_HEAP_SIZE, to remove the previous comments. All other configurations are in conf/iotdb-engine.properties.

You may need to add an execution permission to the script, and then start server in the background:

Nohup sbin/start-server.sh > / dev/null 2 > & 1 &

Directory structure

After starting with the default configuration and writing the data, the project root generates the following folders, all under the data directory:

It mainly includes three parts, data file data/data, system file data/system, pre-write log data/wal.

Data catalog: data/data

The directory is organized as follows: sequential / disordered data-> Storage Group-> Partition number (currently the default is a partition)-> data file .tsfile and index file .resource.

Data file TsFile (Timeseries File) is our designed column storage file format, which mainly stores the original data of each time series. TsFile has a separate API and can be used as an independent tool, just like Parquet and ORC. Data files can be configured in multiple directories through the data_dirs parameter.

System directory: data/system

There are several important documents in it.

Mlog.txt: metadata log, which records all metadata operations, including adding and deleting storage groups, adding and deleting time series, etc. The log in this file will be redone when restarting, and will be skipped if an error is encountered.

System.properties: system attribute that records some parameters that cannot be changed after startup, such as partition granularity, time precision, and so on.

Tlog.txt: tag and attribute information, if not created, this file is empty.

Pre-write log directory: data/wal

The written data is first recorded in the pre-write log and then written to memory. When the data in memory is flushed, the log before writing will be cleaned. When you find that you have written some data, the data directory is still empty, and the data is in the pre-write log and memory.

Pre-write logs are organized by storage group, with a pre-write log file corresponding to a TsFile file.

If you want to force the data in memory to be flushed, you can execute the flush command through CLI. Memory data is persisted and pre-write logs are clearly written.

Syslog: data/logs

In addition to the above three most important, there are system logs, log files are archived according to the date, there is no date is today's.

If you have any problems, you can copy the log here and send it to us.

Start all over again

After testing some, want to start over, it is very convenient, delete the above three directories, restart it. Or the prettiest IoTDB in the whole street.

Data migration

Before migrating, it is best to use the CLI console to execute a flush command to persist the in-memory data (so that there is no pre-write log). A restart of IoTDB has a similar effect. The migrated IoTDB version should be the same.

Introduce several scenarios:

1. Migrate the IoTDB from machine A to machine B (debugging, data backup, etc.)

You can copy the entire data directory (including data directory and system directory), configure the root directory on machine B, and start IoTDB on machine B.

two。 Migrate some of the storage group data from IoTDB on machine A to machine B

If you want to migrate data, metadata must be migrated. First copy the data/system directory of machine A to machine B, then copy some of the storage group directories in the data directory of machine A to machine B, and configure the corresponding directory on machine B. Just start the IoTDB of machine B.

3. Migrate the metadata from IoTDB on machine A to machine B, but do not want the data

Copy the data/system directory of machine A to the appropriate location of machine B, and start the IoTDB of machine B.

System monitor and control

Monitoring the memory and CPU of the system is still very cool, now let everyone also have fun, the following is applicable to the version after 0.9.2, mainly using jvisualvm, local need to have jdk8.

If you are running IoTDB locally, you can connect to the native IoTDB process by typing jvisualvm directly on the local command line.

If you want to run IoTDB on the server, you need to modify the configuration file conf/iotdb-env.sh first

JMX_LOCAL= "false" JMX_IP= "the_real_iotdb_server_ip" # enter the IP address of the actual IoTDB

Check out conf/jmx.password, where the default JMX user and password are recorded and can be changed. The permissions of each user are in conf/jmx.access.

Then set up a remote connection, which defaults to port 31999. After connecting, you can see:

At this point, I believe you have a deeper understanding of "what is the method of Apache IoTDB deployment and operation and maintenance". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.