Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "what are the new functions of Kudu1.7". In the operation of actual cases, many people will encounter such a dilemma. Then let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

New features of 1.Kudu1.7

1.Kudu now supports field types for decimal.

two。 Improve the policy used by tablets to automatically repair due to the loss of copies of Kudu due to server or disk failure. The strategy for replicating new replicas is to first add a replacement copy of tablet, and then remove the failed copy. Previously, the failed copy was removed and then a replacement copy of tablet was added. In the case of a tablet server failure and a quick return within 5 minutes, the new replica management scheme allows tablets to recover quickly. The new scheme also provides better overall stability on clusters with frequent server failures.

The 3.kudu fs update_dirs tool now supports deleting directories. Kudu does not allow you to delete the directory where the configuration stores tablet data unless the-- force parameter is specified. If the-- force parameter is specified, all tablet copies configured to use that directory will fail at startup and start replicating elsewhere, provided there are a majority of copies elsewhere.

4. Users can use the new-- fs_metadata_dir command to specify a directory specific to tablet metadata. It is recommended that you put it on a device with high performance and low latency, such as SSD. If not specified, the metadata will be placed in the directory specified by-- fs_wal_dir or in the directory specified by the first entry of-- fs_data_dirs (if metadata already exists in the earlier deployment). Kudu does not automatically move existing metadata based on this configuration.

5.READ_YOUR_WRITES is a new scan read mode. You can specify READ_YOUR_WRITES when creating a new scanner in the Clipper Java and Python clients. If you use this mode, the client knows all previous writes and reads from the client when it performs a read operation. Reading in this mode ensures that the session is based on "read-your-writes" and "read-your-reads" while minimizing latency caused by waiting for outstanding write transactions. However, this is still an experimental feature that needs to be improved and stable in subsequent versions.

The scanning dashboard (/ scans) of 6.tablet server's web UI adds the following new features:

Displays the most recently completed scan.

Pseudo-SQL scan descriptor, which succinctly displays the selected fields and applied predicates.

More complete and better document scanning statistics.

7.Kudu daemons now opens a web page that dump the current stack trace information of each thread running on each server. This information may be helpful when diagnosing performance problems.

Optimization and improvement of 2.Kudu1.7

two。 By default, each tablet copy now stripes data blocks across three data directories instead of all. This reduces the impact on tablet if a single disk fails. Performance tests show that this feature has no impact on performance. This feature applies to new copies created after upgrading to Kudu1.7.

The 2.Kudu server previously provided the ability to enable a separate metrics log, which stores periodic snapshots of all metrics available on the server. This feature is now available as part of the Diagnostic Log, which is enabled by default. The diagnostic log includes a collection of server metrics and thread stacks that are periodically dump. By default, the diagnostic log does not exceed 640MB because it is actually smaller with compression.

3. Error handling in the synchronous Java client has been improved so that when an exception is thrown, the stack trace indicates the correct location of the calling client function instead of the call stack of the internal worker thread. The original call stack from the worker thread can be used to suppress exceptions (suppressed exception).

4. The Java client now uses a predefined priority list of TLS passwords when establishing an encrypted connection with the Kudu server. This password list matches the list of preferred passwords for server-to-server communication and ensures that the most effective and secure passwords are also preferred. When Kudu clients are running on Java 8 or later, read and write performance can be significantly improved.

5. The report of the kudu cluster ksck tool is updated, and the tablets and tables of the tablet copy in progress appear to be in the process of being restored. This report update allows more common scenarios, especially tablet copies, to have fewer alerts.

6. Improved performance for inserting rows that contain many string or binary fields, especially in the case of highly concurrent write workloads.

7. By default, the Spark task of scanning Kudu can now scan non-leader copies. This makes it easier for Spark to schedule kudu-spark tasks and ensures the locality of the data. You can disable this feature by passing leader_only to the kudu.scanLocality option.

8. The number of OS threads used during steady state and burst activity, such as Raft leader elections triggered by node failures, has been significantly reduced and should no longer exceed the value of ulimit-u. In most cases, servers running Kudu tablet no longer need to increase the value of ulimit-u or / proc/sys/kernel/threads-max.

9. Fixed an issue where sparse column predicates could cause too many data blocks to be read. Previously, in such scans, sparsely matched predicates were used on multiple columns, and Kudu would read and decode the same block of data multiple times. This fix can improve the performance of such scans by 5-10 times.

10. Improved performance for updating larger values, such as workloads that update values greater than 1KB.

3.Wire Protocol compatibility

Kudu 1.7.0 / CDH 5.15.0 and previous Kudu versions of wire-compatible

1.Kudu1.7 clients can connect to servers running Kudu1.0 or later. If the client uses a function that is not available on the target server, an error is returned.

two。 Rolling upgrades to Kudu1.7 have not been fully tested. We do not recommend rolling upgrades for Kudu. You can shut down all nodes in the cluster, upgrade Kudu, and then restart.

3. In addition to the restrictions on secure clustering mentioned below, Kudu1.0 clients can connect to servers running Kudu 1.7.

The authentication feature introduced in Kudu1.3 has the following restrictions on wire compatibility between Kudu1.7 and versions prior to 1.3:

1. If the Kudu1.7 cluster is configured with authentication or encryption is set to required, clients earlier than Kudu1.3 cannot connect.

two。 If the Kudu1.7 cluster is configured with authentication and encryption is optional or disabled, the old clients can still connect.

4. Client library compatibility

1.Kudu1.7 's Java client library keeps API and ABI compatible with Kudu1.6. Applications written for Kudu1.6 can be compiled and run in the Kudu1.7 client library, and vice versa.

The 2.Kudu1.7 C++ client is forward compatible with Kudu1.6 's API and ABI. Applications written and compiled for Kudu1.6 client libraries can run without modifying the Kudu1.7 client libraries. Applications written and compiled for Kudu1.7 client libraries can run without modifying the Kudu1.6 client libraries.

The 3.Kudu1.7 Python client maintains API compatibility with Kudu1.6. Applications written for Kudu1.6 can continue to run on the Kudu1.7 client and vice versa.

4. On Kudu1.6 's server, an error is reported when using the Kudu1.7 client to create a table with fields of type decimal. Similarly, an error is reported when using the Kudu1.6 client to create a table with fields of type decimal.

5.Kudu1.7/CDH5.15 incompatibility change

1. The newly introduced replica management scheme is not compatible with the old one, so it is not possible to run the pre-Kudu1.7 Master Server with Kudu1.7 's Tablet Server, and vice versa. This is server-side incompatibility and will not affect client compatibility. The old version of the Kudu client can run on the new Kudu server.

two。 The previously optional metrics log adds a readable timestamp to each line. The path to the log file has also changed, and "diagnostics" replaces "metrics" in the file name. It is optimized that the metrics log contains only those metrics that change between consecutive samples, and does not contain entity attributes such as tablet partition information.

This is the end of the content of "what are the new features of Kudu1.7"? thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.