Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to install the presto sql engine to access kudu data", the content of the article is simple and clear, easy to learn and understand, the following please follow the editor's ideas slowly in depth, together to study and learn "how to install the presto sql engine to access kudu data"!

Presto is a high performance distributed SQL query engine for big data. Its architecture allows users to query a variety of data sources, such as Hadoop,AWS S3 AlluxioMagi MySQL Magi CassandraMagi Kafka and MongoDB, and even query data from multiple data sources in a single query, which is really powerful.

Here to sort out the installation process, the official website http://prestodb.github.io/, the previous article has written how to install kudu, the following is my process of installing presto:

Download presto-server-0.225.tar.gz and presto-cli-0.225-executable.jar on the official website first.

After decompressing, add the appropriate configuration under this folder:

Create the etc directory in the presto-server-0.90 directory and create the following files:

Node.properties: environment configuration for each node

Jvm.config:jvm parameter

Config.properties: configuring Presto Server parameter

Log.properties: configure log level

Configuration of Catalog Properties:Catalog

The sample configuration of etc/node.properties is as follows:

Node.environment=productionnode.id=ffffffff-ffff-ffff-ffff-ffffffffffffnode.data-dir=/var/presto/data

Parameter description:

Node.environment: environment name. The names of all nodes in a cluster node should be consistent.

Node.id: the name that the node uniquely identifies.

Node.data-dir: the path where data and logs are stored.

The sample configuration of etc/jvm.config is as follows:

-server-Xmx16G-XX:+UseConcMarkSweepGC-XX:+ExplicitGCInvokesConcurrent-XX:+CMSClassUnloadingEnabled-XX:+AggressiveOpts-XX:+HeapDumpOnOutOfMemoryError-XX:OnOutOfMemoryError=kill-9% p-XX:ReservedCodeCacheSize=150M

Etc/config.properties contains Presto Server-related configurations, and each Presto Server can be used as a coordinator and worker.

Coordinator=truenode-scheduler.include-coordinator=truehttp-server.http.port=8090discovery-server.enabled=truediscovery.uri= http://cdh2:8090

Parameter description:

Whether the coordinator:Presto instance provides services using coordinator

Node-scheduler.include-coordinator: whether to allow scheduling tasks on coordinator

Port of the http-server.http.port:HTTP service

Discovery-server.enabled: whether to use Discovery service to discover every node in the cluster.

Url of discovery.uri:Discovery server

Do not add this configuration task.max-memory=1GB, or you will report an error. You don't know why there is a problem with the parsing.

Etc/log.properties can set the log level of a java package:

Com.facebook.presto=INFO

For the configuration of Catalog, first create the etc/catalog directory, and then create the corresponding configuration file according to the connector you want to use. For example, if you want to use the jmx connector, create jmx.properties:

Connector.name=jmx

If you want to use kudu's connector, create a kudu.properties in etc/catalog:

Connector.name=kudukudu.client.master-addresses=localhost:7051

You can see that there are many connectors in the root directory of presto:

Here we can start sudo bin/launcher run,sudo bin/launcher start to run with the background.

Then we can use presto-cli-0.225-executable.jar to operate, of course, I shortened such a long jar to presto-cli, for aspects to use

. / presto-cli-- server localhost:8090-- catalog kudu-- schema default

The users above was created by me:

Then, as with the sql query, insert two pieces of data and query:

Thank you for reading, the above is the content of "how to install the presto sql engine to access kudu data". After the study of this article, I believe you have a deeper understanding of how to install the presto sql engine to access kudu data, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.