Shulou(Shulou.com)06/02 Report--

Introduction

Elasticsearch is the cornerstone of ELK (Elasticsearch/Logstash/Kibana). In this article, we will use Rancher Catalog to deploy stack and use it to track tag and brand on Twitter.

Tracking hashtag on Twitter is useful for measuring the impact of Twitter-based marketing activities. You can extract useful information such as the number of times your tweets have been retweeted and how many new followers your marketing campaign has brought you.

Install ELK stack

Elasticsearch

If you already have a working Elasticsearch cluster, you only need to adjust some of the configuration in the cluster. We will use JSON to create an index template to adjust the configuration.

Get the JSON template on GitHub

Enter http://[ in your browser the path of your kopf on the rancher host]

In kopf, click "more" and then select "index templates" from the drop-down menu.

Now let's give our index template a name and promote its configuration.

Use twitter_elk_example as the template name

Paste the contents of the JSON file you downloaded earlier

Click the "save" button

So much for the configuration of the Elasticsearch cluster. Now let's keep going.

Logstash

Logstash allows you to analyze the data and transfer it to your Elasticsearch cluster. It natively supports many data sources (such as Twitter APIs, collectd, Apache logs, etc.).

When processing your data, Logstash can help you extract or format the correct part of your data. In this way, you don't have to push some unnecessary or (worse) wrong data, which will make your Kibana dashboard inconsistent with the actual situation.

Before we begin, we need to create a Twitter application key

Special attention should be paid to the following:

Consumer Key

Consumer Secret

Access Token

Access Token Secret

Note: make sure that the clocks of all your Rancher hosts are synchronized, otherwise you will not be able to use Twitter certificates correctly.

Now jump to the catalog page and select Logstash (preferably the latest version). You need to add the following to the "Logstash inputs*" input box (replace the CAP text with your own APIs authentication key):

Twitter {consumer_key = > "INSERT YOUR CONSUMER KEY" consumer_secret = > "INSERT YOUR CONSUMER SECRET" oauth_token = > "INSERT YOUR ACCESS TOKEN" oauth_token_secret = > "INSERT YOUR ACCESS TOKEN SECRET" keywords = > ["docker", "rancher_labs", "rancher", "kubernetes"] full_tweet = > true}

Note: do not use the "@" or "#" symbol in the keyword array, or Logstash will fail with a "unauthorized message" error.

In the "Logstash output*" input box, you need to add the following output {elasticsearch {host = > "elasticsearch:9200" protocol = > "http" cluster_name = > "NAME OF YOUR ELASTICSEARCH CLUSTER" index = > "twitter_elk_example" document_type = > "tweets"}

Finally, select "elasticsearch-clients as the Elasticsearch stack/service" and click the "launch" button!

Rancher will do the next things for you, including deploying a fully configured Logstash. If all goes well, you should be able to see that the data has been added to your Elasticsearch home page within a few minutes. You can check it in http://[ your ElasticSearch host address] / # kopt.

Kibana

Kibana can help you create a powerful dashboard based on the data in the Elasticsearch cluster. To deploy Kibana, you only need to do two things: choose the correct version of Rancher Catalog and connect it to the elasticsearch-clients container.

In this way, a properly configured Kibana is ready to be used! We will configure it later.

Now the entire ELK stack is deployed. Although Elasticsearch and Logstash have been deployed, we still need to do something with Kibana.

In this example, we just need to import a JSON dashboard into Kibana.

Click here to get the JSON file

Go to Settings-> Object, then click "import", and then select the file you just downloaded. You should see an interface similar to the one below.

All that's left is to create an appropriate index setting in Kibana.

Go to the "Indices" page and click the "New" button. You should see the index created and the selected @ timestamp (timestamp).

So far, you have a Kibana dashboard that helps you monitor hashtag and brand on Twitter. To load the imported dashboard, all you have to do is click its name here.

A few minutes later, re-look at dashboard and you will see an interface similar to the following:

At this point, you can successfully monitor tag and brand on Twitter!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.