Shulou(Shulou.com)06/01 Report--

Install and use Elasticsearch

Elasticsearch is a new member of the open source search platform. It is an artifact of real-time data analysis. It is developing rapidly. It is based on Lucene, RESTful, distributed, cloud-oriented design, real-time search, full-text search, stable, highly reliable, scalable, easy to install + use, and the introduction is very good. It is not easy to take it out for a walk.

A simple test was done. On two identical virtual machines with about 20 million pieces of data, Elasticsearch inserted data much slower than MongoDB (tolerable), but the search/query speed was more than 10 times faster. This was only a single machine case. Elasticsearch performed better in a multi-machine cluster case. The following installation steps are completed on Ubuntu Server 14.04 LTS.

Install Elasticsearch

Install Oracle Java 7 after upgrading the system. Since Elasticsearch officially recommends Oracle JDK 7, don't try JDK 8 and OpenJDK:

$ sudo apt-get update

$ sudo apt-get upgrade

$ sudo apt-get installsoftware-properties-common

$ sudo add-apt-repositoryppa:webupd8team/java

$ sudo apt-get update

$ sudo apt-get installoracle-java7-installer

Install elasticsearch after joining Elasticsearch official source:

$ wget -O -http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add -

$ sudo echo "debhttp://packages.elasticsearch.org/elasticsearch/1.1/debian stable main">> /etc/apt/sources.list

$ sudo apt-get update

$ sudo apt-get install elasticsearch

Add it to the system startup file and start elasticsearch service. Use curl to test whether the installation is successful:

$ sudo update-rc.d elasticsearch defaults95 1

$ sudo /etc/init.d/elasticsearch start

$ curl -X GET 'http://localhost:9200'

{

"status" : 200,

"name" : "Fer-de-Lance",

"version" : {

"number" : "1.1.1",

"build_hash" :"f1585f096d3f3985e73456debdc1a0745f512bbc",

"build_timestamp" : "2014-04-16T14:27:12Z",

"build_snapshot" : false,

"lucene_version" : "4.7"

},

"tagline" : "You Know, for Search"

}

Elasticsearch cluster and data management interface Marvel is very good, unfortunately only for the development environment free, if this tool is also free invincible, installation is very simple, after the completion of the restart service visit http://192.168.2.172:9200/_plugin/marvel/ you can see the interface:

$ sudo/usr/share/elasticsearch/bin/plugin -i elasticsearch/marvel/latest

$ sudo /etc/init.d/elasticsearch restart

*Stopping Elasticsearch Server [ OK]

*Starting Elasticsearch Server [ OK]

Install Python client drivers

Like MongoDB, we generally use programs to interact with Elasticsearch. Elasticsearch also supports client drivers in multiple languages. Only Python drivers are installed here. For other languages, please refer to the official documentation.

$ sudo apt-get install python-pip

$ sudo pip install elasticsearch

Write a simple program to import gene_info.txt data into Elasticsearch:

#!/ usr/bin/python

# -*- coding: UTF-8 -*-

import os, os.path, sys, re

import csv, time, string

from datetime import datetime

from elasticsearch import Elasticsearch

def import_to_db():

data = csv.reader(open('gene_info.txt', 'rb'), delimiter='\t')

data.next()

es = Elasticsearch()

for row in data:

doc = {

'tax_id': row[0],

'GeneID': row[1],

'Symbol': row[2],

'LocusTag': row[3],

'Synonyms': row[4],

'dbXrefs': row[5],

'chromosome': row[6],

'map_location': row[7],

'description': row[8],

'type_of_gene': row[9],

'Symbol_from_nomenclature_authority': row[10],

'Full_name_from_nomenclature_authority': row[11],

'Nomenclature_status':row[12],

'Other_designations': row[13],

'Modification_date': row[14]

}

res = es.index(index="gene", doc_type='gene_info', body=doc)

def main():

import_to_db()

if __name__ == "__main__":

main()

Kibana is a powerful data display client, integrated with Elasticsearch through plug-ins, easy to install, download and decompress, and then restart Elasticsearch service to visit 192.168.2.172:9200/_plugin/kibana/ to see the interface:

$ wgethttps://download.elasticsearch.org/kibana/kibana/kibana-3.0.1.tar.gz

$ tar zxvf kibana-3.0.1.tar.gz

$ sudo mv kibana-3.0.1/usr/share/elasticsearch/plugins/_site

$ sudo /etc/init.d/elasticsearch restart

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.