Shulou(Shulou.com)05/31 Report--

This article mainly introduces "elasticsearch how to achieve import and export CSV". In daily operation, I believe many people have doubts about how to achieve import and export CSV in elasticsearch. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the doubts about "how to import and export CSV by elasticsearch". Next, please follow the editor to study!

Frankly speaking, this is the first python program, although it looks badly written, but you can rest assured that I tried non-toxic, and run the results, since it is correct!

Export CSV

Import csvimport sysimport loggingimport datetime from elasticsearch import Elasticsearchreload (sys) sys.setdefaultencoding ('gbk') logging.basicConfig () es = Elasticsearch () def exportCSV (indexName): count = 0 finish=False csvfile = file (indexName+'.csv','wb') writer = csv.writer (csvfile) starttime = datetime.datetime.now () searchRes = es.search (index=indexName,size=100,body= {"query": {"match_all": {}, search_type= "scan" Scroll= "60s") while True: scrollRes=es.scroll (scroll_id=searchRes ["_ scroll_id"], scroll= "60s", ignore= [400s) Res_list = scrollRes ["hits"] ["hits"] data= [] if not len (res_list) or finish: break if count==0: writer.writerow (tuple (res_list [0] ["_ source"]. Keys ()) for item in res_list: # print tuple (item ["_ source"]. Values () data.append (tuple (item ["_ source"]. Values ()) count+=1 if count > = 100000: finish=True break writer.writerows (data) csvfile.close () endtime = datetime.datetime.now () print "export size =" + str (count) print "export cost =" + str (endtime-starttime) if _ _ name _ _ = "_ _ main__": exportCSV ("test")

Import CSV

#-*-coding:utf-8-*-import csvimport sysimport osimport loggingimport datetime from elasticsearch import Elasticsearchfrom elasticsearch import helpersreload (sys) sys.setdefaultencoding ('gbk') logging.basicConfig () es= Elasticsearch () def importCSV (indexName,typeName,fileName): if not os.path.exists (fileName): print "file not found" return actions= [] if not es.indices.exists (index=indexName,allow_no_indices=True): # print "not found index" es.indices.create (index=indexName Body= {}, ignore=400) for item in csv.DictReader (open (fileName, 'rb')): actions.append ({"_ index": indexName, "_ type": typeName, "_ source": encoding (item)}) res = helpers.bulk (es,actions Chunk_size=100) es.indices.flush (index= [indexName]) return len (actions) def encoding (item): for i in item: item [I] = str (item.encode ('utf-8') return item if _ name__== "_ main__": starttime = datetime.datetime.now () result=importCSV ("test", "base") "test.csv") print "import size =" + str (result) endtime = datetime.datetime.now () print "import cost =" + str (endtime-starttime) so far On the "elasticsearch how to achieve import and export CSV" study is over, I hope to be able to solve your doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.