Shulou(Shulou.com)06/02 Report--

The content of this article is mainly around the es+flask search small project to achieve paging and highlight how to write the sample code about, the article is clear and clear, very suitable for beginners to learn, worth reading. Interested friends can follow the editor to read together. I hope you can get something through this article!

Environment

Front end: html,css,js,jQuery,bootstrap

Backend: flask

Search engine: elasticsearch

Data source: home of so-and-so

Project presentation

Project catalog

Main source code

Get the data source and write to es

From lxml import etreefrom concurrent.futures import ThreadPoolExecutorfrom elasticsearch import Elasticsearchfrom elasticsearch import helpersimport requestsheaders = {'user-agent':' ua'} es = Elasticsearch () if not es.indices.exists (index='car'): es.indices.create (index='car', mappings= {'properties': {' url': {'type':' text'}) 'img': {' type': 'text'},' title': {'type':' text'}, 'desc': {' type': 'text'}) def task (url Page): res = requests.get (url, headers) text = res.text tree = etree.HTML (text) ul_list = tree.xpath ('/ / ul [@ class= "article"]') actions = [] for ul in ul_list: li_list = ul.xpath ('. / li') for li in li_list: url = li.xpath ('. / a _) Img = li.xpath ('. / an if title), desc = li.xpath ('. / a/p/text ()'), title = li.xpath ('. / a/h4/text ()') if title: doc = {'_ index': 'car' 'url': f'https: {url [0] [0]},' img': img [0] [0], 'desc': desc [0] [0],' title': title [0],} actions.append (doc) helpers.bulk (es) Actions=actions) print (f' page {page} complete!') Def main (): with ThreadPoolExecutor () as pool: for i in range (1,11): url= f 'https://www.autohome.com.cn/all/{i}/' pool.submit (task, url=url,page=i) if _ _ name__ ='_ main__': main ()

View function

From flask import Blueprintfrom flask import requestfrom flask import render_templatefrom flask import jsonifyfrom web.ext import esfrom pprint import pprintsearch_bp = Blueprint ('search', _ _ name__, url_prefix='/search') @ search_bp.route (' /', methods= ['get') 'post']) def search (): if request.method =' GET': return render_template ('search.html') elif request.method =' POST': content = request.values.get ('content') size = 10 current = int (request.values.get (' current','0')) if content: res = es.search (index='car') Query= {'match': {"title": content}}, highlight= {"pre_tags": "," post_tags ":" "fields": {"title": {}}, size=1000) else: res = es.search (index='car', query= {'match_all': {}}) Size=1000) new_res = res ['hits'] [' hits'] total = int (res ['hits'] [' total'] ['value']) need_page = (total / / size) + 1 data = {' res': new_ res [current * size:current * size + size], 'need_page': need_page 'total': total} return jsonify (data)

Front end

General Search .title {font-size: 25px; font-weight: 10;} .result {color: red } Toggle navigation General Search search General found about 0 function getData (current) {let content=$ ('# content'). Val () let tags=$ ('# tags') $('# count'). Text () tags.empty () $.ajax ({url: ") Type:'post', data: {'content':content,' current':current}, dataType:'JSON' Success:function (res) {let length=res.total let need_page=res.need_page $('# count') .text (length) $.each (res.res,function (index) Value) {let source=value._source let title=source.title let highlight=value.highlight if (highlight) {title=highlight.title} let div= `

${title}

${source.desc}

`tags.append (div)}) $('# page'). Empty () for (let iTuno

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.