How does Python crawler realize the function of querying the list of people who break faith in the whole country?
This article mainly introduces the relevant knowledge of "Python crawler how to achieve the national untrusted person list query function". The editor shows you the operation process through the actual case, and the operation method is simple, fast and practical. I hope that this article "Python crawler how to achieve the national untrusted person list query function" can help you solve the problem.
I. description of requirements
The interface of Baidu is used to realize the query function of the list of people who break faith in the whole country. Enter the name to find out whether it is on the list of people who break faith in the country.
2. Python implementation
Version 1:
#-*-coding:utf-8*-import sysreload (sys) sys.setdefaultencoding ('utf-8') import timeimport requeststime1=time.time () import pandas as pdimport jsoniname= [] icard= [] def person_executed (name): for i in range (0Pie30): try: url= "https://sp0.baidu.com/8aQDcjqpAAV3otqbppnN2DJv/api.php?resource_id=6899"\" & query=%E5%A4%B1%E4%BF%A1%E8%A2%AB%E6%89 % A7%E8%A1%8C%E4%BA%BA%E5%90%8D%E5%8D%95 "\" & cardNum=& "\" iname= "+ str (name) +\" & areaName= "\" & pn= "+ str (iTun10) +\" & rn=10 "\" & ie=utf-8&oe=utf-8&format=json "html=requests.get (url). Content html_json=json.loads (html) Html_data=html_json ['data'] for each in html_data: k=each [' result'] for each in k: print each ['iname'] Each ['cardNum'] iname.append (each [' iname']) icard.append (each ['cardNum']) except: passif _ name__ = =' _ main__': name= "Guo *" person_executed (name) print len (iname) # organizes the data into data boxes # # data=pd.DataFrame ({"name": iname "IDCard": icard}) # data box deduplicates # # data1=data.drop_duplicates () print data1 print len (data1) # write data to excel# # # pd.DataFrame.to_excel (data1 "F:\\ iname_icard_query.xlsx", header=True,encoding='gbk',index=False) time2=time.time () print u'ok, crawler ends!' Print u 'total time:' + str (time2-time1) +'s'
Third, effect display
"D:\ Program Files\ Python27\ python.exe" D:/PycharmProjects/learn2017/ nationwide untrustworthy executor query. Py
Guo * * 34122319790 / 5119
Guo * * 320321198811419
Guo * * 320321198811419
three
IDCard name
034122319790 / 5119 Guo * *
1 320321198812419 Guo * *
two
Ok, crawler is over!
Total time: 7.72000002861s
Process finished with exit code 0
Version 2:
#-*-coding:utf-8*-import sysreload (sys) sys.setdefaultencoding ('utf-8') import timeimport requeststime1=time.time () import pandas as pdimport jsoniname= [] icard= [] courtName= [] areaName= [] caseCode= [] duty= [] performance= [] disruptTypeName= [] publishDate= [] def person_executed (name): for i in range (0P30): try: url= "https://sp0.baidu.com/8aQDcjqpAAV3otqbppnN2DJv/api.php?resource_id=6899"\" & query=% E5%A4%B1%E4%BF%A1%E8%A2%AB%E6%89%A7%E8%A1%8C%E4%BA%BA%E5%90%8D%E5%8D%95 "\" & cardNum=& "\" iname= "+ str (name) +\" & areaName= "\" & pn= "+ str (iTun10) +\" & rn=10 "\" & ie=utf-8&oe=utf-8&format=json " Html=requests.get (url) .content html_json=json.loads (html) html_data=html_json ['data'] for each in html_data: k=each [' result'] for each in k: print each ['iname'] Each ['cardNum'], each [' courtName'], each ['areaName'], each [' caseCode'], each ['duty'], each [' performance'], each ['disruptTypeName'] Each ['publishDate'] iname.append (each [' iname']) icard.append (each ['cardNum']) courtName.append (each [' courtName']) areaName.append (each ['areaName']) caseCode.append (each [' caseCode']) duty.append (each ['duty']) performance.append (each [' performance']) DisruptTypeName.append (each ['disruptTypeName']) publishDate.append (each [' publishDate']) except: passif _ _ name__ = ='_ main__': name= "Guo * *" person_executed (name) print len (iname) # organize the data into data frames # # # data=pd.DataFrame ({"name": iname "IDCard": icard}) detail_data=pd.DataFrame ({"name": iname, "IDCard": icard, "courtName": courtName, "areaName": areaName, "caseCode": caseCode, "duty": duty, "performance": performance,\ "disruptTypeName": disruptTypeName "publishDate": publishDate}) # data box deduplication # # data1=data.drop_duplicates () # print data1 # print len (data1) detail_data1=detail_data.drop_duplicates () # print detail_data1 # print len (detail_data1) # # write data to excel## pd.DataFrame.to_excel (detail_data1 "F:\\ iname_icard_query.xlsx", header=True,encoding='gbk',index=False) time2=time.time () print u'ok, crawler ends!' Print u' takes a total of time:'+ str (time2-time1) +'s' about how the Python crawler realizes the function of querying the list of people who break faith in the country. Thank you for your reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.
