Shulou(Shulou.com)06/01 Report--

This article introduces you how Python simulates landing on a teacher education network, the content is very detailed, interested friends can refer to, hope to be helpful to you.

I have to say, the recent Python is very popular, I also know a little bit about it, and try to use Python to crawl the data on the website.

However, some data can only be obtained after logging in. Every year, we have to go to a teacher education network to learn some public needs subjects. Let's try this website. The key is the analysis of the website.

Open the browser, enter the website URL http://www.jste.net.cn, press F12 to bring up the developer tool of the browser, select Network, and check Preserve log to prevent the loss of information when switching web pages

Enter the account number on the web page, enter the password "123456", enter the verification code "abcde", do not enter the correct verification code, otherwise the password is wrong 5 times, the account will be locked by the website for 30 hours, and the verification code can be wrong at will.

After logging in (of course, if you can't log in, you will jump to another landing page) and see the data exchange with the server in the developer's tool.

The first is for the get CAPTCHA picture, and the second is for submitting data to the website. Click on the second message.

This is a Post request, focusing on the submission data in the red box. RandomCode is the input CAPTCHA. XPowerY should be the location of the button control clicked. There is no submission of this data after cookie. It can be ignored that returnURL and appId,encrypt are the same every time, regardless of it. The key point is the two key values of reqId and req. ReqId guesses that it is the time stamp taken when the button is clicked. You can copy this data to verify that the Unix timestamp (Unix timestamp) conversion tool unit is selected in milliseconds. It is indeed the time when the data has just been submitted, and there is only one data left. The value of this key is very long. Let's find out where the data came from.

You can see that under login.jsp, you can see encode.js, string.js and des.js. You can see these are used to encrypt the submitted data from the name. Right-click login.jsp and select "Open in Sources panel".

You can jump to the "Source" tab and see the source code of 'login.jsp'. If the format is confused, for example, all the code is in one line, which is not easy to view, you can click at the bottom of the interface.

The developer tool will automatically reformat your code.

Take a closer look at the login.jsp code and see

Function doOk (frm) {var el = frm.elements ["loginName"]; var loginName = el.value.replace (/ / g, ""); el.value = loginName; if (isEmpty (loginName)) {alert ("Please enter login name"); el.focus (); return false;} el = frm.elements ["pwd"]; el.value = el.value.replace (/ / g, "); var pwd=el.value If (isEmpty (el.value)) {alert ("Please enter login password"); el.focus (); return false;} var d = new Date (); pwd = encode (loginName, pwd); / / password is encrypted for the first time, you can follow frm.elements ["encrypt"]. Value= "1"; var validCode= "; el=frm.elements [" randomCode "]; if (el) {el.value=el.value.replace (/ / g,") If (isEmpty (el.value)) {alert ("Please enter login password"); el.focus (); return false;} validCode=el.value;} loginName=encodeURI (loginName); / / avoid Chinese problems with URL encoding var reqId= (new Date ()). GetTime () + "; / / get the timestamp to reqId var str=strEnc (loginName+"\ n "+ pwd,reqId,validCode); / / key encryption code, you can follow up and analyze frm.elements [" loginName "] .codes =" true " Frm.elements ["pwd"] .value = pwd; frm.elements ["pwd"] .value = "true"; frm.elements ["req"] .value = str; frm.elements ["reqId"] .value = reqId; return true;}

Find this code, which is mainly the part of checking the input, focusing on these two places.

Pwd = encode (loginName, pwd)

The password is encrypted for the first time here

LoginName=encodeURI (loginName); / / avoid Chinese problems var reqId= (new Date ()). GetTime () + "; var str=strEnc (loginName+"\ n "+ pwd,reqId,validCode)

First line: encode the user name in URL format

The second line, take the timestamp and assign it to reqId

The third line passes in the user name, the encrypted password and verification code are verified, and the return value of the function is assigned to the variable str, which is the value of the req that submitted the data.

Set the breakpoint at the two encryption function entrances, and if the developer tool sets the breakpoint, just click the mouse on the line number of the code. after setting the breakpoint, enter the user name and password and verification code again, resubmit, and the program is cut off:

F11 enters the first breakpoint step by step. Here, you need to reformat the code by clicking the square brackets at the bottom of the interface. After stepping forward, you can see:

Var _ $_ 7151 = ["encode", "ABCDEFGHIJKLMNOP", "QRSTUVWXYZabcdef", "ghijklmnopqrstuv", "wxyz0123456789+/", "=", "charCodeAt", "charAt", "length", "join", "reverse", "split"] Window [_ $_ 7151 [0]] = function (c, e) {function a (p) {var Q = _ $_ 7151 [1] + _ $7151 [2] + _ $7151 [3] + _ $7151 [4] + _ $7151 [5]; p = encodeURI (p); var r = _ $7151 [6]; var g, h, j = _ $_ 7151 [6]; var k, l, m, o = _ $7151 [6]; var b = 0 Do {g = p [_ $_ 0 [7]] (baked +); / / the first character h = p [_ $_ 7151 [7]] (baked +); / / the second character j = p [_ $_ 7151 [7]] (baked +); / / the third character k = g > > 2; / / get k l = (g & 3) > 4); / get I m = ((h & 15) > 6); / get m = j & 63) / / get o if (isNaN (h)) {/ / if there is no second character m = o = 64 / / then take the 64th character in the table to replace} else {if (isNaN (j)) {/ / if there is no third character o = 64 / / then replace the 64th character in the table}} R = r + Q [_ $_ 7151 [8]] (k) + Q [_ $_ 7151 [8]] (l) + Q [_ $_ 7151 [8]] (m) + Q [_ $7151 [8]] (o); g = h = j = _ $7151 [6]; k = l = m = o = _ $7151 [6]} while (b)

< p[_$_7151[9]]);;return r } var d = c[_$_7151[9]]; var f = a(e)[_$_7151[12]](_$_7151[6])[_$_7151[11]]()[_$_7151[10]](_$_7151[6]); for (var b = 0; b < (d % 2 == 0 ? 1 : 2); b++) { f = a(f)[_$_7151[12]](_$_7151[6])[_$_7151[11]]()[_$_7151[10]](_$_7151[6]) } ;return f} 这个函数返回的 f 就是密码第一次加密后的结果了，这个代码是用什么工具变成这样的不太清楚，如果出现 _$_7151[n] 这样的字符可以查询代码最上面的列表 代换，大致过程不详说，跟一遍就知道了，就是循环从密码中取三个字符 g、h、j，然后将三个字符的ascii码左移或右移，或和其他结果加加减减，得到的结果 k、l、m、o 查询表格替换字符，如果密码长度不是 3 的整数倍，则查表结果用 "=" 替换，将循环得到的查表结果依次连接，并反序，得到一个密码加密后的密码 至少将密码进行两次这样的加密计算，如果用户名的长度是奇数，再进行一次加密，加密的过程只需要复制代码到 python 中，修改成 python 的格式就可以了。 步过了对密码的第一次加密后，继续步进上面设下的第二个断点 function strEnc(data,firstKey,secondKey,thirdKey){ var leng = data.length;//取 data 的长度 var encData = ""; var firstKeyBt,secondKeyBt,thirdKeyBt,firstLength,secondLength,thirdLength; if(firstKey != null && firstKey != ""){ firstKeyBt = getKeyBytes(firstKey);//取 firstkey 在每个字符之间插入一个字节的 0 firstLength = firstKeyBt.length;//取得插入 0 后的长度 } if(secondKey != null && secondKey != ""){ secondKeyBt = getKeyBytes(secondKey);//取 secondkey 在每个字符之间插入一个字节的 0 secondLength = secondKeyBt.length;//取得插入 0 后的长度 } if(thirdKey != null && thirdKey != ""){ //登陆过程中，并没用到 thirdkey，即 thirdKey = None thirdKeyBt = getKeyBytes(thirdKey);//取 thirdkey 在每个字符之间插入一个字节的 0 thirdLength = thirdKeyBt.length;//取得插入 0 后的长度 } if(leng >

0) {if (leng

< 4){ 如果 data 的长度＜4，因为跳过，代码用省略号替换 //省去一些代码…… }else{ var iterator = parseInt(leng/4);//data 的长度除 64，得到循环次数 var remainder = leng%4; //data 的长度是否是 64 位的整数倍，保存余数 var i=0; for(i = 0;i < iterator;i++){ //开始循环 var tempData = data.substring(i*4+0,i*4+4); //循环取 data 的64 位 var tempByte = strToBt(tempData);//转换成 bits var encByte ; if(firstKey != null && firstKey !="" && secondKey != null && secondKey != "" ){ var tempBt; var x,y; tempBt = tempByte; for(x = 0;x < firstLength ;x ++){ tempBt = enc(tempBt,firstKeyBt[x]);//循环从firstkey 中取得64 位做密钥，依次对 data 中的某一段加密 } for(y = 0;y < secondLength ;y ++){ tempBt = enc(tempBt,secondKeyBt[y]);//循环从second中取得64 位做密钥，依次对 data 中的某一段加密 } encByte = tempBt;//保存加密结果 } //………… if(remainder >

0) {/ / if data has extra length, less than 64 bits var remainderData = data.substring (iterator*4+0,leng); var tempByte = strToBt (remainderData); / / divide the rest into four 16-bit arrays var encByte; if (firstKey! = null & & firstKey! = "& & secondKey! = null & & secondKey! =" & & thirdKey! = null) {var tempBt; var x dagronomy; tempBt = tempByte; for (x = 0x x

< firstLength ;x ++){ tempBt = enc(tempBt,firstKeyBt[x]);循环从firstkey 中取得64 位做密钥，依次对 data 中的某一段加密 } for(y = 0;y < secondLength ;y ++){ tempBt = enc(tempBt,secondKeyBt[y]);循环从secondkey中取得64 位做密钥，依次对 data 中的某一段加密 } encByte = tempBt;//保存加密结果 } encData += bt64ToHex(encByte);//将加密后的文本转为16进制文本 } } } return encData;//返回加密结果} 这是一段循环进行 DES 加密的代码，先将data, firstkey, secondkey进行字符间插入一个字节的0, 然后不是 64 位整数倍长度的从上面代码看，相当于在后面补上 0 了 从data中取出一段64位数据，循环用 firstkey 和 second 中的 64 位做密钥，层层加密，得到的结果和 data 中其他 64 位加密的结果串联后就是 req 的值了 因为 key 都是 64 位的，再加上本身 sources 中也看到了 DES.js 文件，所以 enc(tempBt,secondkeyBt)应该就是 DES 算法了。 但是自己写代码模拟登陆确发现结果和自己跟的结果不同，从代码中看，DES 采用了 ECB 模式，不是 CBC 模式，PAD_mode 也没问题，都64位，不需要 DES 自己填充啊。没办法，只得硬着头皮继续跟进 DES 加密的代码 我们知道，DES 加密需要先对 key 进行 置换，得到 56 位密钥，标准的 DES 都有个置换表，正常的 DES 置换表是这样的 Permutation and translation tables for DES __pc1 = [56, 48, 40, 32, 24, 16, 8, 0, 57, 49, 41, 33, 25, 17, 9, 1, 58, 50, 42, 34, 26, 18, 10, 2, 59, 51, 43, 35, 62, 54, 46, 38, 30, 22, 14, 6, 61, 53, 45, 37, 29, 21, 13, 5, 60, 52, 44, 36, 28, 20, 12, 4, 27, 19, 11, 3 ] 即将 key 的第 56 位放到第 0 位，第 48 位放到第 1 位…………最后置换出 56 位的 key，再分成 2 个28 密钥，循环左移和右移，然后 对 IP 置换后的 data 加密，进行 Sbox 盒替换 和 Pbox 替换，再进行一次 IP-1 置换得到密文，解密算法一样。 但跟进 DES 加密函数没多久就发现问题了，找到密钥置换的函数 var keys = generateKeys(keyByte); 并跟进： function generateKeys(keyByte){ var key = new Array(56); var keys = new Array(); keys[ 0] = new Array(); keys[ 1] = new Array(); keys[ 2] = new Array(); keys[ 3] = new Array(); keys[ 4] = new Array(); keys[ 5] = new Array(); keys[ 6] = new Array(); keys[ 7] = new Array(); keys[ 8] = new Array(); keys[ 9] = new Array(); keys[10] = new Array(); keys[11] = new Array(); keys[12] = new Array(); keys[13] = new Array(); keys[14] = new Array(); keys[15] = new Array(); var loop = [1,1,2,2,2,2,2,2,1,2,2,2,2,2,2,1];//看到了循环移位的表，没看到置换表 for(i=0;i 第 1 位，40 位 -->

No. 2,. Bit 0-- > bit 7

The original key 57-> 8th, 49-- > 9th, 41-- > 10th,. 1 bit-- > 15th bit

.

Finally, discard the original key's 63rec. 55jr. 47pr. 39pr. 31g. 23g.

The standard DES library cannot be used directly in python. You can copy the pyDes.py file from the standard library to the same directory of the project, rename it to Des,py, and import it into the project.

From Des import *

In addition, find the replacement table of key in Des.py and modify it to

_ _ pc1 = [56, 48, 40, 32, 24, 16, 8, 0, 57, 49, 41, 33, 25, 17, 9, 1, 58, 50, 42, 34, 26, 18, 10, 2, 59, 51, 43, 35, 27, 19, 11, 3, 60, 52, 44, 36, 28, 20, 12, 4, 61, 53, 45, 37, 29, 21, 13, 5, 62, 54, 46, 38, 30, 22, 14, 6]

You can use Des normally.

Finally, the python code is attached:

From Des import * from urllib.parse import quotefrom time import time, sleepfrom PIL import Imageimport requestsimport sysfrom bs4 import BeautifulSoups = requests.session () headers = {'Cache-Control':' max-age=0', 'Connection':' keep-alive', 'Referer':' http://www.jste.net.cn/uids/login.jsp', 'User-Agent':' Mozilla/5.0 (Windows NT 10.0 WOW64) AppleWebKit/537.36 (KHTML, like Gecko)\ Chrome/58.0.3029.110 Safari/537.36 SE 2.x MetaSr 1.0'} def custom_encode (data): # too lazy to comment, copy it directly from js and change it to python code tab = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=' data_bytes = list (data.encode ()) while len (data_bytes)% 3! = 0: data_bytes.append (0) b = 0 length = len (data_bytes) r =' 'while b

< length: g = data_bytes[b] h = data_bytes[b + 1] j = data_bytes[b + 2] k = g >

> 2 m = (g & 3) > 4) n = (h & 15) > 6) o = j & 63 third_char ='='if h = = 0 else tab [n] fourth_char ='='if j = = 0 else tab [o] r = r + tab [k] + tab [m] + third_char + fourth_char b = b + 3 return r [::-1] # reverse output def encode_pwd (str_name Str_pwd): encoded_pwd = custom_encode (str_pwd) encoded_pwd = custom_encode (encoded_pwd) # encrypt the password twice in a row if len (str_name)% 2 = = 1: encoded_pwd = custom_encode (encoded_pwd) # if the user name length is odd Then encrypt return encoded_pwddef strenc (data, firstkey) again Secondkey): bts_data = extend_to_16bits (data) # extends data length to a multiple of 64 bits bts_firstkey = extend_to_16bits (firstkey) # extends first_key length to a multiple of 64 bit bts_secondkey = extend_to_16bits (secondkey) # extends second_key length to a multiple of 64 bit I = 0 bts_result = [] while I < len (bts_data): bts_temp = bts_ data [I: I + 8 ] # divide the data into segments every 64 bits Segmented encryption j, k = 0,0 while j < len (bts_firstkey): des_k = des (bts_ firstkey [j: J + 8], ECB) # take 64 bits of first_key as keys bts_temp = list (des_k.encrypt (bts_temp)) j + = 8 while k < len (bts_secondkey): des_k = des (bts_ secondkey [k: K + 8]) ECB) # take 64 bits of second_key as the key bts_temp = list (des_k.encrypt (bts_temp)) k + = 8 bts_result.extend (bts_temp) I + = 8 str_result =''for each in bts_result: str_result + =' X'% each # encrypt each segment of data respectively Concatenate into a string return str_resultdef extend_to_16bits (data): # inserts 0 before each character of the string into 16 bits, followed by 0 Make its length 64-bit integer multiple bts = data.encode () filled_bts = [] for each in bts: filled_bts.extend ([0, each]) # insert 0 while len (filled_bts)% 8! = 0: # the length extends to a multiple of 8 filled_bts.append (0) # is not a multiple of 8, followed by 0 For DES encryption, grouping return filled_btsdef get_rand_code (): random_code_url = r 'http://www.jste.net.cn/uids/genImageCode?rnd=' time_stamp = str (int (time () * 1000)) random_code_url + = time_stamp try: req = s.get (random_code_url, headers=headers, stream=True) with open (' rand_code.jpg' 'wb') as f: for chunk in req.iter_content (chunk_size=1024): f.write (chunk) except requests.RequestException: print (' network link error Please try again later / (thanks) / ~') sys.exit () with Image.open ('rand_code.jpg') as img: img.show () def login_site (reqid, randomcode, reqkey): post_data = {' randomCode': randomcode, 'returnURL': None,' appId': 'uids',' site': None, 'encrypt': 1,' reqId': reqid 'req': reqkey} try: req = s.post (' http://www.jste.net.cn/uids/login.jsp', headers=headers, data=post_data) print ('Status Code:%s'% req.status_code) # I don't know why the browser successfully logged in and returned 302 200 if 'Set-Cookie' in req.headers.keys (): # fortunately, when you see Set-Cookie in response, you log in successfully: return True else: return False except requests.RequestException: print (' network link error) Please try / (login) / ~') return Falsedef main (): print ('.center (100,') uname = input (' Please enter your user name:') pwd = input ('Please enter your login password:') get_rand_code () secondkey = input ('Please enter the CAPTCHA you see:) # to get the CAPTCHA as second_key Firstkey = str (int (time () * 1000)) # as the value of randomCode when the data is submitted, as the first_key Submit the data as the value of reqId crypt_pwd = encode_pwd (uname, pwd) # encrypt the entered password for the first time data = quote (uname) +'\ n'+ crypt_pwd # user name URI encoded and password encrypted text link waiting to be encrypted by DES post_req = strenc (data, firstkey, secondkey) # mainly DES calculation Submit data as the value of req if login_site (reqid=firstkey, randomcode=secondkey, reqkey=post_req) is True: print ('.center (100,-') print ('login successful, O (∩ _ ∩) O ~.') Try: req = s.get ('http://www.jste.net.cn/train/credit_hour/top.jsp') # Open a web page to test soup = BeautifulSoup (req.text,' html5lib') # the web page is multi-frame Test to access the text print (soup.select ('.b') [0] .text.replace ('\ nlinks,') .replace ('',') except requests.RequestException in the TOP framework: print ('network link error, please try again later / (resume) / ~') if _ _ name__ = ='_ main__': # launcher main ()

Test results:

Finally, after thinking about it, the data of many websites are submitted in explicit code, or simply encrypted, and this site has spent some time on encryption.

But the biggest problem with the js script is that others can see the source code. Although the js file was deleted immediately after the successful landing of the website, it will be found as long as it appears. I searched the way to hide the source code on the Internet, but the level is not good. I haven't learned java, nor do I understand it.

Finally, add: DES encrypted data data is the user name "URL format + newline + password encrypted text for the first time"

Firstkey is the timestamp obtained at the time of submission, and secondkey is the verification code entered.

About Python how to simulate landing on a certain network teacher education network to share here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.