Shulou(Shulou.com)06/01 Report--

This article introduces the relevant knowledge of "Python crawler data decryption process analysis of a service platform". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Target website

AHR0cDovL2p6c2MubW9odXJkLmdvdi5jbi9kYXRhL2NvbXBhbnk=

Our goal today is to analyze the decryption of the data on this site, while other parameters are left to follow.

Bag grab analysis

Open the target site, you can see the site's loaded multi-page list, open the console, clear the cache and grab the package, re-refresh, make sure that the package is not missed.

You can see a more obvious packet by refreshing the website.

Looking at the return of this request, you can see that the returned data is a string of encrypted ciphertext data.

This is the data we need to analyze, novice friends may not be able to start with this kind of data, here we will use the XHR breakpoint tool to help us locate the location of data encryption.

Why mention this tool again, because both the private message column and the message area of the blog are

"Boss, how to play XHR breakpoint analysis?"

Questions like this are full.

So write here for the last time.

XHR breakpoint

Press "F12" to open the developer tools and switch to the sources panel

Find the icon below in the toolbar on the right and fill it in as I did

As for why this string is, you just have to remember that the string is part of the request you want to analyze, so give it a try. (the request to be analyzed is a xhr request.)

After hitting the XHR breakpoint, you can refresh the page and reload. Don't ask me why the breakpoint is not broken without refreshing.

After refreshing, such a situation on the page means that it is broken.

All right, part of the XHR breakpoint is over.

At this point, we have completed the part of bag capture analysis.

Location encryption

After successfully hitting the breakpoint, we need to debug the js code step by step, and we must be proficient in using the tools of the sources panel.

Now we need to clarify the requirements again, we are going to analyze how the encrypted data returned by the request is decrypted, and the location of our breakpoint is where the request was sent, so don't analyze the stack foolishly, it's useless if you don't understand it.

So we need to continue to execute the breakpoint until we get to the location in the following figure, where encrypted return data appears for the first time. (it should be the first time to appear, not paying much attention)

So slow down the debugging.

We continue to execute js until we find the decryption logic at this location.

Novice friends must ask, how do you know this is the decrypted data?

Very simple, as long as you continue to debug, a few steps later you will see the decrypted data, then the decrypted code will be in the position between the last encryption parameters and plaintext parameters, hit the breakpoint and then debug can accurately locate.

Analytical encryption

You can see that in the following figure, using JSON.parse (h (t.data)); decrypts the data, except for the json serialization code, there is only h method left, so go ahead and have a look. (the shortcut key is F11)

You can see that the logic of the h function is very clear.

The decryption result can also be seen through the console.

The whole logic is very simple, how to buckle it?

No need to buckle, just import the encryption library of js.

Var CryptoJs = require ('crypto-js')

Var f = CryptoJs.enc.Utf8.parse ("jo8j9wGw%6HbxfFn")

Var m = CryptoJs.enc.Utf8.parse ("0123456789ABCDEF")

Function h (t) {

Var e = CryptoJs.enc.Hex.parse (t)

, n = CryptoJs.enc.Base64.stringify (e)

, a = CryptoJs.AES.decrypt (n, f, {

Iv: m

Mode: CryptoJs.mode.CBC

Padding: CryptoJs.pad.Pkcs7

})

, r = a.toString (CryptoJs.enc.Utf8)

Return r.toString ()

}

/ / t is the encrypted data returned

Console.log (h (t))

Just give me the code.

The content of the decryption result "Python crawler data decryption process analysis of a service platform" is introduced here. Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.