Shulou(Shulou.com)06/01 Report--

This article mainly explains what tools Android7.0 uses to grab the package. Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn what tools Android7.0 uses to grab the bag.

Preface

Recently, when I was testing an app, I tried to grab data packets and found that the previous method of use was invalid, because with the improvement of the Android version, the restrictions on certificates became more and more stringent, and the old machine around me was left in school, not around, so I had to study how to get around this restriction.

After searching by one side, I finally found a relatively simple way. Here is an article to record it.

Process

First download the required tools:

Charles-- https://www.charlesproxy.com/download/

Charles-Crack (capable students, please support the authentic version)-- https://github.com/8enet/Charles-Crack

VirtualXposed-- https://github.com/android-hacker/VirtualXposed/releases

TrustMeAlready-- https://github.com/ViRb3/TrustMeAlready/releases

PC end

After the PC side downloads and installs it, run Charles.

Click Proxy → Proxy Settings on the menu bar, and check it as follows.

Then click on the menu bar: Proxy → SSL Proxying Settings

Follow the figure below to configure:

If you determine the Host and Port information to be crawled in advance, you can change it yourself.

Then click Proxy → Windows Proxy (note: different versions have different names), and local traffic will no longer be grabbed.

Then click Help → SSL Proxying → Install Charles Root Certificate on a Mobile Device or Remote Browser to mount the certificate to the address chls.pro/ssl.

The mobile phone uses a browser to open the above address and download the certificate to the local. You can also choose to click on the menu bar: Help → SSL Proxying → Save Charles Root Certificate..., to download the certificate to PC and send it to the phone via data line, QQ and other methods.

Mobile phone end

First of all, you need to connect the phone and the computer to the same network.

Open WLAN and select the corresponding WIFI for proxy configuration. As shown in the following figure:

The host name is the private network IP on the PC, and the port is the Port content in the Http Proxy in the Proxy Settings configuration when the Charles is configured.

Students with unclear content on the PC can open the terminal and run the following command:

Windows: ipconfig

Linux: ifconfig

Mac: ifconfig

After a successful proxy, all traffic on the mobile phone will pass through the Charles on the PC. If it is configured for the first time, Charles will pop up a window indicating a new connection and click the Allow button.

Secondly, you need to install the certificate of Charles on the computer side.

If you choose Install Charles Root Certificate on a Mobile Device or Remote Browse on the PC side, you first need to visit the chls.pro/ssl URL through a mobile browser to obtain the certificate file.

The mobile phone I use here is Xiaomi 8 Lite. Different mobile phone brands / models may have different methods of installing certificates. You can get the installation method through the search engine.

Click: set → more settings → system security → encryption and credentials → installed from the SD card, and then select the certificate you downloaded in the file browser.

Then give the certificate credential a name, which can be filled in freely, and choose VPN and application according to the purpose of the credential.

After installation, click the trusted credential → user, and you can see the Charles certificate that has just been added-XK72 Ltd.

In the past, we only need to execute here to successfully crawl the packet. However, due to the update of the version of Android, there are restrictions on the certificates added by users themselves, resulting in HTTPS packets that cannot be crawled directly. In Charles, we will find that the traffic of HTTPS shows Unknown.

My choice here is to use the VirtualXposed tool to bypass, if you do not want to use this method, you can also consider Root the phone and add the Charles certificate to the system-level certificate, but this is beyond the scope of this article.

Download two apk files, VirtualXposed and TrustMeAlready, and run them after installing VirtualXposed.apk.

In VirtualXposed, go to the settings page, click add Apps, select the application for which you want to grab traffic, and the TrustMeAlready.apk file to install. Note: since TrustMeAlready.apk is not installed, you need to click the + button and select the apk file in the corresponding directory.

Then go back to the settings page, click Module Management, and you can see that the TrustMeAlready we just added is in it. Click the box on the right to check, then go back to the settings page, and click restart and OK at the bottom.

At this point, we have completed all the operations, and then you can open any app within the VirtualXposed framework to successfully get the HTTPS traffic.

On the use of Charles, this article will not repeat, there are many tutorials on the Internet, just take a little time to search.

Analysis.

Why can't Android7.0+ intercept HTTPS traffic directly

In the Android developer platform documentation, we can find the following paragraph:

By default, secure connections from all applications (using protocols such as TLS and HTTPS) trust the pre-installed system CA, while applications for Android 6.0 (API level 23) and earlier also trust the CA store that users add. Applications can customize their own connections using base-config (application-wide customization) or domain-config (domain-wide customization).

Custom trusted CA | Android Developer

The above shows that Android version 6.0 and above, by default, the application will not trust the CA certificate added by the user, so we can not get the HTTPS data issued by the application completely using the old method.

Principle of VirtualXposed+TrustMeAlready implementation

To put it simply, VXP is equivalent to a virtual machine on the phone, creating a virtual space on the original system of the phone, similar to the effect of sandboxie, and through the TrustMeAlready plug-in, you can HOOK all the API used to verify the SSL certificate in APK (for details, you can click on the "JustTrustMe principle Analysis" in the reference), thus bypassing the certificate verification, thus achieving the effect of https grabbing the package.

What is Virtual Xposed?

Xposed

It is well known that Xposed is an open source Android framework developed by rovo89 from foreign XDA forums. It is a special Android App, and its main function is to provide a new application platform. After players install the Xposed framework, they will be able to install more system-level applications and achieve many magical functions through the platform built by the Xposed framework.

The principle of the Xposed framework is to modify the system file, replace the / system/bin/app_process executable file, load the additional jar file (/ data/data/de.robv.android.xposed.installer/bin/XposedBridge.jar) when starting Zygote, and perform some initialization operations (execute the main method of XposedBridge). Then we can do some hook operations in this Zygote context.

What is really powerful about Xposed is the methods it can call with hook. When you decompile and modify apk, you can insert xposed commands into it, so you can inject your own code before and after the method call. Github open source address: https://github.com/rovo89/Xposed because the biggest drawback of Xposed is that the device needs root, and you need to restart the phone after writing the plug-in module (of course, there are ways not to restart it), so you have VirtualApp.

VirtualApp

VirtualApp is an App virtualization engine (VA).

VirtualApp creates a virtual space within your App (constructed a virtual systemserver), in which you can install, start, and uninstall APK at will, all isolated from the outside, just like a sandboxie.

APK running in VA does not need to be installed externally, that is, VA supports running APK without installation.

Those who are familiar with the boot process of android system should know that each services is started by system server to start a series of system core services (AMS,WMS,PMS, etc.) ViratualApp is to build a virtual system_process process, in which there are a series of core services.

The main technology of VirtualApp is realized by reflection and dynamic proxy.

Github open source address: https://github.com/asLody/VirtualApp

VirtualXposed

VirtualXposed is based on the implementation of the Xposed module running in a non-ROOT environment based on VirtualApp and epic (supports 5.0 8.1).

Github open source address: https://github.com/android-hacker/VirtualXposed Source: http://jackzhang.info/2018/04/09/VirtualXposed/

Why does HTTPS need to install the CA certificate of the package grab tool?

In order to understand this problem, we first need to understand the principle of SSL/TLS encryption.

Generally speaking, both SSL and TLS are asymmetric encrypted, with a public key and a private key. The public key is public, the private key is private, and exists on the server side. The content returned by the server is encrypted by the private key, and the client needs to use the public key for decryption. Similarly, the data on the client side is encrypted by public key and decrypted by private key.

And we all know that with SSL, we can protect our site from man-in-the-middle attacks. Then what is a man-in-the-middle attack?

For example, if user A wants to use a computer to access the website http://example.com, and this computer has been compromised by attacker B, then attacker B can modify the hosts file on computer A to point the parsing of example.com to B's own server, so that user An is "unknowingly" caught. If the website uses SSL/TLS encryption, user A needs to request the content of the public key from the server when accessing https://example.com, and because the public key is placed in the CA certificate, and the CA certificate is usually issued by the relevant authoritative CA institutions (the authority is determined by operating system giants such as Microsoft), similar to our Civil Affairs Bureau can release shenfenzheng. This makes it impossible for an attacker to forge a CA certificate, because after receiving the CA certificate, the client will verify it according to different authoritative CA institutions, and if the authority issuing the certificate is not authoritative enough (which makes the authority will not issue the CA certificate at will, lest its own authority be revoked), it will not be trusted by the system. This series of operations enable websites that use SSL/TLS not to be affected by man-in-the-middle attacks.

OK back to the point, so what does this HTTPS grab have to do with man-in-the-middle attacks? In fact, the principles of both are the same, except that the role of the attacker becomes a bag grabbing tool.

So, the question this time is how does a man-in-the-middle attack work in HTTPS communications?

We note that the biggest difficulty of man-in-the-middle attack is the authority of CA certificates, and we cannot apply for certificates from authoritative CA institutions without the right to resolve domain names. That being the case, why don't we consider opening an authority ourselves, so that the certificates we generate will be trusted?

At this point, we need to install the CA certificate of our package capture tool, which is different from the certificate applied to the authority by the domain name owner, who is the root certificate.

Because the verification process of the CA certificate of the domain name is also asymmetric encryption verification, that is, the verification of the CA certificate is decrypted and verified by the public key in the root certificate. Usually, the root certificates of a number of authorities have been trusted by default in the operating system.

So, when we add our own root certificate to the operating system, we are equivalent to "opening" an authoritative CA organization, which can solve the previous problem.

Borrow a net picture:

At this point, I believe you have a deeper understanding of "what tools Android7.0 uses to grab packages". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.