Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "what is the Android signature mechanism". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

What is an Android signature?

Students who are familiar with HTTPS communication know that in message communication, at least two problems must be solved: one is to ensure the authenticity of the source of the message, and the other is to ensure that the message will not be tampered with by a third party.

Similarly, when installing apk, you also need to ensure that the source of the apk is authentic and that the apk has not been tampered with by a third party. In order to solve this problem, Android officials require developers to sign the apk, which is the process of encrypting apk. To understand how to implement a signature, you need to understand two basic concepts: message digest, digital signature, and digital certificate.

Message digest

Message digest (Message Digest), also called digital digest (Digital Digest) or digital fingerprint (Finger Print). To put it simply, the message digest is to execute an one-way Hash function on the message data to generate a fixed-length hash value, which is the message digest.

The encryption Hash function mentioned above is the message digest algorithm. It has the following characteristics:

No matter how long the message is entered, the length of the calculated message digest is always fixed. For example, the message digested by MD5 algorithm has 128bits, the message digested by SHA-1 algorithm finally has 160bit output, and the variant of SHA-1 can produce 192bit and 256bit message digest. It is generally believed that the longer the final output of the summary, the more secure the summary algorithm.

The message digest looks "random". These bits seem to be cluttered together at random. A large number of inputs can be used to verify whether the output is the same, in general, different inputs will have different outputs, and the output summary messages can pass the randomness test. However, a summary is not really random, because if you use the same algorithm to summarize the same message twice, the result must be the same; if it is really random, it cannot be reproduced anyway. Therefore, the message digest is "pseudo-random".

The message digest function is an one-way function, that is, it can only perform a positive information summary, but can not recover any messages from the summary, or even can not find any information related to the original information at all. Of course, you can use the brute force attack method, that is, try every possible information, calculate its summary to see if it is the same as the existing summary, if you do so, the summary message will eventually be restored. But in fact, the information you want to get may be one of the infinite messages, so this brute force attack is almost ineffective.

No one can find a "collision" in a good summary algorithm. In other words, two messages cannot be found so that their summaries are the same. Although "collisions" are certain to exist (because Hash with short summaries generated by Changming text is bound to cause collisions). That is, for a given summary, it is impossible to find a piece of information so that the summary happens to be given.

Because of the above characteristics, message digest algorithm is widely used in the field of "digital signature" as a digest algorithm for plaintext. The famous message digest algorithms include RSA's MD5 algorithm and SHA-1 algorithm and a large number of variants. SHA-256 is an upgraded version of SHA-1, and now the default algorithm used for Android signature has been upgraded to SHA-256.

It is precisely because of this feature that message digests are suitable for verifying the integrity of data. For example, if we download a large file BigFile during network transmission, we will download both BigFile and BigFile.md5,BigFile.md5 to save the summary of BigFile from the network. We will generate a message summary of BigFile locally and compare it with BigFile.md5. If the content is the same, it means that the download process is correct.

Digital signature

The function of digital signature is to ensure the integrity of information transmission, the identity authentication of the sender, and prevent the occurrence of repudiation in the transaction. Digital signature technology is to encrypt the summary information with the sender's private key and transmit it to the receiver together with the original text. Only by using the sender's public key can the receiver decrypt the encrypted summary information and then use the HASH function to generate a summary message to the original text, which is compared with the decrypted summary information. If it is the same, it means that the information received is complete and has not been modified during the transmission, otherwise the information has been modified, so the digital signature can verify the integrity of the information.

For example, when RSA is used as a digital signature scheme, its usage process is as follows: this kind of signature actually encrypts the message with the private key of the source, and the encrypted message becomes the signature body; while it is verified with the corresponding public key, if the message decrypted by the public key is the same as the original message, the message is complete, otherwise the message is incomplete.

RSA is exactly the opposite of public key cryptography for message secrecy. Because only the source has its own private key, others cannot re-encrypt the source message, so even if someone intercepts and changes the source message, the signature body cannot be regenerated, because the correct signature body can only be formed with the source's private key.

Similarly, as long as the sink verifies whether the message decrypted with the public key of the source is the same as the plaintext message, it can know whether the message has been changed, and can verify whether the message really comes from the intended source. it also enables the source not to deny the message that has been sent. So this can complete the function of digital signature.

However, this scheme is too simple, it can only ensure the integrity of the message, but not the confidentiality of the message. Moreover, this scheme needs to encrypt all messages, which is very inefficient when the length of the message is relatively large, mainly due to the inefficiency of the encryption and decryption process of the public key system. Therefore, this kind of scheme is generally undesirable.

Almost all digital signature schemes are used with a fast and efficient digest algorithm (Hash function). When the public key algorithm is combined with the digest algorithm, it constitutes an effective digital signature scheme.

Signing certificate

Through digital signature technology, the problem of reliable communication can indeed be solved. Once the signature is passed, the receiver can be sure that the message was sent by the desired sender, and the sender cannot deny that the message has been sent.

Have you noticed that the premise of the digital signature method mentioned earlier is that the recipient of the message must get the correct public key in advance. If the public key is tampered with in the first place, the bad guy will be regarded as a good person, and the message sent to you by the real sender will be regarded as invalid. Moreover, in many cases, there is no information channel to communicate the public key in advance.

So how to ensure the security and credibility of the public key? This will be solved by digital certificates.

A digital certificate is a file digitally signed by a Certificate Authorization (Certificate Authentication) center that contains information about the owner of the public key and the public key. The format of digital certificates generally adopts the X.509 V3 international standard, and a standard X.509 digital certificate usually contains the following:

Certificate issuer (Issuer): which institution (CA center) issued the certificate.

The period of validity of the certificate (Validity): the period of validity, or duration of use, of the certificate. After that date, the certificate becomes invalid.

The public key of the certificate owner (Public-Key): the public key that the certificate owner wants to publish.

The name of the certificate owner (Subject): to whom the certificate is issued, or the owner of the certificate, usually the name of a person or company, the name of the institution, the URL of the company's website, etc.

Certificate signature algorithm (Signature algorithm): the encryption algorithm used for the digital signature of this digital certificate, so that the fingerprint can be decrypted according to this algorithm using the public key in the certificate of the certificate authority.

Certificate issuer's digital signature (Thumbprint): the hash value (fingerprint) of a digital certificate, which is used to ensure the integrity of the digital certificate and to ensure that the certificate has not been modified.

The principle of the digital certificate is that when the certificate is issued, the CA institution will calculate the hash value (fingerprint) of the entire certificate according to the signature algorithm (Signature algorithm) and put it together with the certificate. When the user opens the certificate, he / she will also calculate the hash value (fingerprint) of the certificate according to the signature algorithm. If it matches the fingerprint recorded in the certificate, it means that the certificate has not been modified.

It can be seen that the digital certificate itself also uses digital signature technology, but the content of the signature is the entire certificate (which contains the public key of the certificate owner and other contents). Different from the ordinary digital signature, the signer of the digital certificate is not a casual ordinary institution, but a CA institution.

To sum up, the general process of digital signature and signature verification is as follows:

Android packaging process

The packaging process of the entire Android is shown in the following figure:

Compile and package steps:

1. Package the resource file and generate the R.java file. The tool for packaging the resource is aapt (The Android Asset Packaing Tool) (E:\ Documents\ Android\ sdk\ build-tools\ 25.0.0\ aapt.exe).

During this process, both the AndroidManifest.xml file and the layout file XML in the project will be compiled, and the corresponding R.java will be generated, and the AndroidManifest.xml will be compiled into binary by aapt.

Resources stored in APP's res directory are mostly compiled into binary files before APP is packaged, and each such file is assigned a resource id. For the access of this kind of resources, the application layer code is accessed through resource id. During the compilation process of Android application, the aapt tool will compile the resource file and generate a resource.arsc file. The resource.arsc file is equivalent to a file index table, recording a lot of information related to the resource.

two。 The tool used in the process of processing aidl files and generating the corresponding Java files is aidl (Android Interface Definition Language), that is, the Android interface description language (E:\ Documents\ Android\ sdk\ build-tools\ 25.0.0\ aidl.exe).

The aidl tool parses the interface definition file and then generates the corresponding Java code interface for the program to call. If the aidl file is not used in the project, you can skip this step.

3. Compiling the project source code and generating all the Java code in the class file, including R.java and .aidl files, will change the Java compiler (javac) to a .class file, and the resulting class file is located in the bin/classes directory in the project.

4. Convert all class files to generate classes.dex files the dx tool generates a classes.dex file that can be executed by the Android system Dalvik virtual machine, which is located at (E:\ Documents\ Android\ sdk\ build-tools\ 25.0.0\ dx.bat).

Any third-party libraries and .class files are converted to .dex files. The main work of dx tools is to convert Java bytecodes into Dalvik bytecodes, compress constant pools, eliminate redundant information, and so on.

5. Package and generate APK files all resources that have not been compiled, such as resources under the images and assets directories (these files are original files, which are not compiled when APP is packaged, but are directly packaged into APP. For the access of such resource files, the application layer code needs to access them through the file name); the compiled resources and .dex files will be packaged into the final .apk file by apkbuilder tools.

The packaged tool apkbuilder is located in the android-sdk/tools directory. Apkbuilder is a script file that actually calls the com.android.sdklib.build.ApkbuilderMain class in the (E:\ Documents\ Android\ sdk\ tools\ lib) file.

6. Sign the APK file once the APK file is generated, it must be signed before it can be installed on the device.

In the development process, the main use is two kinds of signed keystore. One is used for debugging debug.keystore, it is mainly used for debugging, in Eclipse or Android Studio directly run after running on the phone is the use of debug.keystore.

The other is keystore which is used to release the official version.

7. Align signed APK files if the apk you released is an official version, you must align the APK using zipalign (E:\ Documents\ Android\ sdk\ build-tools\ 25.0.0\ zipalign.exe)

The main process of alignment is to offset all resource files in the APK package by an integer multiple of 4 bytes from the start of the file, which makes it faster to access apk files through memory mapping. The purpose of alignment is to reduce runtime memory usage.

As you can see from the figure above, the signature occurs in the penultimate step of the packaging process, and the signature is aimed at the existing apk package, which does not affect the code we write. In fact, this is indeed the case. The general principle of Android signature is to calculate the hash of all the files in the unsigned apk, then save them (MANIFEST.MF), then calculate the hash of these hash (CERT.SF), then calculate the hash, and then encrypt and save (CERT.RSA) through the private key in the keystore generated above.

Android signature scheme

From its birth to the current version 1.0, Android system has experienced three generations of application signature schemes, namely, v1, v2 and v3 schemes.

V1 scheme: based on JAR signature. V2 scheme: APK signature scheme v2, introduced in Android 7. 0. V3 scheme: APK signature scheme v3, introduced in Android 9.0.

Among them, v1 to v2 is subversive, mainly to solve the security problem of JAR signature scheme, but when it comes to v3 scheme, there is not much adjustment in structure, which can be understood as the upgraded version of v2 signature scheme.

The upgrade from V1 to v2 has the greatest impact on developers, which is the issue of channel signing. The signature of v2 is also to make the installation packages of different channels and markets different, carrying the unique logo of the channel, that is, what we commonly call the channel package. Fortunately, all the major companies have opened up their own signature channel solutions, such as Walle (Meituan) and VasDolly (Tencent).

V1 signature

Signature tool

There are two signature tools for Android applications: jarsigner and apksigner. Their signature algorithms are no different, mainly because the files used for signature are different. The differences are as follows:

Jarsigner:jdk comes with a signature tool that allows you to sign jar. Use a keystore file for signing. The generated signature file is named using the alias of keystore by default.

A signature tool provided by apksigner:Android sdk specifically for Android applications. Use pk8 and x509.pem files for signing. Where pk8 is the private key file and x509.pem is the file containing the public key. The generated signature files are uniformly named with "CERT".

Since both tools are signed for APK, is there any connection between the keystore file and pk8,x509.pem? The answer is yes, they can be transformed, here is no longer to analyze how they are transformed, there are many examples on the Internet.

There is another knowledge point to pay attention to. If we look at the contents of a keystore file, we will find that it contains a summary of MD5 and SHA1, which is the data summary of the private key in the keystore file. This information is also the information we need to fill in when applying for many development platform accounts.

Signature process

First of all, we randomly select a signed APK (Sample-release.APK) to decompress, and we will get the file shown in the following figure.

You can see that there are three files under the META-INF folder: MANIFEST.MF, CERT.SF, and CERT.RSA. They are the files generated during the signing process, and they serve the following purposes.

MANIFEST.MF

What is saved in this file is to traverse all the entries in APK one by one, if it is a directory, skip it, if it is a file, use SHA1 (or SHA256) message digest algorithm to extract the summary of the file, then encode it with BASE64, and write it to a block in the MANIFEST.MF file as the value of the "SHA1-Digest" attribute. The block has a "Name" attribute whose value is the path of the file in the APK package.

Open the MANIFEST.MF file, and the file content is as follows:

CERT.SF

Open the CERT.SF file, the contents of which are as follows:

SHA1-Digest-Manifest-Main-Attributes: SHA1 (or SHA256) the blocks in the MANIFEST.MF header and then encode them with Base64. SHA1-Digest-Manifest: SHA1 (or SHA256) the entire MANIFEST.MF file and then encode it with Base64. SHA1-Digest: SHA1 (or SHA256) each entry of MANIFEST.MF and then encode it with Base64.

CERT.RSA

The signature of the previously generated CERT.SF file is calculated with the private key, and then the signature and the digital certificate containing the public key information are written and saved in CERT.RSA. It should be noted that the CERT.RSA certificate in Android APK is self-signed and does not require a certificate issued or certified by a third-party authority. Users can generate this self-signed certificate on the local machine. Android currently does not authenticate the application certificate with CA.

Open the CERT.RSA file, as shown below:

What we see here are all binaries, and because they are encrypted with RSA, we need to use the openssl command to see their contents, as shown below.

$openssl pkcs7-inform DER-in / / Sample-release_new/original/META-INF/CERT.RSA-text-noout-print_certs

After executing the above command, the result is as follows:

To sum up, a complete signature process is shown in the following figure:

Signature verification

Signature verification occurs during the installation of APK, which is divided into three steps:

Check all files contained in APK, and the corresponding summary values are the same as those recorded in the MANIFEST.MF file. Use the certificate file (RSA file) to verify that the signature file (SF file) has not been modified. Use the signature file (SF file) to verify that the MF file has not been modified.

To sum up, a complete signature verification process is as follows:

V2 signature

APK signature scheme v2 is a full-file signature scheme that discovers all changes made to protected parts of APK, helping to speed up verification and enhance integrity guarantees. Through the previous analysis, we can find that the v1 signature can be improved in two aspects:

In the process of slow signature verification, all files in apk need to be digested. Signature verification on machines with many APK resources and poor performance will take a long time, resulting in slow installation.

Integrity protection is not enough for the META-INF directory to store signatures. Naturally, this directory itself is not included in the signature verification process. You can add files to this directory at will. For example, some fast batch packaging solutions choose to add channel files to this directory.

In order to solve these two problems, a new APK Signature Scheme v2 is introduced in Android version 7. 0.

Improvement of V2

Because the v1 validates only a single ZIP entry, many changes can be made after the APK is signed-the file can be moved or even recompressed. In fact, this is what the ZIPalign tool used during compilation does, which is used to adjust ZIP entries to the correct byte limits to improve runtime performance. And we can also use this thing to modify the contents of the META-INF directory after packaging, or modify the comments of ZIP to achieve multi-channel packaging, which can be verified in the v1 signature.

V2 signing verifies all bytes in the archive, not a single ZIP entry, so ZIPalign can no longer be run after signing (must be done before signing). Because of this, Google now merges compression, adjustment, and signing into one step during compilation.

V2 signature

V2 signature will add a new block (signature block) to the original APK block, which stores information such as signature, summary, signature algorithm, certificate chain and additional attributes. This block has a specific format. The final signature APK actually has four pieces: the header file area, the V2 signature block, the central directory, and the tail. The following figure shows the composition of the V1 signature and the V2 signature.

The format of the entire signature block is as follows:

Size of block, in bytes (excluding this field) (uint64) "ID- value" pair sequence with uint64 length prefix: size of block, in bytes-same as the first field (uint64) magic "APK signature Block 42" (16 bytes)

In multiple "ID- value" pairs, the ID of APK signature information is 0x7109871a and contains the following contents: signer with length prefix:

Signed data with length prefix, including digests sequence, X.509 certificates sequence, additional attributes sequence with length prefix signatures (with length prefix) sequence public key (SubjectPublicKeyInfo,ASN.1 DER form) value with length prefix may contain multiple signer, because Android allows multiple signatures.

To sum up: the signature information of a signature block that can contain multiple ID-VALUE,APK will be stored in the key-value pair whose ID is 0x7109871a. Its content can contain the signature information of multiple signers, and each signature information includes signed data, signatures and public key, in which signed data mainly stores summary sequence, certificate chain and additional attributes, signatures contains signature values calculated by multiple signature algorithms, and public key represents the signer's public key, which is used to verify the signature.

Signature process

First of all, let's talk about the APK summary calculation rules. For each summary algorithm, the results are as follows:

The contents of the ZIP entry of the file in APK, the central directory of ZIP, and the end of the central directory of ZIP are divided into small blocks according to the 1MB size. Calculate the data summary for each small block, the data content is 0xa5 + block byte length + block content. Calculate the overall data summary, the data content is 0x5a + the number of blocks + the summary content of each block

In a word, the APK is divided according to 1m size, and the summary of these segments is calculated respectively. Finally, the summary of these segments is calculated to get the final summary, that is, the summary of APK. Then write the APK abstract + digital certificate + other attribute generation signature data to the APK Signing Block block.

Signature verification process

Next, let's take a look at the verification process of v2 signature. The overall process is shown in the following figure.

Among them, the v2 signature mechanism is only supported in Android 7.0 and above. Therefore, for Android 7.0 and above, if you find a v2 signature block during the installation process, you must follow the v2 signature mechanism and cannot be bypassed. Otherwise, downgrade the v1 signature mechanism. The v1 and v2 signature mechanisms can exist at the same time, and for both v1 and v2 versions, there is a X-Android-APK-Signed attribute in the .SF file attribute of the v1 version of META_INF.

X-Android-APK-Signed: 2

The previous channel package generation scheme is to add an empty file under the META-INF directory and use the name of the empty file as the unique identification of the channel. However, under the new application signature scheme, META-INF has been listed as a protected area, and the scheme of adding empty files to META-INF will affect blocks 1, 3 and 4. For this problem, you can refer to Meituan multi-channel packaging summary.

V3 signature

The new version of v3 signature is still checked by checking the entire compressed package on the basis of v2. The difference is that you can add a new certificate (Attr block) in the signature section. In this new block, our previous signature information and new signature information will be recorded, and the scheme of key wheel will be used to replace and upgrade the signature. This means that as long as the old signing certificate is in hand, we can use it to change the signature in the new APK file.

The new block (attr) added to the v3 signature stores all the signature information, which is stored in the form of a linked list by smaller Level blocks.

Each of these nodes contains a signing certificate used to sign the previous version of the application, and the oldest signing certificate corresponds to the root node, and the system asks the certificate in each node to sign the next certificate in the list, thus providing evidence for each new key to prove that it should be as trusted as the old key.

Signature verification process

Android's signature scheme, no matter how much it is upgraded, is to ensure backward compatibility. Therefore, after introducing the v3 scheme, Android 9.0 and later can try to verify APK according to the APK signature scheme, v3-> v2-> v1. Older platforms would ignore the v3 signature and try the v2 signature before verifying the v1 signature. The whole verification process is shown in the following figure:

It is important to note that in the case of overriding installation, signature verification only supports upgrade, not degradation. In other words, a v1-signed APK is installed on the device, which can be overridden with a v2-signed APK, otherwise it is not allowed.

This is the end of the content of "what is the Android signature mechanism". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.