Android code signing

We covered a new security feature introduced in the last Jelly Bean maintenance release in our last post and, before you know it, a new tag has already popped up in AOSP. Google I/O is just around the corner, and some interesting bits and pieces are trickling into the AOSP master branch, so it's probably time for a new post. There are plenty of places where you can get your rumour fix regarding I/O 2013 and it looks like build JDQ39E is going to be somewhat boring, so we will explore something different instead: code signing. This particular aspect of Android has remained virtually unchanged since the first public release, and is so central to the platform, that is pretty much taken for granted. While neither Java code signing, nor its Android implementation are particularly new, some of the finer details are not particularly well-known, so we'll try to shed some more light on those. The first post of the series will concentrate on the signature formats used while the next one will look into how code signing fits into Android's security model.

Java code signing

As we all know, Android applications are coded (mostly) in Java, and Android application package files (APKs) are just weird-looking JARs, so it pays to understand how JAR signing works first. 

First off, a few words about code signing in general. Why would anyone want to sign code? For the usual reasons: integrity and authenticity. Basically, before executing any third-party program you want to make sure that it hasn't been tampered with (integrity) and that it was actually created by the entity that it claims to come from (authenticity). Those features are usually implemented by some digital signature scheme, which guarantees that only the entity owning the signing key can produce a valid code signature. The signature verification process verifies both that the code has not been tampered with and that the signature was produced with the expected key. One problem that code signing doesn't solve directly is whether the code signer (software publisher) can be trusted. The usual way trust is handled is by requiring the code signer to hold a digital certificate, which they attach to the signed code. Verifiers decide whether to trust the certificate either based on some trust model (e.g., PKI or web of trust), or on a case-by-case basis. Another problem that code signing does not solve (or event attempt to) is whether the signed code is safe to run. As we have seen, code that has been signed (or appears to be) by a trusted third party is not necessarily safe (e.g., Flame or pwdump7).

Java's native code packaging format is the JAR file, which is essentially a ZIP file bundling together code (.class files or classes.dex in Android), some metadata about the package (.MF manifest files in the META-INF/ directory) and, optionally, resources the code uses. The main manifest file (MANIFEST.MF) has entries with the file name and digest value of each file in the archive. The start of the manifest file of a typical APK file is show below (we'll use APKs instead of actual JARs for all examples).

Manifest-Version: 1.0
Created-By: 1.0 (Android)

Name: res/drawable-xhdpi/ic_launcher.png
SHA1-Digest: K/0Rd/lt0qSlgDD/9DY7aCNlBvU=

Name: res/menu/main.xml
SHA1-Digest: kG8WDil9ur0f+F2AxgcSSKDhjn0=

Name: ...

Java code signing is implemented at the JAR file level by adding another manifest file, called a signature file (.SF) which contains the data to be signed, and a digital signature over it (called a 'signature block file', .RSA, .DSA or .EC). The signature file is very similar to the manifest, and contains the digest of the whole manifest file (SHA1-Digest-Manifest), as well as digests for each of the individual entries in MANIFEST.MF.

Signature-Version: 1.0
SHA1-Digest-Manifest-Main-Attributes: ZKXxNW/3Rg7JA1r0+RlbJIP6IMA=
Created-By: 1.6.0_45 (Sun Microsystems Inc.)
SHA1-Digest-Manifest: zb0XjEhVBxE0z2ZC+B4OW25WBxo=

Name: res/drawable-xhdpi/ic_launcher.png
SHA1-Digest: jTeE2Y5L3uBdQ2g40PB2n72L3dE=

Name: res/menu/main.xml
SHA1-Digest: kSQDLtTE07cLhTH/cY54UjbbNBo=

Name: ...

The digests in the signature file can easily be verified by using the following OpenSSL commands:

$ openssl sha1 -binary MANIFEST.MF |openssl base64
zb0XjEhVBxE0z2ZC+B4OW25WBxo=
$ echo -en "Name: res/drawable-xhdpi/ic_launcher.png\r\nSHA1-Digest: \
K/0Rd/lt0qSlgDD/9DY7aCNlBvU=\r\n\r\n"|openssl sha1 -binary |openssl base64
jTeE2Y5L3uBdQ2g40PB2n72L3dE=

The first one takes the SHA1 digest of the entire manifest file and encodes it to Base 64 to produce the SHA1-Digest-Manifest value, and the second one simulates how the digest of a single manifest entry is being calculated. The actual digital signature is in binary PKCS#7 (or more generally, CMS) format and includes the signature value and signing certificate. Signature block files produced using the RSA algorithm are saved with the extension .RSA, those generated with DSA or EC keys with the .DSA or .EC extensions, respectively. Multiple signatures can be performed, resulting in multiple .SF and .RSA/DSA/EC files in the JAR file's META-INF/ directory. The CMS format is rather involved, allowing not only for signing, but for encryption as well, both with different algorithms and parameters, and is extensible via custom signed or unsigned attributes. A thorough discussion is beyond the scope of this post, but as used for JAR signing it basically contains the digest algorithm, signing certificate and signature value. Optionally the signed data can be included in the SignedData CMS structure (attached signature), but JAR signatures don't include it (detached signature). Here's how an RSA signature block file looks like when parsed into ASN.1 (certificate info trimmed):

$ openssl asn1parse -i -inform DER -in CERT.RSA
    0:d=0  hl=4 l= 888 cons: SEQUENCE          
    4:d=1  hl=2 l=   9 prim:  OBJECT            :pkcs7-signedData
   15:d=1  hl=4 l= 873 cons:  cont [ 0 ]        
   19:d=2  hl=4 l= 869 cons:   SEQUENCE          
   23:d=3  hl=2 l=   1 prim:    INTEGER           :01
   26:d=3  hl=2 l=  11 cons:    SET               
   28:d=4  hl=2 l=   9 cons:     SEQUENCE          
   30:d=5  hl=2 l=   5 prim:      OBJECT            :sha1
   37:d=5  hl=2 l=   0 prim:      NULL              
   39:d=3  hl=2 l=  11 cons:    SEQUENCE          
   41:d=4  hl=2 l=   9 prim:     OBJECT            :pkcs7-data
   52:d=3  hl=4 l= 607 cons:    cont [ 0 ]        
   56:d=4  hl=4 l= 603 cons:     SEQUENCE          
   60:d=5  hl=4 l= 452 cons:      SEQUENCE          
   64:d=6  hl=2 l=   3 cons:       cont [ 0 ]        
   66:d=7  hl=2 l=   1 prim:        INTEGER           :02
   69:d=6  hl=2 l=   1 prim:       INTEGER           :04
   72:d=6  hl=2 l=  13 cons:       SEQUENCE          
   74:d=7  hl=2 l=   9 prim:        OBJECT            :sha1WithRSAEncryption
   85:d=7  hl=2 l=   0 prim:        NULL              
   87:d=6  hl=2 l=  56 cons:       SEQUENCE          
   89:d=7  hl=2 l=  11 cons:        SET               
   91:d=8  hl=2 l=   9 cons:         SEQUENCE          
   93:d=9  hl=2 l=   3 prim:          OBJECT            :countryName
   98:d=9  hl=2 l=   2 prim:          PRINTABLESTRING   :JP
...
  735:d=5  hl=2 l=   9 cons:      SEQUENCE          
  737:d=6  hl=2 l=   5 prim:       OBJECT            :sha1
  744:d=6  hl=2 l=   0 prim:       NULL              
  746:d=5  hl=2 l=  13 cons:      SEQUENCE          
  748:d=6  hl=2 l=   9 prim:       OBJECT            :rsaEncryption
  759:d=6  hl=2 l=   0 prim:       NULL              
  761:d=5  hl=3 l= 128 prim:      OCTET STRING      [HEX DUMP]:892744D30DCEDF74933007...

If we extract the contents of a JAR file, we can use the OpenSSL smime (CMS is the basis of S/MIME) command to verify its signature by specifying the signature file as the content (signed data). It will print the signed data and the verification result:

$ openssl smime -verify -in CERT.RSA -inform DER -content CERT.SF signing-cert.pem
Signature-Version: 1.0
SHA1-Digest-Manifest-Main-Attributes: ZKXxNW/3Rg7JA1r0+RlbJIP6IMA=
Created-By: 1.6.0_43 (Sun Microsystems Inc.)
SHA1-Digest-Manifest: zb0XjEhVBxE0z2ZC+B4OW25WBxo=

Name: res/drawable-xhdpi/ic_launcher.png
SHA1-Digest: jTeE2Y5L3uBdQ2g40PB2n72L3dE=

...
Verification successful

The official tools for JAR signing and verification are the jarsigner and keytool commands from the JDK. Since Java 5.0 jarsigner also supports timestamping the signature by a TSA, which could be quite useful when you need to ascertain the time of signing (e.g., before or after the signing certificate expired), but this feature is not widely used. Using the jarsigner command, a JAR file is signed by specifying a keystore file, the alias of the key to use for signing (used as the base name for the signature block file) and, optionally, a signature algorithm. One thing to note is that since Java 7, the default algorithm has changed to SHA256withRSA, so you need to explicitly specify it if you want to use SHA1. Verification is performed in a similar fashion, but the keystore file is used to search for trusted certificates, if specified. (again using an APK file instead of an actual JAR):

$ jarsigner -keystore debug.keystore -sigalg SHA1withRSA test.apk androiddebugkey
$ jarsigner -keystore debug.keystore -verify -verbose -certs test.apk
....

smk      965 Mon Apr 08 23:55:34 JST 2013 res/drawable-xxhdpi/ic_launcher.png

      X.509, CN=Android Debug, O=Android, C=US (androiddebugkey)
      [certificate is valid from 6/18/11 7:31 PM to 6/10/41 7:31 PM]

smk   458072 Tue Apr 09 01:16:18 JST 2013 classes.dex

      X.509, CN=Android Debug, O=Android, C=US (androiddebugkey)
      [certificate is valid from 6/18/11 7:31 PM to 6/10/41 7:31 PM]

         903 Tue Apr 09 01:16:18 JST 2013 META-INF/MANIFEST.MF
         956 Tue Apr 09 01:16:18 JST 2013 META-INF/CERT.SF
         776 Tue Apr 09 01:16:18 JST 2013 META-INF/CERT.RSA

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore
  i = at least one certificate was found in identity scope

jar verified.

The last command verifies the signature block and signing certificate, ensuring that the signature file has not been tampered with. It then verifies that each digest in the signature file (CERT.SF) matches its corresponding section in the manifest file (MANIFEST.MF). One thing to note is that the number of entries in the signature file does not necessarily have to match those in the manifest file. Files can be added to a signed JAR without invalidating its signature: as long as none of the original files have been changed, verification succeeds. Finally, jarsigner reads each manifest entry and checks that the file digest matches the actual file contents. Optionally, it checks whether the signing certificate is present in the specified key store (if any). As of Java 7 there is a new -strict option that will perform additional certificate validations. Validation errors are treated as warnings and reflected in the exit code of the jarsigner command. As you can see, it prints certificate details for each entry, even though they are the same for all entries. A slightly better way to view signer info when using Java 7 is to specify the -verbose:summary or -verbose:grouped, or alternatively use the keytool command:

$ keytool -list -printcert -jarfile test.apk
Signer #1:

Signature:

Owner: CN=Android Debug, O=Android, C=US
Issuer: CN=Android Debug, O=Android, C=US
Serial number: 4dfc7e9a
Valid from: Sat Jun 18 19:31:54 JST 2011 until: Mon Jun 10 19:31:54 JST 2041
Certificate fingerprints:
         MD5:  E8:93:6E:43:99:61:C8:37:E1:30:36:14:CF:71:C2:32
         SHA1: 08:53:74:41:50:26:07:E7:8F:A5:5F:56:4B:11:62:52:06:54:83:BE
         Signature algorithm name: SHA1withRSA
         Version: 3

Once you know the signature block file name (by listing the archive contents, for example), you can also use OpenSSL in combination with the zip command to easily extract the signing certificate to a file:

$ unzip -q -c test.apk META-INF/CERT.RSA|openssl pkcs7 -inform DER -print_certs -out cert.pem

Android code signing

As evident from the examples above, Android code signing is based on Java JAR signing and you can use the regular JDK tools to sign or verify APKs. Besides those, there is an Android specific tool in the AOSP build/ directory, aptly named signapk. It performs pretty much the same task as jarsigner in signing mode, but there are also a few notable differences. To start with, while jarsigner requires keys to be stored in a compatible key store file, signapk takes separate signing key (in PKCS#8 format) and certificate (in DER format) files as input. While it does appear to have some support for reading DSA keys, it can only produce signatures with the SHA1withRSA mechanism. Raw private keys in PKCS#8 are somewhat hard to come by, but you can easily generate a test key pair and a self-signed certificate using the make_key found in development/tools. If you have existing OpenSSL keys you cannot use them as is however, you will need to convert them using OpenSSL's pkcs8 command:

echo "keypwd"|openssl pkcs8 -in mykey.pem -topk8 -outform DER -out mykey.pk8 -passout stdin

Once you have the needed keys, you can sign an APK like this:

$ java -jar signapk.jar cert.cer key.pk8 test.apk test-signed.apk

Nothing new so far, except the somewhat exotic (but easily parsable by JCE classes) key format. However, the signapk has an extra 'sign whole file' mode, enabled with the -w option. When in this mode, in addition to signing each individual JAR entry, the tool generates a signature over the whole archive as well. This mode is not supported by jarsigner and is specific to Android. So why sign the whole archive when each of the individual files is already signed? In order to support over the air updates (OTA), naturally :). If you have ever flashed a custom ROM, or been impatient and updated your device manually before it picked up the official update broadcast, you know that OTA packages are ZIP files containing the updated files and scripts to apply them. It turns out, however, that they a lot more like JAR files on the inside. They come with a META-INF/ directory, manifests and a signature block, plus a few other extras. One of those is the /META-INF/com/android/otacert file, which contains the update signing certificate (in PEM format). Before booting into recovery to actually apply the update, Android will verify the package signature, then check that the signing certificate is one that is trusted to sign updates. OTA trusted certificates are completely separate from the 'regular' system trust store, and reside in a, you guessed it, a ZIP file, usually stored as /system/etc/security/otacerts.zip. On a production device it will typically contain a single file, likely named releasekey.x509.pem.

Going back to the original question, if OTA files are JAR files, and JAR files don't support whole-file signatures, where does the signature go? The Android signapk tool slightly abuses the ZIP format by adding a null-terminated string comment in the ZIP comment section, followed by the binary signature block and a 6-byte final record, containing the signature offset and the size of the entire comment section. This makes it easy to verify the package by first reading and verifying the signature block from the end of the file, and only reading the rest of the file (which for a major upgrade might be in the hundreds of MBs) if the signature checks out. If you want to manually verify the package signature with OpenSSL, you can separate the signed data and the signature block with a script like the one below, where the second argument is the signature block file, and the third one is the signed ZIP file (without the comments section) to write:

#!/bin/env python

import os
import sys
import struct

file_name = sys.argv[1]
file_size = os.stat(file_name).st_size

f = open(file_name, 'rb')
f.seek(file_size - 6)
footer = f.read(6)

sig_offset = struct.unpack('<H', footer[0:2])
sig_start = file_size - sig_offset[0]
sig_size = sig_offset[0] - 6
f.seek(sig_start)
sig = f.read(sig_size)

f.seek(0)
# 2 bytes comment length + 18 bytes string comment
sd = f.read(file_size - sig_offset[0] - 2 - 18)
f.close()

sf = open(sys.argv[2], 'wb')
sf.write(sig)
sf.close()

zf = open(sys.argv[3], 'wb')
zf.write(sd)
zf.close()

Summary

Android relies heavily on the Java JAR format, both for application packages (APKs) and for system updates (OTA packages). APK signing uses a subset of the JAR signing specification as is, while OTA packages use a custom format that generates a signature over the whole file. Standalone package verification can be performed with standard JDK tools or OpenSSL (after some preprocessing). The Android OS and recovery system follow the same verification procedures before installing APKs or applying system updates. In the next article we will explore how the OS uses package signatures and how they fit into Android's security model. 

Comments

roman said…
Hi Nikolay,

I’m trying to sign an Android apk file directly on the server with PHP and your article was a great source of information.

 Thanks to your article I was able to create the entries in .MF and .SF files. However, I don’t know anything about cryptography and therefore I’m struggling with the RSA file.

 I know that I could produce public/private keys (with ssh) and then I could create a self-signed certificate using the keys with phpseclib. Additionally, there is also a function in openssl for php that can sign data(openssl_pkcs7_sign() ) after providing a signing certificate and private key...but I just don’t know how to convert the result to the .RSA file(or something near it).

 I also checked the source code of jarsigner, but I don’t know much about java and it didn’t help.

 Can this file be even created (without too much effort) with openssl? Currently I’m stuck and would appreciate any help.

 Thank you again for the great article :)

R
Unknown said…
You should generate your keys directly with openssl, ssh might use a different format. I haven't use PHP for signing, but openssl_pkcs7_sign() should produce a PKCS#7 signature which is what the signature block file is (.RSA). Assuming you generated the signature file (.SF) correctly, you just need to save the output of openssl_pkcs7_sign() as a binary file to produce the .RSA file. If the openssl smime and asn1parse commands can parse it, it is probably OK.
roman said…
When I was trying to create a new CERT.RSA file, I actually used a properly signed apk file from which I only removed the CERT.RSA file. The signature file(.SF) remained the same (properly generated by jarsigner).

I also tried to create the keys and a self-signed certificate completely with openssl as you suggested. But jarsigner is still saying that the file is unsigned/signature is not parsable.

It is possible that I didn't convert the file produced by opennssl_pkcs7_sign() properly. This file contains also some regular text in the header, lines end with LF "\n", and the actual signature(and other stuff) is base64 encoded. So I removed the text and "\n"s and base64 decoded the remaining string. Asn1parse is able to parse it, but I'm not sure if that automatically means that I converted it correctly.

Another issue might be that I set the parameters for the keys/certificate/signing functions incorrectly. I have to admit that most of the time I wasn't sure what I'm actually doing.

I used the following PHP code:


$configs = array(
'config' => 'D:\test\openssl.cnf',
'digest_alg' => 'sha1',
'x509_extensions' => 'v3_ca',
'req_extensions' => 'v3_req',
'private_key_bits' => 1024,
'private_key_type' => OPENSSL_KEYTYPE_RSA,
'encrypt_key' => true,
'encrypt_key_cipher' => OPENSSL_CIPHER_3DES
);

//=== Generate a new private (and public) key pair
$privkey = openssl_pkey_new($configs);

//=== Create data array for certificate information
$dn = array(
"countryName" => "UK",
"stateOrProvinceName" => "Cambridge",
"localityName" => "Cambs",
"organizationName" => "UniServer",
"organizationalUnitName" => "Demo",
"commonName" => "localhost",
"emailAddress" => "me@example.com"
);

//=== Generate a certificate signing request
$csr = openssl_csr_new($dn, $privkey, $configs);

//== Create a self-signed certificate valid for 365 days
$sscert = openssl_csr_sign($csr, NULL, $privkey, 365, $configs);

// sign file with openssl_pkcs7_sign, detached signature, use the previously created self-signed certificate
if (openssl_pkcs7_sign ( 'D:\test\CERT.SF' , 'D:\test\CERT.CRT' , $sscert , $privkey, array(), PKCS7_BINARY ) ){
$resStr = "ok";
}

file_put_contents('D:\test\CERT.RSA', pkcs7PemToDer(file_get_contents('D:\test\CERT.CRT')),FILE_BINARY);
Unknown said…
I can't test your code, but it looks OK to me. As for converting from PEM (text based) to DER (binary), stripping the text header and footer and converting should be sufficient. If openssl asnparse is producing output similar to the one in the blog (look for the 'pkcs7-signedData' line), the output is probably OK. For the JAR format, compare file names and sizes to the original file to see what might be different.
roman said…
Thank you for your quick reply. I’ll try to compare the files in more detail when I’m back home…

It might be that I’m missing something important from the overall signing concept.
Here is again what I did to generate the CERT.RSA and test it with jarsigner:
- I created a simple test app in eclipse (test.apk), signed properly with debug key (jarsigner also verified that it’s properly signed).
- I renamed the test.app file to test.zip, so that it’s easier to work with in regular file explorer in windows (jarsigner still says: signed)
- I made a copy of CERT.RSA in a different folder outside of the test.zip file and deleted the CERT.RSA file inside of test.zip (jarsinger: “signature missing…” or something similar)
- I copied the previously backed up CERT.RSA file back to test.zip just to see what happens (jarsinger: signed)
- I removed CERT.RSA from test.zip again and generated new CERT.RSA with the previously shown PHP; I copied this new CERT.RSA file to test.app (jarsigner: “Signature missing…”)
 
All other files in test.zip remained the same (MANIFEST.MF, CERT.SF, and files outside of META-INF dir).
 
My understanding was that I don’t need to create a signature over the whole archive (like with signapk.jar –w) to generate a properly signed .apk file, which can be directly installed to your phone. And the final zip byte-alignment shouldn’t be required for .apk to properly install on your phone (I thought it’s only optimization).
 
When I checked the asnparse output, somewhere under the “pkcs7-signedData” section there also was the whole content of the CERT.SF file displayed (as regular text as you view it a regular text editor). This didn’t show up in the original CERT.RSA file. From my understanding, adding the signed content to the signature is “optional”, so jarsigner might still be able to parse it.
 
The original RSA file has also different certificate descriptive texts/attributes (“Owner: CN=Bob, OU=testing, O=signer….” Vs “localityName:Cabs…”), but I assumed this doesn’t matter.
 
Additionally, the validity time in original .RSA file seems to have a different type (TCTIME)…not sure if this matters.
Unknown said…
You don't have to generate a signature over the whole file, just over the SF file. The ASN.1 format might support attached data, but I don't think the JAR format allows it. So you should find the option/flag that lets you generate detached signatures. If you can't get it to work in PHP, try to generate the signature block file with OpenSSL commands and then replicate in PHP.
roman said…
You were absolutely right about the finding the right flag for detached signature.

I changed the flags in openssl_pkcs7_sign () to "PKCS7_DETACHED|PKCS7_BINARY|PKCS7_NOATTR" and now it seems to be working. At first it looked like only PKCS7_DETACHED is required (should be enabled by default, if no other flags are specified). However, jarsigner verified the file only after I added the last flag - PKCS7_NOATTR.

Thank you very much for your help and great article :)
roman said…
This comment has been removed by the author.
Rick said…
Excellent article.

I would like to insert files in an apk without invalidating the signature. I found that for some reason it only worked if the files I inserted were under META-INFO. Any clue why?
Unknown said…
Android's APK format (unlike JAR) requires that all files are signed with the same key/certificate. So that's not going to work. META-INF is not supposed to include signed files, and is ignored when validating. That's why it works.
Anonymous said…
Hey there
Great article, learned much about code signing, and now i know how to verify .jar´s and .apk´s :)

Now i want to verify the firmware.zip for my phone or the whole downloaded OTA.zip. But i don't know how..
I took formed the public-key with openssl from inside the OTA.zip in META-INF/com/android/otacert :

openssl x509 -noout -in otacert > pubkey.pem

Then i tried to verify with the signature block CERT.RSA in META-INF and openssl:

openssl dgst -verify pubkey.pem -signature CERT.RSA firmware.zip
openssl dgst -verify pubkey.pem -signature CERT.RSA OTA.zip

It returns me verification Error, i am not sure, if i'm doing right..
I also tried your python script on my OTA.zip and used the outputs and mixed them with untouched .RSA and OTA.zip, but this also seems to be wrong for verfication of the whole OTA.zip

would be very thankful for a little hint ;)

Cheers sodan
Unknown said…
OTA files are signed using CMS, so openssl dgst won't work. You need to use openssl smime, where the content is the unsigned OTA ZIP (extracted using the script above). Something like:

$ openssl smime -verify -in CERT.RSA -inform DER -content OTA.zip signing-cert.pem
Anonymous said…
Hello Nikolay, thanks for sharing the information. I would like to know that is it possible to update the apk on play store if we have lost the keystore. Any possible workaround or tips for that?
Unknown said…
Hi,

Do you know if OTA updates check the MANIFEST.MF and associated signature or does it only use the whole-file signature in the zip comment?

Great work, love your book.
Unknown said…
AFAIK, only the whole-file signature is checked, but have a look at the latest recovery code to confirm.
SOW1987 said…
Je lui tout les commentaires mes OpenSSL je veux connaitre OpenSSL signification svp

Popular posts from this blog

Decrypting Android M adopted storage

Unpacking Android backups

Using app encryption in Jelly Bean