In this multi-part blog series, I will attempt to explore how Android turn a line of code into machine code on your device and subsequently the pixels rendering your UI (User Interface) on an Android device.
If you’re familiar to Android Studio, you probably used the “Run app” function without thinking twice. Ever wondered how it actually runs it?
There are a few major steps involved in the process, in which I’ll break down into the following parts:
- Android build process: From Code to APK
- APK to running app
- Running code to rendering
Without further ado, lets get started with part 1.
Part 1 — Android Build Process: From Code to APK
In part 1 of the series, we’ll look at how high level code and resources gets turned into compact, distributable files that Android devices can understand and install; APKs.
What are APKs?
Now, before we understand the build process, we need to know what APK is in the first place. APK stands for Android PacKage and to put it simply, its an archive file containing compiled codes, resource-related files and a special file called AndroidManifest.xml We’ll go through each of these soon.
You can easily inspect APKs using any archiving tool such as winrar or Android-specific tool like apkanalyzer (Android Studio has it built-in under the “Build -> Analyze APK...” menu)
From code to APK
Now we can begin exploring what exactly does “building” an Android app entail.
Keep in mind this is for demonstration purposes and I do not recommend doing this for production codes
For starters, let’s have a new project with just one class MainActivity.kt which contain a text view with will render the words “Hello World” on screen. You can grab the sample project here or just create a new project in Android Studio.
To keep things simple, I’ve removed unnecessary dependencies such as the material library & constrain layout.
To produce an installable APK without the help of Android Studio’s built-in gradle, we turn to Android’s build tools for the job.
Without gradle — the manual build process
Android provides several command line tools that gradle also uses to convert an Android project into a installable APK. The complete set of tools can be found here.
To give you an overview, the main steps involved in the Android build process are detailed in this chart.
Don’t fret, we’ll go through each of the steps in the following sections.
Resource compilation
To start off from the top left of the overview chart, we need to convert the app’s, along with any dependencies/libraries’s resources into a format that can be packaged into an APK. In our project’s case, the dependencies include appcompat & androidx core libray in order to use Android specific resources such as themes & colors.
To do that, we’re using one of Android’s build tool called aapt2. There are two steps involved, compiling resources into .flat files and linking the .flat files to generate an APK. Aapt2 does it in this way to enable incremental compilation, allowing the build system to only compile any changed xml and reuse previous pre-compiled .flat files in the link stage on subsequent builds.
Aapt2 — compile stage
During the compile stage, aapt2 compiles all Android resources into a binary, intermediately format called .flat The file name is prefixed with the asset’s containing folder and separated by _ For example, activity_main.xml in the layout folder will be named layout_activity_main.xml.flat
To compile the app resources using aapt2, we run following compile command:
aapt2 compile --dir app\src\main\res -o output\res.zip
The --dir param specifies the resource path and -o specifies the output file which can be a zipped file.
Additional resources are needed as the project reference resources in appcompat & androidx core libraries. The aapt2 link command used later on will result in error if any required resources are missing. To compile those, we run the same command on the libraries’ res folder.
aapt2 compile --dir appcompat-1.2.0\res -o output\res-appcompat.zipaapt2 compile --dir core-1.3.2\res -o output\res-core.zip
Resources that hold values (in values folder) however, such as colors, strings, etc. will generate files ending with
.arsc.flatinstead. We’ll discuss more on this in the next stage.
Aapt2 — link stage
In the link stage, aapt2 links all the .flat files generated in the compile stage and packages them into an APK. AndroidManifest.xml, Android platform .jar, as well as any raw resources that can’t be compiled such as video or sound assets, are also packaged in to the APK. aapt2 does this by parsing and indexing all resource references such as theme references in AndroidManifest.xml or Android platform drawables used in layout xmls to their actual values or resource path.
android.jar contains all the Android framework classes and are platform specific; that is to say android.jar in sdk30(Android 11) and android.jar in sdk29(Android 10) are different, and we usually choose the platform we’re targeting.
*Also, android.jar in the sdk platform folder is actually just a mock that have placeholder methods so the code can actually compile. The actual framework codes & resources resides on the device itself.
To link resources, we run the aapt2 link command with all the compiled .flat files (the tool accepts archive containing the .flat files as well), platform’s android.jar and the Android manifest file.
aapt2 link output\res.zip output\res-appcompat.zip output\res-core.zip -I android.jar --manifest app\src\main\AndroidManifest.xml -o samplebuild.apk --java output
The--java output tells aapt2 to generate the R.java file which is used in runtime code to locate any required resource. The R.java file must be included in the project codebase and we’ll discuss how it is used to link resources later on.
If all goes well, there should be a new file — in our case the samplebuild.apk, as defined by the -o param.
Do note that this is not the final APK yet and it will not work on an Android device. We’ll reuse this apk in later steps.
Wondering what that resources.arsc file is? Let’s find out.
Resources.arsc
As we saw previously, resources.arsc is generated from aapt2’s link command. The file indexes and, together with the previous arsc.flat files, form a lookup table of sorts for resources. It contains multiple tables, each for one resource type that contains data that either points to the resource path or the actual resource value itself as seen below with cases like strings or integer.
resources.arsc also contains different version of the resource such as other languages for strings, different Android versions for layout. All of these are linked to an integer ID value which is also set in the R.java file.
For example, we can observe the app_name row in above image has the ID “0x70c001b”. This can be cross referenced with the R.java file below under the same app_name variable.
R.java file
The R.java file exists in the codebase and acts like a lookup table linking accessible int values to the actual resource. The end result looks something like this (in pseudo code):
inflate(R.layout.activity_main) will reference R.java class’s layout inner class’s activity_main variable which is the integer value of “0x7f0a001c “
With resource.arsc we can find the file path which in this case is res/layout/activity_main.xml
That’s it for the resources! Next up the line we’ll need to compile the source codes as indicated on the top right of the overview chart.
Code compilation
First, lets compile the R.java file. Since this is a java file, we’ll need to use java compiler javac to execute the following command:javac -source 1.8 -target 1.8 -bootclasspath android.jar R.java
We target java version 1.8 (aka Java 8; don’t ask me why they versioned it like so) as that is the default java version as of typing this article. After executing the command, the java class is compiled into java bytecode which has the extension .class
Java bytecode is the instruction set or machine code for the Java Virtual Machine (JVM) which interprets the bytecode to run on different platforms and architectures. This allow for the same bytecode to work on different devices/cpu without extra effort for the developer.
We then place these files in a folder named deps which will be needing later.
To compile kotlinc files however, we use the kotlin complierkotlinc instead. Kotlin classes are also targeted for the JVM hence they compile to .class files as well. More info on kotlinc tool can be found here.
To compile MainActivity.kt, we execute the following command:
kotlinc -cp android.jar:deps MainActivity.kt
As MainActivity.kt contain reference to R.java and Android specific codes (e.g. Activity class), we need to include android.jar & the R classes in deps folder. The needed dependencies are specified with -cp
Dexer
The complied .class files are not the final bytecode format for Android though. To understand why, we look back onto the earlier days of Java & Android.
Java was not intended for use in portable devices. In fact, when it was designed in 1995, mobile computing is nothing but a dream. To account for the low memory (in the early days at least) and computing power of devices, Android needed to design its own runtime and bytecode.
The solution runtime is named Dalvik (up to Android 4.4), which was more efficient and worked better on mobile devices. The accompanying bytecode for Dalvik has the file format .dex (Dalvik EXecutable). Although the runtime has since been replaced with ART (Android RunTime), the bytecode format remain unchanged.
Dexer tool — D8
To convert .class files into the more optimised .dex files, we use the D8 dexer in Android build tools. Running d8 is fairly straight forward — we execute the following command:
./d8 *.class — output test --lib android.jar
The android.jar is needed in this case as android.app.Activity.java class contains a default or static interface methods which uses Java8 language feature. In short, the — lib flag is used to support this. More information on this can be found here.
If everything goes as planned, you should see the a classes.dex file in the output folder.
Dexer tool — R8
There is another option of using R8 instead of D8, however, for the purpose of this article we don’t need the additional features of R8 and we’ll be sticking to D8.
Packing it all together
Its finally time to package all these files into the APK. Let’s continue on!
Remember we have the sampleBuild.apk generated from aapt2 tool earlier? We’ll add the compiled source files into that. We can use any zip tool for this. For bash shell, we use the following command to add classes.dex into the APK:
zip -uj samplebuild.apk classes.dex
If all goes well, you should see the classes.dex in the APK now.
Aligning APK
Android build process requires the APK to be aligned before it can be installed. Aligning the APK provides additional memory usage optimisation to uncompressed files; specifically, it allow files to be accessed directly by the process without first copying it to memory. We use zipalign tool with the following command for this.
./zipalign -p -f -v 4 sampleBuild.apk aligned.apk
The aligned APK file can be verified by running this command:
./zipalign -c -v 4 aligned.apk
Signing the APK
For the final step, we’ll need to sign the APK so that Android system can identify the integrity of the file. For the purpose of this article, we will sign it using the debug key meant for development use.
To generate the key to sign with, we can use keytool built into the terminal and execute the following command:
keytool -genkey -v -keystore debug.keystore -storepass android -alias androiddebugkey -keypass android -keyalg RSA -keysize 2048 -validity 10000
Note: We can leave blanks for any prompted fields such as name or country.
The above command creates a key in a keystore named debug.keystore with the keystore password “android”.
To actually sign the aligned APK, we need another Android build tool named apksigner.
./apksigner sign --ks debug.keystore --ks-pass pass:android --out signed.apk aligned.apk
You should now see the signed APK named signed.apk
That concludes the last step to building an installable APK. To verify the APK is indeed working, let’s grab an Android device to view the fruits of our labor.
Running the app
Lastly (finally!), install the app on to any Android device and launch it. You should see “Hello world” text displayed.
And.. that’s it! We have successfully built an Android app from scratch. Of course, this is all for demonstration purposes and there are a lot more going on in the build process when building it via gradle.
Nevertheless, I hope you gained some insights on the internal process of building an app and had some fun :)
In the next part, we will look into how Android takes an APK and executes its code. Stay tuned!
Disclaimer: This article also serves as a personal research & learning exercise. Please leave comments if there are any inaccuracies or confusion & let’s improve together.
