Building Open Source libraries with Android NDK

Having scrambled through a NDK documentation and a lot of hit and trials and experimentation, finally I could figure out how could one build (even though partially) Open Source libraries with Android NDK.

Some background on how Autotools work:

The way autootool and friends work is this:

You write a configure.ac file, which is read in by Autoreconf and generates a configure script. Configure script test the build system for headers, libraries etc. and generates a config.h which contains definitions like #define HAVE_XXX 1 and also generates Makefiles after reading in the Makefile.in files which in turn is generated using Automake by reading in Makefile.am.

configure.ac —> input to –> autoreconf –> generates -> configure script –> checks host system for headers, libraries etc. -> generates -> config.h

Also

Makefile.am –> input to automake –> generates –> Makefile.in –> input to –> configure script –> generates Makefile

The code uses the generated config.h in the following fashion:

#ifdef HAVE_XXXX
#include <some-header>
#endif
….
….
#ifdef HAVE_XXXX
Call some function which is declared in the header
#else
Provide some other mechanism or report error to user or do whatever you want
#endif

The problem?

Most Open Source libraries use GNU autotools and its friends for building.

The first problem is that the Autools generate some configuration headers based on build time probe of the system. By build time probe I mean checking for things like if a header, library or a tool is present or not. In cross compiling scenario some of these probe should be done on the target system and not on the build system.

Second, the build system for most cross compiler tools have their own quirks which need passing some extra flags.

Third, in case of Android, it provides its own build system which are essentially some Makefile definitions and rules. This is provided so that people can build their code easily without having to deal with usual cross compiling issues. Thus there is a gap that while your autotools would generate the Makefiles while Android build system requires its own styled Makefiles in the form of Android.mk. One cannot simply generate Android.mk files using autotools.

Fourth, even if one gets to write Android specific Makefiles, the build would most probably fail as during the build it would look for a file config.h included in the fashion shown below, while no such file would exist as it is generated by the configure script.

#ifdef HAVE_CONFIG_H
#include “config.h”
#endif

Simply copying a config.h file from a run of configure script on another build system wouldn’t really work as the header files and other libraries present on Android may not match with the header and libraries present on the build system. Thus config.h would probably differ if it is somehow generated for Android with the one generated on a build system.

So how does one build an open source library for Android?

Solution:

The way I have managed to work around this trouble is to run the configure script with right cross compilation variables so that a config.h matching my Android system gets built and then writing Android.mk files which would simply use the Android build system.

The way I figured out the right flags was by building the Android source tree which displayed what flags are being used for building and then taking the cues from there, I passed the right flags to the configure script.

It looks something like this for building Android-3 target API on a linux host:

export ANDROID_ROOT=/home/divkis01/mydroid

The command above is sets the path where the Android sources are checked out from git respository.

NOTE: It is not necessary to check out the ANDROID sources and you can replace the ANDROID_ROOT with NDK_ROOT in all the commands below, along with proper path to the NDK cross compiler.

export PATH=$PATH:$ANDROID_ROOT/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/

The command above is necessary so that configure can find out the path to the cross compiler which is needed to build some test programs during the configure run process. Please note that you can also set the path to the NDK compiler root

./configure –host=arm-eabi CC=arm-eabi-gcc CPPFLAGS=”-I$ANDROID_ROOT/build/platforms/android-3/arch-arm/usr/include/” CFLAGS=”-nostdlib” LDFLAGS=”-Wl,-rpath-link=$ANDROID_ROOT/build/platforms/android-3/arch-arm/usr/lib/ -L$ANDROID_ROOT/build/platforms/android-3/arch-arm/usr/lib/” LIBS=”-lc “

The command above has several points that should be well understood.

–host=arm-eabi –> This tells the configure script if the cross compilation is being done or not. It is also used as a prefix to some of the cross compiler tools like strip, nm etc.

CC=arm-eabi-gcc –> This tells the compiler that should be used for building

CPPFLAGS –> This tells the location where the header files should be searched which were specified in configure.ac with AC_CHECK_HEADER macro

CFLAGS=”-nostdlib” passes the option to build some test programs during configure process run. If you don’t pass this the compiler would link the standard C library of the host system which wouldn’t be compatible with the C library of the target system. You will end up getting error something like this, if you don’t pass this option:

/home/divkis01/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/../lib/gcc/arm-eabi/4.2.1/../../../../arm-eabi/bin/ld: crt0.o: No such file: No such file or directory

LIBS=”-lc” –> This option tells that it should explicitly link to a library called libc.so which is present in the location specified using the -L in the LDFLAGS option. If you are wondering that usually to build a C executable one doesn’t need to provide -lc as libc is automatically linked, then why do we need to specify this here? The answer lies in -nostdlib flag, which instructs not to link with the standard C library on the build system.

You will end up getting error something like this, if you don’t pass this option:

/home/divkis01/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/../lib/gcc/arm-eabi/4.2.1/../../../../arm-eabi/bin/ld: crt0.o: No such file: No such file or directory
collect2: ld returned 1 exit status

LDFLAGS = –> This option is also passed to build some test programs during configure process run.If you don’t pass the -Wl,-rpath-link option, then linker does not know where do the libraries dependent on the library specific using LIBS reside. If you don’t pass the -L option then the linker doesn’t know where do the libraries specified in LIBS reside.

You will end up getting error something like this, if you don’t pass the -Wl,-rpath-link option:

/home/divkis01/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/../lib/gcc/arm-eabi/4.2.1/../../../../arm-eabi/bin/ld: warning: libdl.so, needed by /home/divkis01/mydroid/development/ndk/build/platforms/android-3/arch-arm/usr/lib//libc.so, not found (try using -rpath or -rpath-link)
/home/divkis01/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/../lib/gcc/arm-eabi/4.2.1/../../../../arm-eabi/bin/ld: warning: cannot find entry symbol _start; defaulting to 00008184
/home/divkis01/mydroid/development/ndk/build/platforms/android-3/arch-arm/usr/lib//libc.so: undefined reference to `dl_unwind_find_exidx’

You will end up getting error something like this, if you don’t pass the -L option:

/home/divkis01/mydroid/prebuilt/linux-x86/toolchain/arm-eabi-4.2.1/bin/../lib/gcc/arm-eabi/4.2.1/../../../../arm-
eabi/bin/ld: cannot find -lc
collect2: ld returned 1 exit status

Once you run the configure script with these flags and options, it will generate the appropriate config.h which is compatible / in sync with your target system. Now you can go ahead and start writing the Android.mk files to build your sources.

Other troubles:

This solves only part of the problem as Autotools not only help in building but also in installing. Given it doesn’t make sense to install the build for a target system on host except for the headers. This can be done by augmenting the Android.mk files or writing some shell scripts to do this manually.

Conclusion:

Autotool is good only on GNU systems and using it for cross compiling can be really tedious, confusing, error prone or even impossible. The method described here is a hack and should be used at your own risk.

Let me know if this post was helpful for you.

CC=arm-eabi-gcc
Advertisements

Android Overview

Why Android?

From what it seems to me, Google had launched this platform not because Google wants to make any profit by directly selling the platform but the key idea is to have the reach of google search beyond desktops to mobiles, PDAs and other similar devices. The benefit that it incurs to Google is pretty obvious, that is, Google Ads to each and every device using Android platform.

What Android Isn’t?

  • It is not really another GNU/Linux distribution.
  • It is not a desktop Operating System.
  • It is not the holy grail to solve all your problems.

What Android is?

Android is a Linux Kernel based OS primarily targeted for small and memory limited devices.

Why Another OS specially Linux based when there are plenty of proprietary OS and flavors of embedded linux?

The trouble with proprietary Operating Systems / platforms:

The trouble with proprietary OS is that they are closed source and access to them is limited:). Until now if you really had to write an application for a device running a proprietary OS, then it would require you to have a hardware device + license to proprietary SDK which would provide tools for building, testing and packaging the apps and sometimes you would need to pay a royalty fees for your application to the OS/ Hardware vendor. Though this situation is changing and there are couple of platforms like Maemo, Moblin, Mobilinux, OpenMoko, BadaOS, most of which are based on open source software but currently there isn’t much thrust that is behind any of these projects.

Trouble with Open Source Operating Systems/ Platforms

As such there is no trouble with Embedded Linux based distributions but for an application developer, there isn’t a consistency that exists on Embedded Linux platforms. From the perspective of application developer, one could not be sure that a certain device, library or feature would be definitely present on the device for which he wants to write the application.

Trouble with Embedded platforms (Open or Closed)

There was no way an application developer could write an application for a particular device, without getting directly involved with the device / OS manufacturer. If someone had the idea of a device or an application for small consumer device, then to build such a device required buying very expensive embedded platforms, lot of software support, cross compilation toolchains, OS, customized bootloaders,  customized kernel, Board support packages etc. Though some of the software components would come for free with the hardware, but the cost of the hardware was so enormous that it was virtually impossible for an individual to even make a prototype of an embedded device. The second problem was that there was no ecosystem in place until now which one could use to develop applications for embedded devices. Thus most device manufacturing companies ended up making complete software platforms for the device with required hardware, software features. There was no contribution or involvement from community in this cycle. This trend really got broken with Iphone where Apple opened up the application development platform to developers. Anyone who could own a Mac book and an Iphone, could now write an application and sell it in the market. Soon after the success of this model, now most platforms like Maemo, BadaOS etc. are trying to follow almost the same model.

The idea behind, Google acquiring Android Inc., the company which made Android, was to have a unified, open platform for mobiles phones, PDA, Nettops, MIDs etc. just like what IPhone offers but at the same time making it an Open Platform for developers and OEMs/ ODMs alike.

What Android offers?

Android Core OS:

  • A modified Linux kernel
  • Libraries for graphics, multimedia, data storage, font etc.
  • Android Runtime consisting of Android Core libraries and a Java Virtual Machine
  • An Application framework for rapid development; consistent look and feel, interaction and behavior of applications.
  • Some basic applications

Android SDK and SDK tools:

  • A comprehensive set of Java APIs for application development
  • Packaging tool
  • Java code Debugging Tools
  • A QEMU based Emulator for running Android applications
  • AVD Manager, an application to manage and create Android Virtual devices
  • ADB, aka Android debugger for connecting and debugging Android Device.

Android NDK:

  • Cross Compiler and other related tools for ARMv5
  • Headers, includes and libraries for building native libraries
  • A build system to easily build native libraries for Android platform

Android Plugin:

  • An Eclipse Plugin for rapid development of Android Apps in Eclipse

Development Model

Java as default programming language

The default or the recommended way of writing apps on Android is using Java with Android provided Java APIs for most tasks. This model has probably been chosen so that there is a consistent way of writing, testing, debugging and deploying an Android application, along with the added benefit of having a consistent look and feel. There is no way one can simply port and install a C application with Main entry point to be run on the Android platform. Even for the apps that written in Java there is no main entry point.

Activity Based instead of Command Based Human <—> Device interaction

Android has abstracted out the way a User interacts with the Operating System in form of activities instead of making User -> Device interaction as file / document based. What that really means is that usually a User wants to do something with the device like making a phone call, editing a spreadsheet, adding contacts, etc. Android calls this as Activity and a user transitions from Activity to Activity.

Summary

Even with so many choices of embedded OS, the application development for a developer is not easy and most of the times developers end up solving lots of platform and systemic issues instead of focusing on application development. Android aims to solve this problem by providing an Open platform for device manufacturers and developers alike, with a comprehensive set of tools for application development, testing and debugging. It comes with all the good features of the Linux kernel with added open source packages which are useful and required in most devices. Access to h/w, devices and OS functionality is hidden to application developers behind the Android’s Java APIs. Overall the platform looks promising but there are two problems that concern most developers.

1. Performance issues as Android applications run on top of a Java Virtual Machine

2. Porting of existing native apps is not possible without rewriting them in Java

Conclusion

With Google’s backing, everyone seems to have an impression that Android will be ‘the’ OS for small and medium size devices. Surely with Google doing a lot of ‘right’ things, strong technical and management leadership, strong financial backing, everyone is bound to believe whatever move Google makes. Only the time and market dynamics will tell if Android is here to stay or not.