Building a wireless Android device using BeagleBone Black

Our previous posts were about code signing in Android, and they turned out to be surprisingly relevant with the announcement of the 'master key' code signing Android vulnerability. While details are yet to be formally released, it has been already patched and dissected, so we'll skip that one and try something different for a change. This post is not directly related to Android security, but will discuss some Android implementation details, so it might be of some interest to our regular readers. Without further ado, let's get closer to the metal than usual and build a wireless Android device (almost) from scratch.

Board introduction -- BeagleBone Black

For our device we'll use the recently released BeagleBone Black board. So what is a BeagleBone Black (let's call it BBB from now on), then? It's the latest addition to the ranks of ARM-based, single board credit-card-sized computers. It comes with an AM335x 1GHz ARM Cortex-A8 CPU, 512MB RAM,  2GB on-board eMMC flash, Ethernet, HDMI and USB ports, plus a whole lot of I/O pins. Best of all, it's open source hardware, and all schematics and design documents are freely available. It's hard to beat the price of $45 and it looks much, much better than the jagged Raspberry Pi. It comes with Angstrom Linux pre-installed, but can run pretty much any Linux flavour, and of course, Android. It is being used for anything from blinking LEDs to tracking satellites. You can hook it up to circuits you've build or quickly extend it using one of the many 'cape' plug-in boards available. We'll use a couple of those for our project, so 'building' refers mostly to creating an Android build compatible with our hardware. We'll detail the hardware later, but let's first outline some simple requirements for our mobile Android device:

1. touch screen input
2. wireless connectivity via WiFi
3. battery powered

Here's what we start with:

Building a kernel for Android

Android support for AM335x-based devices is provided by the rowboat project. It integrates the required kernel and OS patches and provides build configurations for each of the supported devices, including the BBB. The latest version is based on Android 4.2.2 and if you want to get started quickly, you can download a binary build from TI's Sitara Android development kit page. All you need to do is flash it to an SD card, connect the BBB to an HDMI display and power it on. You will instantly get a fully working, hardware-accelerated Jelly Bean 4.2 device you can control using standard USB keyboard and mouse. If that is all you need, you might as well stop reading here. Our first requirement, however is a working touch screen, not an HDMI monitor, so we have some work to do. As it happens, a number of LCD capes are already available for the BBB (from circuitco and others), so those are our first choice. We opted for the LCD4 4.3" cape which offers almost reasonable resolution and is small enough to be directly attached to the BBB. Unfortunately it doesn't work with the rowboat build from TI. To understand why, let's take a step back and discuss how the BBB supports extension hardware, including capes.

Linux Device Tree and cape support

If you look at the expansion header pinout table in the BBB reference manual, you will notice that each pin can serve multiple purposes, depending on configuration. This is called 'pinmuxing' and is the method modern SoC's use to multiplex multiple peripheral functions to a limited set of physical pins. The AM335x CPU the BBB uses is no exception: it has pins with up to 8 possible peripheral functions. So, in order for a cape to work, the SoC needs to be configured to use the correct inputs/outputs for that cape. The situation becomes more complicated when you have multiple capes (up to 4 at a time). BBB capes solve this by using EEPROM that stores enough data to identify the cape, its revision and serial number. At boot time, the kernel identifies the capes by reading their EEPROMs, computes the optimal configuration (or outputs and error if the connected capes are not compatible) and sets the expansion header pinmux accordingly. Initially, this was implemented in a 'board file' in the Linux kernel, and adding a new cape required modifying the kernel and making sure all possible cape configurations were supported. Needless to say, this is not an easy task, and getting it merged into Linux mainline is even harder. Since everyone is building some sort of ARM device nowadays, the number of board files and variations thereof reached critical mass, and Linux kernel maintainers decided to decouple board specific behaviour from the kernel. The mechanism for doing this is called Device Tree (DT) and its goal is to make life easier for both device developers (no need to hack the kernel for each device) and kernel maintainers (no need to merge board-specific patches every other day). A DT is a data structure for describing hardware which is passed to the kernel at boot time. Using the DT, a generic board driver can configure itself dynamically. The BBB ships with a 3.8 kernel and takes full advantage of the new DT architecture. Cape support is naturally implemented using DT source (DTS) files and even goes a step further than mainline Linux by introducing a Cape Manager, an in-kernel mechanism for dynamically loading Device Tree fragments from userspace. This allows for runtime (vs. boot time) loading of capes via sysfs, resource conflict resolution (where possible), manual control over already loaded capes and more.

Going back to Android, the rowboat Android port is using the 3.2 kernel and relies on manual porting of extension peripheral configuration to the kernel board file. As it happens, support for our LCD4 cape is not there yet. We could try to patch the kernel based on the 3.8 DTS files, or take the plunge and attempt to run Android using 3.8. Since all BBB active development is going on in the 3.8 branch, using the newer version is the better (if more involved) choice.

Using the 3.8 kernel

As we know, Android adds a bunch of 'Androidisms' to the Linux kernel, most notably wakelocks, alarm timers, ashmem, binder, low memory killer and 'paranoid' network security. Thus you could not use a vanilla Linux kernel as is to run Android until recently, and a number of Android-specific patches needed to be applied first. Fortunately, thanks to the Android Mainlining Project, most of these features are already merged (in one form or another) in the 3.8 kernel and are available as staging drivers. What this means is that we can take a 3.8 kernel that works well on the BBB and use it run Android. Unfortunately, the BBB can't quite use a vanilla 3.8 kernel yet and requires quite a few patches (including Cape Manager). However, building a 3.8 kernel with all BBB patches applied is not too hard to do, thanks to instructions and build scripts by Robert Nelson. Even better, Andrew Henderson has successfully used it in Android and has detailed the procedure. Following Andrew's build instructions, we can create an Android build that has a good chance of supporting our touch screen. As Andrew's article mentions, hardware acceleration (support for the BBB's PowerVR SGX 530 GPU) is not yet available for the 3.8 kernel, so we need to disable it in our build. One thing that is missing from Andrew's instruction is that you also need to disable building and installing of the SGX drivers, otherwise Android will try to use them at boot and fail to start SurfaceFlinger due to driver-kernel module incompatibility. You can do this by commenting out the dependency on sgx in rowboat's top-level Makefile like this:

@@ -11,7 +13,7 @@
 CLEAN_RULE = sgx_clean wl12xx_compat_clean kernel_clean clean
 else
 ifeq ($(TARGET_PRODUCT), beagleboneblack)
-rowboat: sgx
+#rowboat: sgx
 CLEAN_RULE = sgx_clean kernel_clean clean
 else
 ifeq ($(TARGET_PRODUCT), beaglebone)

Note that the kernel alone is not enough though: the boot loader (Das U-Boot) needs to be able to load the (flattened) device tree blob, so we need to build a recent version of that as well. Android seems to run OK with this configuration, but there are still a few things that are missing. The first you might notice is ADB support.

ADB support

ADB (Android Debug Bridge) is one of the best things to came out of the Android project, and if you have been doing Android development in any form for a while, you probably take it for granted. It is a fairly complex piece of software though, providing support for debugging, file transfer, port forwarding and more and requires kernel support in addition to the Android daemon and client application. In kernel terms this is known as the 'Android USB Gadget Driver', and it is not quite available in the 3.8 kernel, even though there have been multiple attempts at merging it. We can merge the required bits from Google's 3.8 kernel tree, but since we are trying to stay as close as possible to the original BBB 3.8 kernel, we'll use a different approach. While attempts to get ADB in the mainline continue, Function Filesystem (FunctionFS) driver support has been added to Android's ADB and we can use that instead of the 'native' Android gadget. To use ADB with FunctionFS:

1. Configure FunctionFS support in the kernel (CONFIG_USB_FUNCTIONFS=y):

Device Drivers -> USB Support -> 
     USB Gadget Support -> USB Gadget Driver -> Function Filesystem

2. Modify the boot parameters in uEnv.txt to set the vendor and product IDs, as well as the device serial number

g_ffs.idVendor=0x18d1 g_ffs.idProduct=0x4e26 g_ffs.iSerialNumber=<serial>

3. Setup the FunctionFS directory and mount it in your init.am335xevm.usb.rc file:

on fs
    mkdir /dev/usb-ffs 0770 shell shell
    mkdir /dev/usb-ffs/adb 0770 shell shell
    mount functionfs adb /dev/usb-ffs/adb uid=2000,gid=2000

4. Delete all lines referencing /sys/class/android_usb/android0/*. (Those nodes are created by the native Android gadget driver and are not available when using FunctionFS.)
   

Once this is done, you can reboot and you should see your device using adb devices soon after the kernel has loaded. Now you can debug the OS using Eclipse and push and install files directly using ADB. That said, this won't help you at all if the device doesn't boot due to some kernel misconfiguration, so you should definitely get an FTDI cable (the BBB does not have an on-board FTDI chip) to be able to see kernel messages during boot and get an 'emergency' shell when necessary.

cgroups patch

If you are running adb logcat in a console and experimenting with the device, you will notice a lot of 'Failed setting process group' warnings like this one:

W/ActivityManager(  349): Failed setting process group of 4911 to 0
W/SchedPolicy(  349): add_tid_to_cgroup failed to write '4911' (Permission denied);

Android's ActivityManager uses Linux control groups (cgroups) to run processes with different priorities (background, foreground, audio, system) by adding them to scheduling groups. In the mainline kernel this is only allowed to processes running as root (EUID=0), but Android changes this behaviour (naturally, with a patch) to only require the CAP_SYS_NICE capability, which allows the ActivityManager (running as system in the system_server process) to add app processes to scheduling groups. To get rid of this warning, you can disable scheduling groups by commenting out the code that sets up /dev/cpuctl/tasks in init.rc, or you can merge the modified functionality form Google's experimental 3.8 branch (which we've been trying to avoid all along...).

Android hardware support

Touchscreen

We now have a functional Android development device running mostly without warnings, so it's time to look closer at requirement #1. As we mentioned, once we disable hardware acceleration, the LCD4 works fine with our 3.8 kernel, but a few things are still missing. The LCD4 comes with 5 directional GPIO buttons which are somewhat useful because scrolling on a resistive touchscreen takes some getting used to, but that is not the only thing they can be used for. We can remap them as Android system buttons (Back, Home, etc) by providing a key layout (.kl) file like this one:

key 105   BACK               WAKE
key 106   HOME               WAKE
key 103   MENU               WAKE
key 108   SEARCH             WAKE
key 28    POWER              WAKE

The GPIO keypad on the LCD identifies itself as 'gpio.12' (you can check this using the getevent command), so we need to name the layout file to 'gpio_keys_12.kl'. To achieve this we modify device.mk in the BBB device directory (device/ti/beagleboneblack):

...
# KeyPads
PRODUCT_COPY_FILES += \
    $(LOCAL_PATH)/gpio-keys.kl:system/usr/keylayout/gpio_keys_12.kl \
...

Now that we are using hardware buttons, we might want to squeeze some more screen real estate from the LCD4 by not showing the system navigation bar. This is done by setting config_showNavigationBar to false in the config.xml framework overlay file for our board:

<bool name="config_showNavigationBar">false</bool>

While playing with the screen, we notice that it's a bit dark. Increasing the brightness via the display settings however does not seem to work. A friendly error message in logcat tells us that Android can't open the /sys/class/backlight/pwm-backlight/brightness file. Screen brightness and LEDs are controlled by the lights module on Android, so that's where we look first. There is a a hardware-specific one under the beagleboneblack device directory, but it only supports the LCD3 and LCD7 displays. Adding support for the LCD4 is simply a matter of finding the file that controls brightness under /sys. For the LCD4 it's called /sys/class/backlight/backlight.10/brightness and works exactly like the other LCDs -- you get or set the brightness by reading or writing the backlight intensity level (0-100) as a string. We modify light.c (full source on Github) to first try the LCD4 device and voila -- setting the brightness via the Android UI now works... not. It turns out the brightness file is owned by root and the Settings app doesn't have permission to write to it. We can change this permission in the board's init.am335xevm.rc file:

# PWM-Backlight for display brightness on LCD4 Cape
chmod 0666 /sys/class/backlight/backlight.10

This finally settles it, so we can cross requirement #1 off our list and try to tackle #2 -- wireless support.

WiFi adapter

The BBB has an onboard Ethernet port and it is supported out of the box by the rowboat build. If we want to make our new Android device mobile though, we need to add either a WiFi adapter or 3G modem. 3G support is possible, but somewhat more involved, so we will try to enable WiFi first. There are a number of capes that provide WiFi and Bluetooth for the original BeagleBone, but they are not compatible with the BBB, so we will try using a regular WiFi dongle instead. As long as it has a Linux driver, it should be quite easy to wire it to Android by following the TI porting guide, right?

We'll use a WiFi dongle from LM Technolgies based on the Realtek RTL8188CUS chipset which is supported by the Linux rtl8192cu driver. In addition to the kernel driver, this wireless adapter requires a binary firmware blob, so we need to make sure it's loaded along with the kernel modules. But before getting knee-deep into makefiles, let's briefly review the Android WiFi architecture. Like most hardware support in Android, it consists of a kernel layer (WiFi adapter driver modules), native daemon (wpa_supplicant), HAL (wifi.c in libharware_legacy, communicates with wpa_supplicant via its control socket), a framework service and its public interface (WifiService and WifiManager) and application/UI ('WiFi' screen in the Settings app, as well as SystemUI, responsible for showing the WiFi status bar indicator). That may sound fairly straightforward, but the WifiService implements some pretty complex state transitions in order to manage the underlying native WiFi support. Why is all the complexity needed? Android doesn't load kernel modules automatically, so the WifiStateMachine will try to load kernel modules, find and load any necessary firmware, start the wpa_supplicant daemon, scan for and connect to an AP, obtain an IP address via DHCP, check for and handle captive portals, and finally, if you are lucky, set up the connection and send out a broadcast to notify the rest of the system of the new network configuration. The wpa_supplicant daemon alone can go through 13 different states, so things can get quite involved when those are combined.

Going step-by-step through the porting guide, we first enable support for our WiFi adapter in the kernel. That results in 6 modules that need to be loaded in order, plus the firmware blob. The HAL (wifi.c) can only load a single module though, so we pre-load all modules in the board's init.am335xevm.rc and set the wlan.driver.status to ok in order to prevent WifiService from trying (and failing) to load the kernel module. We then define the wpa_supplicant and dhcpd services in the init file. Last, but not least, we need to set the wifi.interface property to wlan0, otherwise Android will silently try to use a test device and fail to start the wpa_supplicant. Both properties are set as PRODUCT_PROPERTY_OVERRIDES in device/ti/beagleboneblack/device.mk (see device directory on Github). Here's how the relevant part from init.am335xevm.rc looks like:

on post-fs-data
    # wifi
    mkdir /data/misc/wifi/sockets 0770 wifi wifi
    insmod /system/lib/modules/rfkill.ko
    insmod /system/lib/modules/cfg80211.ko
    insmod /system/lib/modules/mac80211.ko
    insmod /system/lib/modules/rtlwifi.ko
    insmod /system/lib/modules/rtl8192c-common.ko
    insmod /system/lib/modules/rtl8192cu.ko

service wpa_supplicant /system/bin/wpa_supplicant \
       -iwlan0 -Dnl80211 -c/data/misc/wifi/wpa_supplicant.conf \
       -e/data/misc/wifi/entropy.bin
       class main
       socket wpa_wlan0 dgram 660 wifi wifi
       disabled
       oneshot

service dhcpcd_wlan0 /system/bin/dhcpcd -ABKL
       class main
       disabled
       oneshot

service iprenew_wlan0 /system/bin/dhcpcd -n
       class main
       disabled
       oneshot

In order to build the wpa_supplicant daemon, we then set BOARD_WPA_SUPPLICANT_DRIVER and WPA_SUPPLICANT_VERSION in device/ti/beagleboneblack/BoardConfig.mk. Note the we are using the generic wpa_supplicant, not the TI-patched one and the WEXT driver instead of the NL80211 one (which requires a proprietary library to be linked in). Since we are preloading driver kernel modules, we don't need to define WIFI_DRIVER_MODULE_PATH and WIFI_DRIVER_MODULE_NAME.

BOARD_WPA_SUPPLICANT_DRIVER      := WEXT
WPA_SUPPLICANT_VERSION           := VER_0_8_X
BOARD_WLAN_DEVICE                := wlan0

To make the framework aware of our new WiFi device, we change networkAttributes and radioAttributes in the config.xml overlay file. Getting this wrong will lead to Android's ConnectionManager totally ignoring WiFi even if you manage to connect and will result in the not too helpful 'No network connection' message. "1" here corresponds to the ConnectivityManager.TYPE_WIFI connection type (the built-in Ethernet connection is "9", TYPE_ETHERNET).

<string-array name="networkAttributes" translatable="false">
...
  <item>"wifi,1,1,1,-1,true"</item>
...
</string-array>
<string-array name="radioAttributes" translatable="false">
  <item>"1,1"</item>
...
</string-array>

Finally, to make Android aware of our newly found WiFi features, we copy android.hardware.wifi.xml to /etc/permissions/ by adding it to device.mk. This will take care of enabling the Wi-Fi screen in the Settings app:

PRODUCT_COPY_FILES := \
...
 frameworks/native/data/etc/android.hardware.wifi.xml:system/etc/permissions/android.hardware.wifi.xml \
...

After we've rebuild rowboat and updated the root file system, you should be able to turn on WiFi and connect to an AP. Make sure you are using an AC power supply to power the BBB, because the WiFi adapter can draw quite a bit of current and you may not get enough via the USB cable. If the board is not getting enough power, you might experience failure to scan, dropping connections and other weird symptoms even if your configuration is otherwise correct. If WiFi support doesn't work for some reason, check the following:

• that the kernel module(s) and firmware (if any) is loaded (dmesg, lsmod)
• logcat output for relevant-lookin error messages
• that the wpa_supplicant service is defined properly in init.*.rc and the daemon is started
• that /data/misc/wifi and wpa_supplicant.conf are available and have the right owner and permissions (wifi:wifi and 0660)
• that the wifi.interface and wlan.driver.status properties are set correctly
• use your debugger if all else fails

That was easy, right? We now have a working wireless connection, it's time to think about requirement #3, powering the device.

Battery power

The BBB can be powered in three ways: via the miniUSB port, via the 5V AC adapter jack, and by using the power rail (VDD_5V) on the board directly. We can use any USB battery pack that provides enough current (~1A) and has enough capacity to keep the device going by simply connecting it to the miniUSB port. Those can be rather bulky and you will need an extra cable, so let's look for other options. As can be expected, there is a cape for that. The aptly named Battery Cape plugs into the BBB's expansion connectors and provides power directly to the power rail. We can plug the LCD4 on top of it and get an integrated (if a bit bulky) battery-powered touchscreen device. The Battery Cape holds 4 AA batteries connected as two sets in parallel. It is not simply a glorified battery holder though -- it has a boost converter that can provide stable 1A current at 5V even if battery voltage fluctuates (1.8-5.5V). It does provide support for monitoring battery voltage via AIN4 input, but does not have a 'fuel gauge' chip so we can't display battery level in Android without adding additional circuitry. That is ways our mobile device cannot display the battery level (yet) and unfortunately won't be able to shut itself down when battery levels become critically low. That is something that definitely needs work, but for now we make the device always believe it's at 100% power by setting the hw.nobattery property to true. The alternative is to have it display the 'low battery' red warning icon all the time, so this approach is somewhat preferable. Four 1900 mAh batteries installed in the battery cape should provide enough power to run the device for a few hours even when using WiFi, so we can (tentatively) mark requirement #3 as fulfilled.

Flashing the device

If you have been following Andrew Henderson's build guide linked above, you have been 'installing' Android on an SD card and booting the BBB from it. This works fine and makes it easy to fix things when Android won't load by simply mounting the SD card on your PC and editing or copying the necessary files. However, most consumer grade SD cards don't offer the best performance and can be quite unreliable. As we mentioned at the beginning of the post, the BBB comes with 2GB of built-in eMMC, which is enough to install Android and have some space left for a data partition. On most Android devices flashing can be performed by either booting into the recovery system or by using the fastboot tool over USB. The rowboat build does not have a recovery image, and while fastboot is supported by TI's fork of U-Boot, the version we are using to load the DT blob does not support fastboot yet. That leaves booting another OS in lieu of a recovery and flashing the eMMC form there, either manually or by using an automated flasher image. The flasher image simply runs a script at startup, so let's see how it works by doing it manually first. The latest BBB Angstrom bootable image (not the flasher one) is a good choice for our 'recovery' OS, because it is known to work on the BBB and has all the needed tools (fdisk, mkfs.ext4, etc.). After you dd it to an SD card, mount the card on your PC and copy the Android boot files and rootfs archive to an android/ directory. You can then boot from the SD card, get a root shell on the Angstrom and install Android to the eMMC from there.

Android devices typically have a boot, system and userdata parition, as well as a recovery one and optionally others. The boot partition contains the kernel and a ramdisk which gets mounted at the root of the device filesystem. system contains the actual OS files and gets mounted read-only at /system, while userdata is mounted read-write at /data and stores system and app data, as well user-installed apps. The partition layout used by the BBB is slightly different. The board ootloader will look for the first stage bootloader (SPL, named MLO in U-Boot) on the first FAT partition of the eMMC. It in turn will load the second state bootloader (u-boot.img) which will then search for a OS image according to its configuration. On embedded devices U-Boot configuration is typically stored as a set of variables in NAND, replaced by the uEnv.txt file on devices without NAND such as the BBB. Thus we need a FAT boot partition to host the SPL, u-boot.img, uEnv.txt, the kernel image and the DT blob. system and userdata will be formatted as EXT4 and will work as in typical Android devices.

The default Angstrom installations creates only two partitions -- a DOS one for booting, and a Linux one that hosts Angstrom Linux. To prepare the eMMC for Android, you need to delete the Linux partition and create two new Linux partitions in its place -- one to hold Android system files and one for user data. If you don't plan to install too many apps, you can simply make them equal sized. When booting from the SD card, the eMMC device will be /dev/block/mmcblk1, with the first partition being /dev/block/mmcblk1p1, the second /dev/block/mmcblk1p2 and so on. After creating those 3 partitions with fdisk we format them with their respective filesystems:

# mkfs.vfat -F 32 -n boot /dev/block/mmcblk1p1
# mkfs.ext4 -L rootfs /dev/block/mmcblk1p2 
# mkfs.ext4 -L usrdata /dev/block/mmcblk1p3 

Next, we mount boot and copy boot related files, then mount rootfs and untar the rootfs.tar.bz2 archive. usrdata can be left empty, it will be populated on first boot.

# mkdir -p /mnt/1/
# mkdir -p /mnt/2/
# mount -t vfat /dev/block/mmcblk1p1 /mnt/1
# mount -t ext4 /dev/block/mmcblk1p2 /mnt/2
# cp MLO u-boot.img zImage uEnv.txt am335x-boneblack.dtb /mnt/1/
# tar jxvf rootfs.tar.bz2 -C /mnt/2/
# umount /mnt/1
# umount /mnt/2

With this, Android is installed on the eMMC and you can shutdown the 'recovery' OS, remove the SD card and boot from the eMMC. Note that the U-Boot used has been patched to probe whether the SD card is available and will automatically boot from it (without you needing to hold the BBB's user boot button), so if you don't remove the 'recovery' SD card, it will boot again.

We now have a working, touch screen Android device with wireless connectivity. Here's how it looks in action:

Our device is unlikely to win any design awards or replace your Nexus 7, but it could be used as the basis of  dedicated Android devices, such as a wireless POS terminal or a SIP phone and extended even further by adding more capes or custom hardware as needed.

Summary

The BBB is fully capable of running Android and by adding off-the shelf peripherals you can easily turn it into a 'tablet' (of sorts) by adding a touch screen and wireless connectivity. While the required software is mostly available in the rowboat project, if you want to have the best hardware support you need to use BBB's native 3.8 kernel and configure Android to use it. Making hardware fully available to the Android OS is mostly a matter of configuring the relevant HAL bits properly, but that is not always straightforward, even with board vendor provided documentation. The reason for this is that Android subsystems are not particularly cohesive -- you need to modify multiple, sometimes seemingly unrelated, files at different locations to get a single subsystem working. This is, of course, not specific to Android and is the price to pay for building a system by integrating originally unrelated OSS projects. On the positive side, most components can be replaced and the required changes can usually be confined to the (sometimes loosely defined) Hardware Abstraction Layer (HAL).