Android online account management

Our recent posts covered NFC and the secure element as supported in recent Android versions, including community ones. In this two-part series we will take a completely different direction: managing online user accounts and accessing Web services. We will briefly discuss how Android manages user credentials and then show how to use cached authentication details to log in to most Google sites without requiring additional user input. Most of the functionality we shall discuss is hardly new -- it has been available at least since Android 2.0. But while there is ample documentation on how to use it, there doesn't see to be a 'bigger picture' overview of how the pieces are tied together. This somewhat detailed investigation was prompted by trying to develop an app for a widely used Google service that unfortunately doesn't have an official API and struggling to find a way to login to it using cached Google credentials. More on this in the second part, let's first see how Android manages accounts for online services.

Android account management

Android 2.0 (API Level 5, largely non-existent, because it was quickly succeeded by 2.0.1, Level 6), introduced the concept of centralized account management with a public API. The central piece in the API is the AccountManager class which, quote: 'provides access to a centralized registry of the user's online accounts. The user enters credentials (user name and password) once per account, granting applications access to online resources with "one-click" approval.' You should definitely read the full documentation of the class, which is quite extensive, for more details. Another major feature of the class is that it lets you get an authentication token for supported accounts, allowing third party applications to authenticate to online services without needing to handle the actual user password (more on this later). It also has a whole of 5 methods that allow you to get an authentication token, all but one with at least 4 parameters, so finding the one you need might take some time, with yet some more to get the parameters right. It might be a good idea to start with the synchronous blockingGetAuthToken() and work your way from there once you have a basic working flow. On some older Android versions, the AccountManager would also monitor your SIM card and wipe cached credentials if you swapped cards, but fortunately this 'feature' has been removed in Android 2.3.4.

The AccountManager, as most Android system API's, is just a facade for the AccountManagerService which does the actual work. The service doesn't provide an implementation for any particular form of authentication though. It only acts as a coordinator for a number of pluggable authenticator modules for different account types (Google, Twitter, Exchange, etc.). The best part is that any application can register an authentication module by implementing an account authenticator and related classes, if needed. Android Training has a tutorial on the subject that covers the implementation details, so we will not discuss them here. Registering a new account type with the system lets you take advantage of a number of Android infrastructure services:

• centralized credential storage in a system database
• ability to issue tokens to third party apps
• ability to take advantage of Android's automatic background synchronization

One thing to note is that while credentials (usually user names and passwords) are stored in a central database (/data/system/accounts.db or /data/system/user/0/accounts.db on Jelly Bean and later for the first system user), that is only accessible to system applications, credentials are in no way encrypted -- that is left to the authentication module to implement as necessary. If you have a rooted device (or use the emulator) listing the contents of the accounts table might be quite instructive: some of your passwords, especially for the stock Email application, will show up in clear text. While the AccountManger has a getPassword() method, it can only be used by apps with the same UID as the account's authenticator, i.e., only by classes in the same app (unless you are using sharedUserId, which is not recommended for non-system apps). If you want to allow third party applications to authenticate using your custom accounts, you have to issue some sort of authentication token, accessible via one of the many getAuthToken() methods. Once your account is registered with Android, if you implement an additional sync adapter, you can register to have it called at a specified interval and do background syncing for you app (one- or two-way), without needing to manage scheduling yourself. This is a very powerful feature that you get practically for free, and probably merits its own post. As we now have a basic understanding of authentication modules, let's see how they are used by the system.

As we mentioned above, account management is coordinated by the AccountManagerService. It is a fairly complex piece of code (about 2500 lines in JB), most of the complexity stemming from the fact that it needs to communicate with services and apps that span multiple processes and threads within each process, and needs to take care of synchronization and delivering results to the right thread. If we abstract out the boilerplate code, what it does on a higher level is actually fairly straightforward:

• on startup it queries the PackageManager to find out all registered authenticators, and stores references to them in a map, keyed by account type
• when you add an account of a particular type, it saves its type, username and password to the accounts table
• if you get, set or reset the password for an account, it accesses or updates the accounts table accordingly
• if you get or set user data for the account, it is fetched from or saves to the extras table
• when you request a token for a particular account, things become a bit more interesting:
• if a token with the specified type has never been issued before, it shows a confirmation activity asking (see screenshot below) the user to approve access for the requesting application. If they accept, the UID of the requesting app and the token type are saved to the grants table.
• if a grant already exits, it checks the authtoken table for tokens matching the request. If a valid one exists, it is returned.
• if a matching token is not found, it finds the authenticator for the specified account type in the map and calls its getAuthToken() method to request a token. This usually involves the authenticator fetching the username and password from the accounts table (via the getPassword() method) and calling its respective online service to get a fresh token. When one is returned, it gets cached in the authtokens table and then returned to the requesting app (usually asynchronously via a callback).
• if you invalidate a token, it gets deleted from the authtokens table

Now that we know how Android's account management system works, let's see how it is implemented for the most widely used account type.

Google account management

Usually the first thing you do when you turn on your brand new (or freshly wiped) 'Google Experience' Android device is to add a Google account. Once you authenticate successfully, you are offered to sync data from associated online services (GMail, Calendar, Docs, etc.) to your device. What happens behinds the scenes is that an account of type 'com.google' is added via the AccountManager, and a bunch of Google apps start getting tokens for the services they represent. Of course, all of this works with the help of an authentication provider for Google accounts. Since it plugs in the standard account management framework, it works by registering an authenticator implementation and using it involves the sequence outlined above. However, it is also a little bit special. Three main things make it different:

• it is not part of any particular app you can install, but is bundled with the system
• a lot of the actual functionality is implemented on the server side
• it does not store passwords in plain text on the device

If you have ever installed a community ROM built off AOSP code, you know that in order to get GMail and other Google apps to work on your device, you need a few bits not found in AOSP. Two of the required pieces are the Google Services Framework (GSF) and the Google Login Service (GLS). The former provides common services to all Google apps such as centralized settings and feature toggle management, while the latter implements the authentication provider for Google accounts and will be the topic of this section.

Google provides a multitude of online services (not all of which survive for long), and consequently a bunch of different methods to authenticate to those. Android's Google Login Service, however doesn't call those public authentication API's directly, but via a dedicated online service, which lives at android.clients.google.com. It has endpoints both for authentication and authorization token issuing, as well as data feed (mail, calendar, etc.) synchronization, and more. As we shall see, the supported methods of authentication are somewhat different from those available via other public Google authentication API's. Additionally, it supports a few 'special' token types that greatly simplify some complex authentication flows.

All of the above is hardly surprising: when you are dealing with online services it is only natural to have as much as possible of the authentication logic on the server side, both for ease of maintenance and to keep it secure. Still, to kick start it you need to store some sort of credentials on the device, especially when you support background syncing for practically everything and you cannot expect people to enter them manually. On-device credential management is one of the services GLS provides, so let's see how it is implemented. As mentioned above, GLS plugs into the system account framework, so cached credentials, tokens and associated extra data are stored in the system's accounts.db database, just as for other account types. Inspecting it reveals that Google accounts have a bunch of Base64-encoded strings associated with them. One of the user data entries (in the extras table) is helpfully labeled sha1hash (but does not exist on all Android versions) and the password (in the accounts table) is a long string that takes different formats on different Android versions. Additionally, the GSF database has a google_login_public_key entry, which when decoded suspiciously resembles a 1024-bit RSA public key. Some more experimentation reveals that credential management works differently on pre-ICS and post-ICS devices. On pre-ICS devices, GLS stores an encrypted version of your password and posts it to the server side endpoints both when authenticating for the first time (when you add the account) and when it needs to have a token for a particular service issued. On post-ICS devices, it only posts the encrypted password the first time, and gets a 'master token' in exchange, which is then stored on the device (in the password column of the accounts database). Each subsequent token request uses that master token instead of a password.

Let's look into the cached credential strings a bit more. The encrypted password is 133 bytes long, and thus it is a fair bet that it is encrypted with the 1024-bit (128 bytes) RSA public key mentioned above, with some extra data appended. Adding multiple accounts that use the same password produces different password strings (which is a good thing), but the first few bytes are always the same, even on different devices. It turns out those identify the encryption key and are derived by hashing its raw value and taking the leading bytes of the resulting hash. At least from our limited sample of Android devices, it would seem that the RSA public key used is constant both across Android versions and accounts. We can safely assume that its private counterpart lives on the server side and is used to decrypt sent passwords before performing the actual authentication. The padding used is OAEP (with SHA1 and MGF1), which produces random-looking messages and is currently considered secure (at least when used in combination with RSA) against most advanced cryptanalysis techniques. It also has quite a bit of overhead, which in practice means that the GLS encryption scheme can encrypt at most 86 bytes of data. The outlined encryption scheme is not exactly military-grade and there is the issue of millions of devices most probably using the same key, but recovering the original password should be sufficiently hard to discourage most attackers. However, let's not forget that we also have a somewhat friendlier SHA1 hash available. It turns out it can be easily reproduced by 'salting' the Google account password with the account name (typically GMail address) and doing a single round of SHA1. This is considerably easier to do and it wouldn't be too hard to precompute a bunch of hashes based on commonly used or potential passwords if you knew the target account name.

Fortunately, newer version of Android (4.0 and later) no longer store this hash on the device. Instead of the encrypted password+SHA1 hash combination they store an opaque 'master token' (most probably some form of OAuth token) in the password column and exchange it for authentication tokens for different Google services. It is not clear whether this token ever expires or if it is updated automatically. You can, however, revoke it manually by going to the security settings of your Google account and revoking access for the 'Android Login Service' (and a bunch of other stuff you never use while you are at it). This will force the user to re-authenticate on the device next time it tries to get a Google auth token, so it is also somewhat helpful if you ever lose your device and don't want people accessing your email, etc. if they manage to unlock it. The service authorization token issuing protocol uses some device-specific data in addition to the master token, so obtaining only the master token should not be enough to authenticate and impersonate a device (it can however be used to login into your Google account on the Web, see the second part for details).

Google Play Services

Google Play Services (we'll abbreviate it to GPS, although the actual package is com.google.android.gms, guess where the 'M' came from) was announced at this year's Google I/O as an easy to use platform that offers integration with Google products for third-party Android apps. It was actually rolled out only a month ago, so it's probably not very widely used yet. Currently it provides support for OAuth 2.0 authorization to Google API's 'with a good user experience and security', as well some Google+ plus integration (sign-in and +1 button). Getting OAuth 2.0 tokens via the standard AccountManager interface has been supported for quite some time (though support was considered 'experimental') by using the special 'oauth2:scope' token type syntax. However, it didn't work reliably across different Android builds, which have different GLS versions bundled and this results in slightly different behaviour. Additionally, the permission grant dialog shown when requesting a token was not particularly user friendly, because it showed the raw OAuth 2.0 scope in some cases, which probably means little to most users (see screenshot in the first section). While some human-readable aliases for certain scopes where introduced (e.g., 'Manage your taks' for 'oauth2:https://www.googleapis.com/auth/tasks'), that solution was neither ideal, nor universally available. GPS solves this by making token issuing a two-step process (newer GLS versions also use this process):

1. the first request is much like before: it includes the account name, master token (or encrypted password pre-ICS) and requested service, in the 'oauth2:scope' format. GPS adds two new parameters: requesting app package name and app signing certificate SHA1 hash (more on this later). The response includes some human readable details about the requested scope and requesting application, which GPS shows in a permission grant dialog like the one shown below.
2. if the users grants the permission, this decision is recorded in the extras table in a proprietary format which includes the requesting app's package name, signing certificate hash, OAuth 2.0 scope and grant time (note that it is not using the grants table). GPS then resends the authorization request setting the has_permission parameter to 1. On success this results in an OAuth 2.0 token and its expiry date in the response. Those are cached in the authtokens table in a similar format.

To be able to actually use a Google API, you need to register your app's package name and signing key in Google's API console. The registration lets services validating the token query Google what app the token was issued for, and thus identify the calling app. This has one subtle, but important side-effect: you don't have to embed an API key in your app and send it with every request. Of course, for a third party published app you can easily find out both the package name and the signing certificate so it is not particularly hard to get a token issued in the name of some other app (not possible via the official API, of course). We can assume that there are some additional checks on the server side that prevent this, but theoretically, if you used such a token you could, for example, exhaust a third-party app's API request quota by issuing a bunch of requests over a short period of time. 

The actual GPS implementation seems to reuse much of the original Google Login Service authentication logic, including the password encryption method, which is still used on pre-ICS devices (the protocol is, after all, mostly the same and it needs to be able to use pre-existing accounts). On top of that it adds better OAuth 2.0 support, a version-specific account selection dialog and some prettier and more user friendly permission grant UIs. The GPS app has the Google apps shared UID, so it can directly interact with other proprietary Google services, including GLS and GSF. This allows it, among other things, to directly get and write Google account credentials and tokens to the accounts database. As can be expected, GPS runs in a remote service that the client library you link into your app accesses. The major selling point against the legacy AccountManager API is that while its underlying authenticator modules (GLS and GSF) are part of the system, and as such cannot be updated without an OTA, GPS is an user-installable app that can be easily updated via Google Play. Indeed, it is advertised as auto-updating (much like the Google Play Store client), so app developers presumably won't have to rely on users to update it if they want to use newer features (unless GPS is disabled altogether, of course). This update mechanism is to provide 'agility in rolling out new platform capabilities', but considering how much time the initial roll-out took, it is to be seen how agile the whole thing will turn out to be. Another thing to watch out for is feature bloat: besides OAuth 2.0 support, GPS currently includes G+ and AdMob related features, and while both are indeed Google-provided services, they are totally unrelated. Hopefully, GPS won't turn into a 'everything Google plus the kitchen sink' type of library, delaying releases even more. With all that said, if your app uses OAuth 2.0 tokens to authenticate to Google API's, which is currently the preferred method (ClientLogin, OAuth 1.0 and AuthSub have been officially deprecated), definitely consider using GPS over 'raw' AccountManager access.

Summary

Android provides a centralized registry of user online accounts via the AccountManager class. It lets you both get tokens for existing accounts without having to handle the actual credentials and register your own account type, if needed. Registering an account type gives you access to powerful system features, such as authentication token caching and automatic background synchronization. 'Google experience' devices come with built-in support for Google accounts, which lets third party apps access Google online services without needing to directly request authentication information from the user. The latest addition to this infrastructure is the recently released Google Play Services app and companion client library, which aim to make it easy to use OAuth 2.0 from third party applications. 

We've now presented an overview of how the account management system works, and the next step is to show how to actually use it to access a real online service. That will be the topic of the second article in the series.