Using KitKat verified boot

Android 4.4 introduced a number of security enhancements, most notably SELinux in enforcing mode. One security feature that initially got some press attention, because it was presumably aiming to 'end all custom firmware', but hasn't been described in much detail, is verified boot. This post will briefly explain how verified boot works and then show how to configure and enable it on a Nexus device.

Verified boot with dm-verity

Android's verified boot implementation is based on the dm-verity device-mapper block integrity checking target. Device-mapper is a Linux kernel framework that provides a generic way to implement virtual block devices. It is used to implement volume management (LVM), full-disk encryption (dm-crypt), RAIDs and even distributed replicated storage (DRBD). Device-mapper works by essentially mapping a virtual block device to one or more physical block devices, optionally modifying transferred data in transit. For example, dm-crypt decrypts read physical blocks and encrypts written blocks before committing them to disk. Thus disk encryption is transparent to users of the virtual dm-crypt block device. Device-mapper targets can be stacked on top of each other, making it possible to implement complex data transformations. 

As we mentioned, dm-verity is a block integrity checking target. What this means is that it transparently verifies the integrity of each device block as it is being read from disk. If the block checks out, the read succeeds, and if not -- the read generates an I/O error as if the block was physically corrupt. Under the hood dm-verity is implemented using a pre-calculated hash tree which includes the hashes of all device blocks. The leaf nodes of the tree include hashes of physical device blocks, while intermediate nodes are hashes of their child nodes (hashes of hashes). The root node is called the root hash and is based on all hashes in lower levels (see figure below). Thus a change even in a single device block will result in a change of the root hash. Therefore in order to verify a hash tree we only need to verify its root hash. At runtime dm-verity calculates the hash of each block when it is read and verifies it using the pre-calculated hash tree. Since reading data from a physical device is already a time consuming operation, the latency added by hashing and verification as relatively low.

[Image from Android dm-verity documentation,  licensed under Creative Commons Attribution 2.5]

Because dm-verity depends on a pre-calculated hash tree over all blocks of a device, the underlying device needs to be mounted read-only for verification to be possible. Most filesystems record mount times in their superblock or similar metadata, so even if no files are changed at runtime, block integrity checks will fail if the underlying block device is mounted read-write. This can be seen as a limitation, but it works well for devices or partitions that hold system files, which are only changed by OS updates. Any other change indicates either OS or disk corruption, or a malicious program that is trying to modify the OS or masquerade as a system file. dm-verity's read-only requirement also fits well with Android's security model, which only hosts application data on a read-write partition, and keeps OS files on the read-only system partition.

Android implementation

dm-verity was originally developed in order to implement verified boot in Chrome OS, and was integrated into the Linux kernel in version 3.4. It is enabled with the CONFIG_DM_VERITY kernel configuration item. Like Chrome OS, Android 4.4 also uses the kernel's dm-verity target, but the cryptographic verification of the root hash and mounting of verified partitions are implemented differently from Chrome OS.

The RSA public key used for verification is embedded in the boot partition under the verity_key filename and is used to verify the dm-verity mapping table. This mapping table holds the locations of the target device and the offset of the hash table, as well as the root hash and salt. The mapping table and its signature are part of the verity metablock which is written to disk directly after the last filesystem block of the target device. A partition is marked as verifiable by adding the verify flag to the Android-specific fs_mgr flags filed of the device's fstab file. When Android's filesystem manager encounters the verify flag in fstab, it loads the verity metadata from the block device specified in fstab and verifies its signature using the verity_key. If the signature check succeeds, the filesystem manager parses the dm-verity mapping table and passes it to the Linux device-mapper, which use the information contained in the mapping table in order to create a virtual dm-verity block device. This virtual block device is then mounted at the mount point specified in fstab in place of the corresponding physical device. As a result, all reads from the underlying physical device are transparently verified against the pre-generated hash tree. Modifying or adding files, or even remounting the partition in read-write mode, results in an integrity verification failure and an I/O error.

We must note that as dm-verity is a kernel feature, in order for the integrity protection it provides to be effective, the kernel the device boots needs to be trusted. On Android, this means verifying the boot partition, which also includes the root filesystem RAM disk (initrd) and the verity public key. This process is device-specific and is typically implemented in the device bootloader, usually by using an unmodifiable verification key stored in hardware to verify the boot partition's signature.

Enabling verified boot

The official documentation describes the steps required to enable verified boot on Android, but lacks concrete information about the actual tools and commands that are needed. In this section we show the commands required to create and sign a dm-verity hash table and demonstrate how to configure an Android device to use it. Here is a summary of the required steps: 
  1. Generate a hash tree for that system partition.
  2. Build a dm-verity table for that hash tree.
  3. Sign that dm-verity table to produce a table signature.
  4. Bundle the table signature and dm-verity table into verity metadata.
  5. Write the verity metadata and the hash tree to the system parition.
  6. Enable verified boot in the devices's fstab file.
As we mentioned earlier, dm-verity can only be used with a device or partition that is mounted read-only at runtime, such as Android's system partition. While verified boot can be applied to other read-only partition's such as those hosting proprietary firmware blobs, this example uses the system partition, as protecting OS files results in considerable device security benefits. 

A dm-verity hash tree is generated with the dedicated veritysetup program. veritysetup can operate directly on block devices or use filesystem images and write the hash table to a file. It is supposed to produce platform-independent output, but hash tables produced on desktop Linux didn't quite agree with Android, so for this example we'll generate the hash tree directly on the device. To do this we first need to compile veritysetup for Android. A project that generates a statically linked veritysetup binary is provided on Github. It uses the OpenSSL backend for hash calculations and has only been slightly modified (in a not too portable way...), to allow for the different size of the off_t data type, which is 32-bit in current versions of Android's bionic library. 

In order to add the hash tree directly to the system partition, we first need to make sure that there is enough space to hold the hash tree and the verity metadata block (32k) after the last filesystem block. As most devices typically use the whole system partition, you may need to modify the BOARD_SYSTEMIMAGE_PARTITION_SIZE value in your device's BoardConfig.mk to allow for storing verity data. After you have adjusted the size of the system partition, transfer the veritysetup binary to the cache or data partitions of the device, and boot a recovery that allows root shell access over ADB. To generate and write the hash tree to the device we use the veritysetup format command as shown below.

# veritysetup --debug --hash-offset 838893568 --data-blocks 204800 format \
/dev/block/mmcblk0p21 /dev/block/mmcblk0p21
...
# Updating VERITY header of size 512 on device /dev/block/mmcblk0p21, offset 838893568.
VERITY header information for /dev/block/mmcblk0p21
UUID:                 0dd970aa-3150-4c68-abcd-0b8286e6000
Hash type:            1
Data blocks:          204800
Data block size:      4096
Hash block size:      4096
Hash algorithm:       sha256
Salt:                 1f951588516c7e3eec3ba10796aa17935c0c917475f8992353ef2ba5c3f47bcb
Root hash:            5f061f591b51bf541ab9d89652ec543ba253f2ed9c8521ac61f1208267c3bfb1

This example was executed on a Nexus 4, make sure you use the correct block device for your phone instead of /dev/block/mmcblk0p21. The --hash-offset parameter is needed because we are writing the hash tree to the same device that holds filesystem data. It is specified in bytes (not blocks) and needs to point to a location after the verity metadata block. Adjust according to your filesystem size so that hash_offset > filesystem_size + 32k. The next parameter, --data-blocks, specifies the number of blocks used by the filesystem. The default block size is 4096, but you can specify a different size using the --data-block-size parameter. This value needs to match the size allocated to the filesystem with BOARD_SYSTEMIMAGE_PARTITION_SIZE. If the command succeeds it will output the calculated root hash and the salt value used, as shown above. Everything but the root hash is saved in the superblock (first block) of the hash table. Make sure you save the root hash, as it is required to complete the verity setup.

Once you have the root hash and salt, you can generate and sign the dm-verity table. The table is a single line that contains the name of the block device, block sizes, offsets, salt and root hash values. You can use the gentable.py script (edit constant values accordingly first) to generate it or write it manually based on the output of veritysetup. See dm-verity's documentation for details about the format. For our example it looks like this (single line, split for readability):

1 /dev/block/mmcblk0p21 /dev/block/mmcblk0p21 4096 4096 204800 204809 sha256 \
5f061f591b51bf541ab9d89652ec543ba253f2ed9c8521ac61f1208267c3bfb1 \
1f951588516c7e3eec3ba10796aa17935c0c917475f8992353ef2ba5c3f47bcb 

Next, generate a 2048-bit RSA key and sign the table using OpenSSL. You can use the command bellow or the sign.sh script on Github.

$ openssl dgst -sha1 -sign verity-key.pem -out table.sig table.bin

Once you have a signature you can generate the verity metadata block, which includes a magic number (0xb001b001) and the metadata format version, followed by the RSA PKCS#1.5 signature blob and table string, padded with zeros to 32k. You can generate the metadata block with the mkverity.py script by passing the signature and table files like this:

$ ./mkverity.py table.sig table.bin verity.bin

Next, write the generated verity.bin file to the system partition using dd  or a similar tool, right after the last filesystem block and before the start of the verity hash table. Using the same number of data blocks passed to veritysetup, the needed command (which also needs to be executed in recovery) becomes:

# dd if=verity.bin of=/dev/block/mmcblk0p21 bs=4096 seek=204800

Finally, you can check that the partition is properly formatted using the veritysetup verify command as shown below, where the last parameter is the root hash:

# veritysetup --debug --hash-offset 838893568 --data-blocks 204800 verify \
/dev/block/mmcblk0p21 /dev/block/mmcblk0p21 \
5f061f591b51bf541ab9d89652ec543ba253f2ed9c8521ac61f1208267c3bfb1

If verification succeeds, reboot the device and verify that the device boots without errors. If it does, you can proceed to the next step: add the verification key to the boot image and enable automatic integrity verification.

The RSA public key used for verification needs to be in mincrypt format (also used by the stock recovery when verifying OTA file signatures), which is a serialization of mincrypt's RSAPublicKey structure. The interesting thing about this structure is that ts doesn't simply include the modulus and public exponent values, but contains pre-computed values used by mincrypt's RSA implementation (based on Montgomery reduction). Therefore converting an OpenSSL RSA public key to mincrypt format requires some modular operations and is not simply a binary format conversion. You can convert the PEM key using the pem2mincrypt tool (conversion code shamelessly stolen from secure adb's implementation). Once you have converted the key, include it in the root of your boot image under the verity_key filename. The last step is to modify the device's fstab file in order to enable block integrity verification for the system partition. This is simply a matter of adding the verify flag, as shown below:

/dev/block/platform/msm_sdcc.1/by-name/system  /system  ext4  ro, barrier=1  wait,verify

Next, verify that your kernel configuration enable CONFIG_DM_VERITY, enable it if needed and build your boot image. Once you have boot.img, you can try booting the device with it using fastboot boot boot.img (without flashing it). If the hash table and verity metadata blcok have been generated and written correctly, the device should boot, and /system should be a mount of the automatically created device-mapper virtual device, as shown below. If the boot is successful, you can permanently flash the boot image to the device.

# mount|grep system
/dev/block/dm-0 /system ext4 ro,seclabel,relatime,data=ordered 0 0

Now any modifications to the system partition will result in read errors when reading the corresponding file(s). Unfortunately, system modifications by file-based OTA updates, which modify file blocks without updating verity metadata, will also invalidate the hash tree. As mentioned in the official documentation, in order to be compatible with dm-verity verified boot, OTA updates should also operate at the block level, ensuring that both file blocks and the hash tree and metadata are updated. This requires changing the current OTA update infrastructure, which is probably one of the reasons verified boot hasn't been deployed to production devices yet.

Summary

Android includes a verified boot implementation based on the dm-verity device-mapper target since version 4.4. dm-verity is enabled by adding a hash table and a signed metadata block to the system partition and specifying the verify flag in the device's fstab file. At boot time Android verifies the metadata signature and uses the included device-mapper table to create and mount a virtual block device at /system. As a result, all reads from /system are verified against the dm-verity hash tree, and any modification to the system partition results in I/O errors. 

Comments

Oud Player said…
Hello Nikolay,

As usual, your articles are very helpful and interesting. Thanks!
Just a simple question: Do you think that this code portion can be ported back to JB4.2? Both Android releases have the DM-MAPPER in the 3.4 Kernel release so I believe it can be.

Do you think so?

Regards,

Fabrice.
Nikolay Elenkov said…
Probably. Most of the work is done by the kernel, the user-level part just parses and verifies the verity metadata. You mostly need system/core/fs_mgr/fs_mgr_verity.c.
Oud Player said…
Thanks. I will probably try that.
Jay, Ma said…
Hello Nikolay,

I'm studying android. I would like to apply the dm-verity on Nexus 7.
So, I have added the dm-verity.c and related things in kernel (tegra-android-tegra3-grouper-3.1-kitkat-mr1) and I have made zImage file. :)
But, I don't have a idea after this. And, I found your articles.

1. How can I compile veritysetup for Android?

2. Nexus 7 only has mmcblk0p1 ~ mmcblk0p9 in /dev/block/. Which mmclbk should I use?

Thanks,
Jay
Nikolay Elenkov said…
Use the linked project (on Github) to compile veritysetup. You need to find the partition that is mounted at /system and write the hash table and metatablock there.
Jay, Ma said…
Thanks.
I have compiled veritysetup after drop in ./android/external/crytesetup. Is this right?
and I have checked about BOARD_SYSTEMIMAGE_PARTITION_SIZE := 681574400 in device/asus/grouper/BoardConfigCommon.mk file.
How and Where should I enter this veritysetup command? on linux
veritysetup --debug --hash-offset 681574400 --data-blocks 204800 format /dev/block/mmcblk0p21 /dev/block/mmcblk0p21

I try this command on ubuntu. but it return below error.
root@jay-VirtualBox:/dev/block# veritysetup --debug --hash-offset 681574400 --data-blocks 204800 format /dev/block/mmcblk0p10 /dev/block/mmcblk0p10
# cryptsetup 1.6.1 processing "veritysetup --debug --hash-offset 681574400 --data-blocks 204800 format /dev/block/mmcblk0p10 /dev/block/mmcblk0p10"
# Running command format.
# Created hash image /dev/block/mmcblk0p10.
# Allocating crypt device /dev/block/mmcblk0p10 context.
# Trying to open and read device /dev/block/mmcblk0p10.
# Initialising device-mapper backend library.
# Formatting device /dev/block/mmcblk0p10 as type VERITY.
# Crypto backend (gcrypt 1.5.3) initialized.
# Setting ciphertext data device to /dev/block/mmcblk0p10.
# Trying to open and read device /dev/block/mmcblk0p10.
Header detected but device /dev/block/mmcblk0p10 is too small.
# Releasing crypt device /dev/block/mmcblk0p10 context.
# Releasing device-mapper backend.
Command failed with code 22: Header detected but device /dev/block/mmcblk0p10 is too small.
Nikolay Elenkov said…
Run it on your Android device, in recovery mode. Do ready veritysetup's documentation before proceeding.
Jay, Ma said…
I have changed to recovery mode and I have tried several times. but I cant run it during recovery mode.
Your articles makes me being wishful but It's too hard to do with only your articles.
I'm sorry to have bothered you and I need more detail way.
Please help me. T T
Thanks.
Bushmakin Pavel said…
Thanks for the great guide,

but could you please explain in greater details part with adjusting system partition size.
If you change the value of BOARD_SYSTEMIMAGE_PARTITION_SIZE it would just extend partition size, but filesystem would still occupy full partition, thus there would be no free space for hash data on partition. Android build system's script responsible for creatation of system.img would create ext4 filesystem on a partition occuping all the space on it.
Nikolay Elenkov said…
You don't extend the partition, you make it smaller. That leaves some space after it for metadata and the hash tree.
Nikolay Elenkov said…
You need to copy the veritysetup binary to a parition you can access from recovery (/cache or /data). chmod +x if needed and execute, adjusting parameters as necessary. You can do a dry run first and write the hash tree to a file instead of to the system partition to make sure it works.
Jay, Ma said…
I have added the cryptsetup source in "./external" and compiled. After that I've download images on device.
cryptsetup is produced library file when I compiled the cryptsetup.
But I do not understand about "chmod +x".
Bushmakin Pavel said…
Yeah, that's probably it. I'm trying dm-verity on emulator device and its adjusting partition's size to match system.img size, so some other approach is needed in this case.
Brian said…
I've seen several comments about how several people have tried your program and techniques on older versions - non-JB4.3 (specific comments about 4.2 and ICS+). If I try to use a device with 4.2 or 4.1, I get a class not found error for KeyPairGeneratorSpec and KeyPairGeneratorSpec$Builder. The exception makes sense as this class wasn't added until 4.3.

How do people get this to work on older Android's? And did AndroidKeyStore not show up until 4.3 as well or is it hidden someone? If it isn't there, then it would seem fruitless to store things in hardware if none of the crypto functions can use them.
Nikolay Elenkov said…
I don't know what comment you are referring to, but it would be quite helpful to post your comments on the relevant blog post.

Assuming your questions is about credential storage (keystore), you can use it on older versions by connecting to the daemon directly, using the private API. The daemon has been there since 1.6, but got a lot of new functionality between 4.0-4.3. The public APIs where added in 4.3. See the sample code on Github for details.
Brian said…
Sorry - for some reason, when I try to post/comment on the keystore article, it always ends up on this post (which is your latest). I also can't comment/reply using Chrome either (having to switch over to FFox). I'll try to move over to the other post and create a better worded message/question.
Oud Player said…
Hello Nikolay,

Do you think that DM-Verity it self can be a subject to attacks? Can there be a need to protect it with trusted environment (like Trustzone for example)?

Regards,

Fabrice.
Oud Player said…
Its seems that it posted an old question of mine with the new one I posted today....strange...
Nikolay Elenkov said…
The integrity of the hash table depends on the root hash, which is signed as part of the dm-verity table. So in order to fake the hash table you need the private key. Of course, because the public key is include din the boot partition, you can compromise the whole system by replacing the key, or adding a compromised kernel that always returns OK when verifying sectors. That's why it's important to verify the boot partition before booting, with some key (root of trust) which is burned into hardware and unchangeable. Most devices have such a mechanism, if the bootloader is locked.
Oud Player said…
Thank you. So I understand that having a secured boot will definitely make DM-Verity safe. Maybe it is not the topic of your post but is there any intention to leverage the DM-Verity for run-time environment and not Boot only. I read on Android document that one can for example run the signature verification only when there is a Read/Write access on the file system. In run-time case (after the boot process) I believe that protecting the verification process may be relevant...
Nikolay Elenkov said…
Not sure what you mean? dm-verity is typically applied to /system and works at runtime too (blocks are verified when read), not only at boot. What do you mean by 'signature verification' and 'verification process'?
Oud Player said…
I thought DM-Verifier was only a boot verifier (same as what is used in Chromium OS). It seems that this link https://source.android.com/devices/tech/security/dm-verity.html only talks about DM-Verity on boot. Also according to the code I could see that fs_mgr_setup_verity() was only called from the mounting routines (fs_mgr_do_mount() & fs_mgr_mount_all()). In that case, how does DM-Verity protect the system in the run-time. I am sorry in advance if I miss something here. By the way, for me, the verification process is the comparison between a computed block HASH and a stored block HASH.
Nikolay Elenkov said…
dm-verity works at the kernel level and verifies each block as it is read, both in Android and Chrome. Maybe you should read the page you linked, as well as dm-verity's documentation again.
Oud Player said…
I will. Thanks.
Cecily Kirkland said…
hi, I am NOT a programmer,but a user of my android Note II..all of a sudden I cannot send any emails that originate from my phone without that requirement for an RSA code, key length 2048 and a password?? what password and how can I get this off!! I can reply to any email, but not originate one. Can you help me? krklnd@aol.com
Lungde Shyu said…
This comment has been removed by the author.
Lungde Shyu said…
Hello Nikolay,
I just found this page.
Your article is really useful for learning Android.
But what's wrong with the hash table generating on desktop Linux?
Nikolay Elenkov said…
It didn't verify properly on the Android device. This doesn't necessarily mean that a table can't be generated off device, maybe my setup was not correct, etc. If you manage to generate a proper table off device, do leave a comment about how you did it.
Lungde Shyu said…
Thanks, I just successfully running all these through by generating table and hash tree off device.
I think the main problem lies in how fastboot flashes the firmware.
Due to system.img is in sparse format, fastboot has the opportunity to explain how it will deal with those sparse areas.
Thus, I first unsparsed the system.img using simg2img by
$ simg2img system.img system.img.raw
Then I calculated hash tree and table as you did in your post and then concatenate them up, i.e., new_system_image = system.img.raw + table + hash_tree.

Btw, there are two typos in your post.
1.
>> 1 /dev/block/mmcblk0p21 /dev/block/mmcblk0p21 4096 4096 204800 204809 sha256 \
1f951588516c7e3eec3ba10796aa17935c0c917475f8992353ef2ba5c3f47bcb \
5f061f591b51bf541ab9d89652ec543ba253f2ed9c8521ac61f1208267c3bfb1
instead, it should be
1 /dev/block/mmcblk0p21 /dev/block/mmcblk0p21 4096 4096 204800 204809 sha256 \
5f061f591b51bf541ab9d89652ec543ba253f2ed9c8521ac61f1208267c3bfb1 \
1f951588516c7e3eec3ba10796aa17935c0c917475f8992353ef2ba5c3f47bcb
2.
>> veirtysetup verify command as shown below
veritysetup verify command
Nikolay Elenkov said…
Cool. I actually did unsparse the image before generating the hash tree, so maybe there was a different problem. What OS/architecture did you use to generate the hash tree?
Lungde Shyu said…
Ubuntu 12.04 64bits
Jack M Tresler said…
This comment has been removed by the author.
Jack M Tresler said…
Excellent post! Thanks much. 1) Do you know if there's any way as a consumer to verify boot.img as manufacturers can? 2) Could you use same method and verify Recovery partition?
Nikolay Elenkov said…
There is no standard signing format for raw images, AFAIK. Some manufacturer-specific tools are available though, so it could be done. OTA packages (ZIP) are signed, so they can be verified fairly easily.
Huan Zheng said…
Hi, i'm having problem convert "new_system_image = system.img.raw + table + hash_tree" back to sparsed system.img; because out/host/linux-x86/bin/make_ext4fs works against a mount point, which does not include the appended table+hash_tree. Any idea?

Besides, when generating hash tree directly on device, i also have a question: you must mount /system in rw mode, which allows you to append hash tree in /dev/block/mmcblk0p21; but after unmount in rw mode, shouldn't the super block of /system change? which could cause hash tree invalid.
Nikolay Elenkov said…
There is no need to convert to sparse, you can flash the inflated image.

If you work in recovery there is no need to remount r/w, when you boot Android it mounts r/o.
Eddy Sy said…
Maybe this isn't for my question, but i don't know where to go to get the answer.

I have an MTK device that running Jellybean, and I 've tried to upgrade to official Kitkat, but always end up with failure even with flasing or recovery...is the changed hardware configuration (power button bypassed) is the cause of my problem...please help me. Thanks
Oud Player said…
Hello Nikolay,

There are still things that are not clear to me:
How does DM-verity check the system partition at boot? Do they go over all the partition blocks? If this is the case, I don't understand how they solved the latency problem. Don't they have to calculate HASH for all the blocks anyway? Even if, as they explained, they are doing the processing in parallel (Read and Hash) devices are not multiprocessor based so the time is shared anyway. Don't you agree?

During run-time I do understand that only read blocks are checked, but again I do have some concerns about the boot process.

Fabrice.
Nikolay Elenkov said…
It doesn't, it only checks that the hash tree is properly signed. If it is, you can be reasonably sure that it has not been modified. Block are only checked as they are read from disk, but not when retrieved from the kernel disk cache. The time needed to calculate the SHA1 of a 4K block is small compared to the time required to read from disk (flash).

Calculating the hash of all block is only done when generating the hash tree (once) and is performed offline (either in recovery or when generating the filesystem image on the build machine).
Oud Player said…
Ok so according to what you say the process in run-time will be:

- Identify the block to be read.
- Calculate the SHA1 of that specific block
- Use the SHA1 result of that block to retrieve the root hash of the HASH tree.
- Abort the Read if the comparison between the root hashes does not match.

Is it ok?
Nikolay Elenkov said…
Something like that, except it doesn't directly use the root hash, it traverses the tree. Keep in mind that, all of this is done transparently by the kernel (via DeviceMapper), so you have no control over it though.
Oud Player said…
Thank you very much for your help!
vinit patel said…
Hi Nikolay

Thank you for the post. It is really helpful. :)

I tried compiling the veritysetup on android codebase, but I am seeing following error.

make: *** No rule to make target `out/target/product//obj/STATIC_LIBRARIES/libcrypto_static_intermediates/export_includes', needed by `out/target/product//obj/EXECUTABLES/veritysetup_intermediates/import_includes'. Stop.

Have you seen this error in the past and know of any workaround?

Thank you.
Vinit
Nikolay Elenkov said…
The program links OpenSSL statically so you need to build it beforehand. Build the whole platform first and then build veritysetup using mm (or change makefiles accordingly).
Evgeny Shtein said…
Hi, I am trying to run the DM-Verity on my device and to build the hash table ofline, but i have problems with it, Can you please share your script or explain how to build the hash tree offline. Thank you
I tried to run it and all the commands run with success except the verify.
i used system.img.raw instead of /dev/block/mmc... and the error i get in the end is
Command failed with code 22: Device system.img.raw is not a valid VERITY device.

do you know why it can happen
Lungde Shyu said…
The most probable reason is that your veritysetup is too old. You have to install cryptsetup >= 1.6.1. If you install from package manager, such as apt, make sure both cryptsetup-bin and libcryptsetup4 are both updated.
Evgeny Shtein said…
I downloaded the latest src from dm-verity site 1.6.6 and compiled it by myself,
do i need to verify something in my linux machine?
Lungde Shyu said…
If you did "make install", then you're fine. If you did not, be sure do ldd veritysetup, and check whether it links to the correct libcryptsetup.so.4.
Evgeny Shtein said…
i cant do make install, i am not root on the server.
i did ./configure and defined a local path
and then go to localpath/sbin and run this from here
how can i check if i am using the correct libcryptsetup.so.4
Lungde Shyu said…
When you do "ldd veritysetup", you should see it point to a libcryptsetup.so.4 that also lies in your local directory. If not, using LD_LIBRARY_PATH could be a good solution.
Evgeny Shtein said…
it lies in my local directory.
any other ideas?
can you share how you did it so that i will see what is wrong with my method?
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
after running ldd veritysetup
i see :
cryptsetup-1.6.6/sbin$ ldd veritysetup
linux-vdso.so.1 => (0x00007fffd96f4000)
libcryptsetup.so.4 => $(local_path)/cryptsetup-1.6.6/lib/libcryptsetup.so.4 (0x00007f8c0d555000)
libpopt.so.0 => /lib/x86_64-linux-gnu/libpopt.so.0 (0x00007f8c0d32d000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8c0cf6d000)
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f8c0cd68000)
libdevmapper.so.1.02.1 => /lib/libdevmapper.so.1.02.1 (0x00007f8c0cb45000)
libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x00007f8c0c8c6000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f8c0c6be000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8c0d77e000)
libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f8c0c49f000)
libudev.so.0 => /lib/x86_64-linux-gnu/libudev.so.0 (0x00007f8c0c291000)
libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007f8c0c08d000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f8c0be70000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f8c0bc6b000)

any other ideas?
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
This comment has been removed by the author.
Evgeny Shtein said…
this so is from my local directory also.
any other ideas?
Evgeny Shtein said…
Hi, i am still having problem with verify.
the so is local.
does it matter if i put system,img.raw name in the table instead /dav/block ...
thank you
Evgeny Shtein said…
Hi, i am still having problem with verify.
the so is local.
does it matter if i put system,img.raw name in the table instead /dav/block ...
thank you
Evgeny Shtein said…
I still have problems with verify and dump of the system image, even that the so is local.
any ideas?
Lungde Shyu said…
Are you trying to verify the image you generated? You should not do that. You can only verify the image ON device. Verifying image off device require mounting it as a loop back device and thus root access.
Evgeny Shtein said…
Stil having problem with verify and dump
the so is local,
maybe the problem is that i replace /dev/block to system.img.raw in all commands?
can you share your commands?
i am sure that my error is something small that i am missing.
thank you
Evgeny Shtein said…
Stil having problem with verify and dump
the so is local,
maybe the problem is that i replace /dev/block to system.img.raw in all commands?
can you share your commands?
i am sure that my error is something small that i am missing.
thank you
Evgeny Shtein said…
Stil having problem with verify and dump
the so is local,
maybe the problem is that i replace /dev/block to system.img.raw in all commands?
can you share your commands?
i am sure that my error is something small that i am missing.
thank you
Evgeny Shtein said…
This comment has been removed by the author.
Lungde Shyu said…
This is my script based on Nikoley's work

#!/usr/bin/env python

import struct
import sys
import os
import subprocess
import re

if len(sys.argv) != 2:
exit()

partition = sys.argv[1]
sparse_img = partition + '.img'
raw_img = partition + '.img.raw'
raw_img_hash = partition + '.img.raw.hash'
subprocess.call(['simg2img', sparse_img, raw_img])
subprocess.call(['mv', partition + '.img', partition + '.img.orig'])

# modify for device
DATA_DEV = '/dev/block/%s' % partition
DEVICE_SIZE_BYTES = os.stat(raw_img).st_size
SALT = ''
ROOT_DIGEST = ''
VERITY_SB_SIZE = 32 * 1024

#veritysetup
try:
subprocess.call(['veritysetup', '-v', 'format', raw_img, raw_img_hash], stdout=open('verity.result','w'))
except:
print "Error: Please install cryptsetup package >= 1.6.1!"
exit(-1)

for line in open('verity.result', 'r'):
if 'Salt' in line:
salt_re = re.compile('Salt:[ \t]*([a-z0-9]*)')
string = line.strip();
loc = re.match(salt_re, string)
SALT = loc.group(1)
elif 'Root hash' in line:
root_re = re.compile('Root hash:[ \t]*([a-z0-9]*)')
string = line.strip();
loc = re.match(root_re, string)
ROOT_DIGEST = loc.group(1)

# from veritysetup output
start = 0
size = DEVICE_SIZE_BYTES / 512
target = 'verity'
hash_dev = DATA_DEV
data_block_size = 4096
hash_block_size = data_block_size
num_blocks = DEVICE_SIZE_BYTES / data_block_size
hash_offset = (DEVICE_SIZE_BYTES + VERITY_SB_SIZE) / hash_block_size + 1
hash_alg = 'sha256'

#0 417792 verity 1 /dev/sdb /dev/sdc 4096 4096 52224 1 sha256 2a... f4...

fmt = "1 %s %s %d %d %d %d sha256 %s %s"
table = fmt % (DATA_DEV, hash_dev, data_block_size, hash_block_size, num_blocks, hash_offset, ROOT_DIGEST, SALT)

print "DM table: \n" + table

f = open('table.bin', 'wb')
f.write(buffer(table))
f.close()
Evgeny Shtein said…
This comment has been removed by the author.
Simone Mutti said…
Hi Nikolay

Thank you for the post, really cool :)
I tried to compile the 'pem2mincrypt' but I get several 'undefined references', all of them related to the openssl library (the library is installed correctly). Is there something special to do in order to compile this tool?

Best,
Simone
Nikolay Elenkov said…
Depending on where the headers and library are installed, you might need to adjust CFLAGS and LDFLAGS in the Makefile. Add -L/usr/local/lib, etc., depending on where the library is installed.
Guoyin Chen said…
This comment has been removed by the author.
Nikolay Elenkov said…
Please add a new comment, do not simply reply to unrelated one. You need to generate keys using OpenSSL, not ssh-keygen.
Guoyin Chen said…
Hello Nikolay&Lungde Shyu, how you convert the publick key by pem2mincrypt? I always failed with below commands. Anything wrong.
~/dm-verity$ ssh-keygen -t rsa -b 2048 -f ~/dm-verity/verity-key
~/dm-verity$ ssh-keygen -f verity-key.pub -e -m pem > verity-key-pub.pem
~/dm-verity$ pem2mincrypt verity-key-pub.pem mincrypt-verity-key
Converting PEM key 'verity-key-pub.pem' to mincrypt publickey 'mincrypt-verity-key'...
read_key 'verity-key-pub.pem'
Failed to read key
Error reading key: 0
Nikolay Elenkov said…
Use `openssl genrsa` to generate a key and do not set a password on the key. There are multiple articles on how to do this.
Evgeny Shtein said…
I have problems with the blog, i cant see my comments this is why i posted so many of them,
Sorry.
I understood that i cant verify off device. trying to do it on device.
I there a way to convert the raw image to sparsed image again, so that i will be able to flash using fastboot?
is is img2simg ?
Nikolay Elenkov said…
You need to press the 'Load more' at the bottom of the page to see all comments. There is no need to convert back to sparsed, you can flash the raw image with fastboot.
Evgeny Shtein said…
thank you,
but i would like to sparse the image to take less space.
the question is it possible?
Evgeny Shtein said…
also, i am trying to verify on the device.
i geting an error:
# cryptsetup 1.3.1 processing "./veritysetup_and --debug --hash-offset 528515072 --data-blocks 129024 verify /dev/block/platform/msm_sdcc.1/by-name/system /dev/block/platform/msm_sdcc.1/by-name/system d6219460270c2d4b81b817a1e6fe61a2dc404ceabf8a8cd837e4934c64a6ec6a"
# Running command verify.
# Allocating crypt device /dev/block/platform/msm_sdcc.1/by-name/system context.
# Trying to open and read device /dev/block/platform/msm_sdcc.1/by-name/system.
# Initialising device-mapper backend library.
# Trying to load VERITY crypt type from device /dev/block/platform/msm_sdcc.1/by-name/system.
# Crypto backend (OpenSSL 1.0.1e 11 Feb 2013) initialized.
# Reading VERITY header of size 512 on device /dev/block/platform/msm_sdcc.1/by-name/system, offset 528515072.
# Setting ciphertext data device to /dev/block/platform/msm_sdcc.1/by-name/system.
# Trying to open and read device /dev/block/platform/msm_sdcc.1/by-name/system.
# device_size: /dev/block/platform/msm_sdcc.1/by-name/system
# ioctl size: 536870912
# Activating volume [none] by volume key.
# *** VERITY_activate
# Trying to activate VERITY device [none] using hash sha256.
# Verification of data in userspace required.
# VERITY_create_or_verify_hash
# verify: 1
# hash_name: sha256
# data_device: /dev/block/platform/msm_sdcc.1/by-name/system
# hash_block_size: 4096
# data_block_size: 4096
# data_blocks: 129024
# hash_device: /dev/block/platform/msm_sdcc.1/by-name/system
# offset: 129033
# Hash verification sha256, data device /dev/block/platform/msm_sdcc.1/by-name/system, data blocks 129024, hash_device /dev/block/platform/msm_sdcc.1/by-name/system, offset 129033.
# Using 3 hash levels.
# Data device size required: 528482304 bytes.
# Hash device size required: 532684800 bytes.
Verification failed at position 0.
Verification of data area failed.
# Releasing crypt device /dev/block/platform/msm_sdcc.1/by-name/system context.
# Releasing device-mapper backend.
Command failed with code 1: Verification of data area failed.

do you know why?
peter zhang said…
This comment has been removed by the author.
fckcco said…
Hi Nikolay

I am trying to implement DM-verity on my device follow your step, but I'm not clear on something.
I'd like to ask you about that.

1. wasted space
following is part of the log for generate a hast tree

# # ioctl size: 4563402752
# # params->data_size: 512000
# # hash size: 32
# # VERITY_create_or_verify_hash
# # verify: 0
# # hash_name: sha256
# # data_device: /dev/block/mmcblk0p13
# # hash_block_size: 4096
# # data_block_size: 4096
# # data_blocks: 512000
# # hash_device: /dev/block/mmcblk0p13
# # offset: 512009
# # Hash creation sha256, data device /dev/block/mmcblk0p13, data blocks 512000, hash_device /dev/block/mmcblk0p13, offset 512009.
# # Using 3 hash levels.
# # Data device size required: 2097152000 bytes.
# # Hash device size required: 2113708032 bytes.

If my original system partition is about 2gb, I need increase it to more than twice the size of system partition. It is very waste on mobile device.

2. what is the correct hash size??
When I try to use veritysetup to verify it, I always get "Invalid root hast string specified."
After calculate, crypt_hex_to_bytes(root_hast, &root_hash_bytes, 0) != hash_size.
What should be the correct size?
Nikolay Elenkov said…
I think you are reading this wrong. If data and hash tree are on the same device, you need to have 2113708032 bytes on the partition, of which you can use 2097152000 bytes for data. That means that the hash tree will be about 16MB. You need to adjust the size of your system partition, so that 16MB are left on the partition after the file system ends.

The root hash is calculated using SHA-256, which is 32 bytes, or a 64 character hex string. Look at the examples in the post, as well as other comments for hints.
Oud Player said…
Hi Nikolay,

I tried to look for documentation on DM-Verity in Release L but unsuccessful.
Generally speaking, do you know how does DM-Verity progress? Is it integrated in platforms? Has OTA issue been solved?
For Relase L, is DM-Verity still there? Any changes of this module?

Regards,

Fabrice.
Nikolay Elenkov said…
I have exactly as much information about this as you do :) It doesn't seem to be enabled in current preview releases, you'll have to wait for the final release and/or the AOSP code for more details. If it gets enabled in Lollipop, it could be only on new devices (Nexus 6, etc.), but we'll have to wait and see.
oud player said…
Hi Nikolay,

Go back to KitKat, do you know to estimate the performance overhead? I believe that if it is triggered to each read, process there should be a latency impact right?

Regards,

Fabrice.
Nikolay Elenkov said…
Probably negligible compared to the actual read time. Also cached blocks are not re-verified, so shouldn't be noticeable in practice. You should benchmark on real hardware to be sure though.
Hi Nikolay

Great post - thanks! I tried following it on Nexus 7 2012 wifi/aosp 4.4.4r1 and everything went pretty well until rebooting for real. Then I got "device-mapper: table: 254:0: verity: unknown target type" in dmesg. Does this suggest that the kernel was built without dm-verity?

Downloading the kernel from https://android.googlesource.com/kernel/tegra/ shows no sign of verity support, configured in or not. Am I doing something wrong? Any suggestions gratefully received!

Cheers,
Lane
Nikolay Elenkov said…
Sounds like it. The kernel needs to have CONFIG_DM_VERITY set. IIRC, KitKat kernels don't have it set. You'll need to build your own kernel. Or build from Lollipop, its much easier there, because the build system has tools and makefiles that generate the verity hash tree and table for you.
That makes sense. Thanks for the information!
Quick update - just realised that all the tegra3-based devices are still on 3.1 kernels, whereas dm-verity wasn't introduced until 3.4. Now found myself a 3.4-based Nexus 7 2013!
Evgeny Shtein said…
Hi,
Nickolay I wanted to say thank you,
also i bought your book and it is great.

i have a question about the mount part.
after i flash the system+table image and reboot the device
i did a test: dd if=/dev/.../system of=/data/sys.dev
and i binary compared it to the original image i flashed.
i saw a difference in 2 bytes
isnt it a problem?
i also tried to load system with flag "noload", but no luck (was even worse).
do you know a solution for this ( KK build).
thank you

Evgeny Shtein said…
Hi,
I have another question.
If i will enable this feature i wont be able to mount system and data in recovery scripts, because they will e loaded as rw and will be changed.

if i need to runs some script/app in recovery mode using shell,
do you have a suggestion how to do it?
thank you
Evgeny Shtein said…
had a lot of issues compiling this.
after some digging and modifications this is what helped:
hope it will help someone

CC = gcc
CFLAGS = -I /usr/include/ -L /usr/lib/x86_64-linux-gnu/
LDFLAGS = -ldl -lcrypto
DEPS =
OBJ = pem2mincrypt.o
EXE = pem2mincrypt

%.o: %.c $(DEPS)
$(CC) -c -o $@ $< $(CFLAGS)

$(EXE): $(OBJ)
gcc $(OBJ) $(CFLAGS) $(LDFLAGS) -o $(EXE)

.PHONY: clean

clean:
rm $(OBJ) $(EXE)

Nikolay Elenkov said…
Recovery typically doesn't mount system as rw, so it shouldn't be a problem. Not sure what recovery you are using, but try TWRP.
Nikolay Elenkov said…
Not sure where the difference comes from, but make sure you never mount system as rw after flashing. Even if you don't modify anything, the superblock will get updated and you will get read errors. Might be related to your other question, maybe try another recovery?
Nikolay Elenkov said…
Yes, the Makefile doesn't even try to be portable, you need to adjust it for your platform. You might want to try the verity tools in Lollipop source, they are much more flexible and better integrated into the platform build.
Evgeny Shtein said…
the recovery is not related to this,
I will try to verify the image and see if i get errors.
i will update you if those 2 blocks make a different.
Evgeny Shtein said…
if you want to mount(system) in updater-script you dont have the option to select read only or not.
and i couldnt see in our recovery where i can force to ro always.
i only found it in the updater-bin code.
the problem is that after i do a mount(system)
mount(data)
the partition is changed (as you stated also because it is mounted as rw).
what would you suggest if i cant change the recovery i am using (AOSP) and i have to run something from ota.zip.
Nikolay Elenkov said…
Not sure what exactly you are trying to do, but I don't think you can migrate to dm-verity from an OTA file without using block-based updates (as Lollipop uses). Flash the whole partition via fastboot and then reboot to Android, which uses ro only. Generally, have a look at Lollipop code, it handles this much better, and you should be able to backport the tools if you really need to work with KitKat.
Evgeny Shtein said…
I enabled the verify flag and the system booted and mounted exactly as you wrote in the blog. thank you.
also i pulled the images using dd
and saw that it is exacly (binary compare) the same as flashed.
Evgeny Shtein said…
I want to share several things:
1. you are able to move to dm verity from ota this way:
a. Include in the ota zip the boot.img, recovery.img, system.raw(system+table)
b. Write all of them using parse_exact_file() command to their appropriate block
2. I will try to explain what I am trying to do.
In my ota.zip I need to upgrade tha system,boot,recovery.. and also to run an application that upgrade one of my peripherals.
This is done by adding an executional to ota.zip and copy it to /data/temp.
After that I need to run it.
In order to run an application I need to mount system because it is not mounted on start, I can see it in the logs an using adb ( I dont have adb shell), and this is good (securitiwise).
If I don’t mount system partition I don’t have shell (/system/bin/sh)
so the commands I run

mount(/system)
mount(/data)
copy(app, /data/app)
run(/data/app)
unmount(/data)
unmount(/system)

the problem is that when I do mount(/system) it automatically mount it as rw.
I cant find a place in recovery it self to forbid it and force to mount( if system then ro)

Maybe some one know about a place to modify it.
I saw this problem as well. Your instructions work great on device, but if I create the hash table in a file, cat the unsparsed system image, metadata and hash table on PC and fastboot flash the whole thing, veritysetup verify says verification fails at position 0. Could fastboot be setting some kind of timestamp in the superblock or something?
In fact, just running veritysetup format with the same salt each time, gives the same root hash until you reflash (the same) system.img, then it changes...
Evgeny Shtein said…
when are you running the test? in recovery or after android booted
In recovery (TWRP), just switching directly between that and bootloader mode to reflash. If I'm reading it right then the difference are at 0x42c, which seems to be 0xc into the super block, which is free block count. Maybe I need to erase the partition before I flash... (That's the next experiement ;-)
Evgeny Shtein said…
Do you have any idea how is it possible to test this feature?

I am able to flash the system image,
the build is engineering but still i am unable to do remount to system.
i want to copy or create something in the system partition and see that it is realy blocking.

have anybode done it?
thank you.
TWRP recovery (http://teamw.in/project/twrp2) will let you mount the system partition rw and adb shell in to create files, etc. Reboot, watch it fail, reboot to recovery and "adb shell cat /proc/last_kmsg" to check it was verity that prevented booting.
I'm being stupid - 0x42c is 0x2c into the superblock - mount time! I wonder if it's booting into recovery that is causing it to change...
Evgeny Shtein said…
you dont need special recovery to mount rw
you just run a script that does mount(system) and exit
the problem is that if i mount rw it changes the superblock and i am not able to test the modification per block (adding su for example).
lanecroberts said…
Oh, right, okay, I'm with you. Could you loop mount the system.img on your PC, update it and reflash it? If you can't figure out how to modify the system partition without verity failing then it sounds like you're done ;-)
Nikolay Elenkov said…
You could take a copy of the superblock, mount rw, install su, unmount and then restore the superblock to the raw partition. You should be able to loop-mount and specify an alternative superblock with sb=xxx, which *should* leave the original superblock unmodified.
Evgeny Shtein said…
thank you, tha tis a good idea, i will try it.
where is the superblock? is it the first 4096 bytes of hash table?
Evgeny Shtein said…
Hi,
I can see that in the scripts of android 5.0 when you creating the verity table they use
data_blocks + (METADATA_SIZE / BLOCK_SIZE)
while you in your scripts used:
data_blocks + (METADATA_SIZE / BLOCK_SIZE) +1
why is it different?
Evgeny Shtein said…
do i have to sign the table.bin file using sha1?
can i run:
openssl dgst -sha1 -sign verity-key.pem -out table.sig table.bin
or will the kernel not recognize this signature?
thank you
Hey Nikolay.

On my tablet, and I assume most of them, the /system partition is MUCH larger than the filesystem on it. When you say to resize the partition, that's what you mean, right? Not the filesystem?

Assuming that's correct. How do I do that when my /system partition is formatted with EXT4? I tried to use parted, but it can't handle EXT4.

I changed the value to my BoardConfig.mk, rebuilt everything, and flashed the images to the device, but the partition size didn't change.

What am I doing wrong?

Thanks!
Nikolay Elenkov said…
Sorry, filesystem size is correct. It's called 'device size' in the Android source code, but it is actually the filesystem size. It use the EXT4 superblock to determine filesystem size. You shouldn't need to change partition size, in fact you could damage your device by doing so.
You say:
"The default block size is 4096, but you can specify a different size using the --data-block-size parameter. This value needs to match the size allocated to the filesystem with BOARD_SYSTEMIMAGE_PARTITION_SIZE."

Do you mean BOARD_FLASH_BLOCK_SIZE?
I really need your help Nikolay. (Lots of into to follow). I'm trying to install Verity on my tablet, I've followed the directions, but I HAVE to be doing something wrong here.

BOARD_SYSTEMIMAGE_PARTITION_SIZE = 692060160
BOARD_FLASH_BLOCK_SIZE = 4096
data-blocks = 168960 (BOARD_SYSTEMIMAGE_PARTITION_SIZE / BOARD_FLASH_BLOCK_SIZE)
hash-offset = 692092928 (BOARD_SYSTEMIMAGE_PARTITION_SIZE + 32K for metadata)

I rebooted to recovery, so that I could have full access to the devices. I then executed this command:
/cache/veritysetup --debug --hash-offset 692092928 --data-blocks 168960 format /dev/block/mmcblk0p8 /dev/block/mmcblk0p8

Which produces this output (truncated):

# cryptsetup 1.3.1 processing "/cache/veritysetup --debug --hash-offset 692092928 --data-blocks 168960 format /dev/block/mmcblk0p8 /dev/block/mmcblk0p8"
# Running command format.
# device_size: /dev/block/mmcblk0p8
# ioctl size: 838860800
# params->data_size: 168960
# data_device: /dev/block/mmcblk0p8
# hash_block_size: 4096
# data_block_size: 4096
# data_blocks: 168960
# hash_device: /dev/block/mmcblk0p8
# offset: 168969
# Hash creation sha256, data device /dev/block/mmcblk0p8, data blocks 168960, hash_device /dev/block/mmcblk0p8, offset 168969.
# Using 3 hash levels.
# Data device size required: 692060160 bytes.
# Hash device size required: 697552896 bytes.
# Updating VERITY header of size 512 on device /dev/block/mmcblk0p8, offset 692092928.
VERITY header information for /dev/block/mmcblk0p8
UUID: 14820f9e-2f11-4df3-a2c1-bb55df82d8e9
Hash type: 1
Data blocks: 168960
Data block size: 4096
Hash block size: 4096
Hash algorithm: sha256
Salt: fa1ac4e1478ad30d54d2b4184c4d3efebb6970802398e9f29cff98feac1f58ca
Root hash: 184077725dea06233f0258c2c6d96d0ada006748c9fdf7d845690f6911ca2d87
# Releasing crypt device /dev/block/mmcblk0p8 context.
# Releasing device-mapper backend.
Command successful.

I rebooted the tablet to the OS, and then rebooted back to recovery where I ran this command:

/cache/veritysetup --debug --hash-offset 692092928 --data-blocks 168960 verify /dev/block/mmcblk0p8 /dev/block/mmcblk0p8 184077725dea06233f0258c2c6d96d0ada006748c9fdf7d845690f6911ca2d87

Which generated THIS output(truncated):

# cryptsetup 1.3.1 processing "/cache/veritysetup --debug --hash-offset 692092928 --data-blocks 168960 verify /dev/block/mmcblk0p8 /dev/block/mmcblk0p8 184077725dea06233f0258c2c6d96d0ada006748c9fdf7d845690f6911ca2d87"
# Trying to activate VERITY device [none] using hash sha256.
# Verification of data in userspace required.
# VERITY_create_or_verify_hash
# verify: 1
# hash_name: sha256
# data_device: /dev/block/mmcblk0p8
# hash_block_size: 4096
# data_block_size: 4096
# data_blocks: 168960
# hash_device: /dev/block/mmcblk0p8
# offset: 168969
# Hash verification sha256, data device /dev/block/mmcblk0p8, data blocks 168960, hash_device /dev/block/mmcblk0p8, offset 168969.
# Using 3 hash levels.
# Data device size required: 692060160 bytes.
# Hash device size required: 697552896 bytes.
Verification failed at position 0.
Verification of data area failed.
# Releasing crypt device /dev/block/mmcblk0p8 context.
# Releasing device-mapper backend.
Command failed with code 1: Verification of data area failed.

What am I doing wrong? Any idea?
I found it. For anyone reading this in the future who is getting the same error. I had screwed up the blocks calculation for the "dd verity.bin (the metadata block), to the partition" step. I was overwriting the start of the hash table with the metadata. So, OF COURSE it failed.

You see where I said "--data-blocks 168960" above? I was passing "seek=168969" to the dd command. STUPID!
Nikolay Elenkov said…
Glad you got it working. It's quite a complex procedure if you do it manually, but fortunately it is integrated in the Lollipop build system. If you want to use it, I recommend building a Lollipop ROM (AOSP or CyanogenMod) from source instead of trying it to configure it 'by hand'.
Lee Ji-Eun said…
This comment has been removed by the author.
martin said…
Hi, I would like to do the same on my tablet (Nexus 7) but there is no how to on the internet.
How can I do it without compiling your tool? (I cannot root my device to push the binary and modifiy the system partition)
I used the simg2img tool to use inflated image.
May then proceed as written in your post?
madhat1 said…
Hi Nikolay,

I'm trying to get dm-verity status from adb shell using veritysetup, but all the devices I'm passing result in:
$ /dev/mapper: opendir failed: No such file or directory
$ Device /dev/block/mmcblk0p30 not found

I'm trying it with: /dev/dm-0, /dev/device-mapper, /system and /dev/block/mmcblk0p30 - all result with the same error.

The command line is:
$ veritysetup --debug --verbose status /dev/block/mmcblk0p30

What am I doing wrong?
Pankaj Kushwaha said…
Hi,

Suppose I have a device with secure boot enabled in it. I unlock its boot-loader and flash a custom ROM, removing verify flash in fstab and DM_VERITY flash in kernel image as well. In that case will I be able to flash my Custom ROM or not ?

Thanks
Pankaj Kushwaha
Unknown said…
This comment has been removed by the author.
Master said…
HI sir

plz can you share with us the binary file after build for veritysetup plz.
Master said…
hi sir

plz if can you share with us the file of veritysetup binary ,we want to use it and try this.

Popular posts from this blog

Decrypting Android M adopted storage

Password storage in Android M

Unpacking Android backups