Reading files from Android SD-cards
For the last few years, there have been two modes you can format a SD card in Android. You can format it as “internal storage” or as “portable storage”. Internal storage means that it works like the built in storage, and that you can move any apps to the SD card, and also store data from any app on it.
But this comes with a cost. The data on the SD card will be encrypted, which means that you can no longer just pop out the SD card, move it to a computer and make backups, copy data etc. off of the card.
It also means that if your SD card gets corrupted, you don’t really have an easy way to recover anything. After this happened to a 400 GB SD card with lots of appdata I did not want to lose, I started looking for solutions, and came by this old blog, which saved the day.
While the info there was fine, it was old, and is missing some details, so I have taken the liberty of adding a few extra bits here.
The first part of the blog is quite correct. But I did things slightly different.
You do need to have a rooted device for this to work!
First of all, I removed the corrupt SD card from the Android device, and inserted it into a linux computer.
I then installed “adb” (part of the “android-tools” package, here at least) on the PC. I then went into developer mode under settings -> system on the Android device and enabled root access from ADB.
I could then read the encrytion key needed directly from adb:
$ adb root
restarting adbd as root
$ adb shell
X704L:/ # cd /data/misc/vold
X704L:/data/misc/vold # ls -l-rw------- 1 root root 16 expand_8838e738a18746b6e435bb0d04c15ccd.key
To read the file, I used the same command as in the original blog
# od -t x1 expand_8838e738a18746b6e435bb0d04c15ccd.key
0000000 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f
0000020I copied the content to a safe place, and disconnected from adb
Back on the Linux computer where I had the SD card inserted, I ran the command:
# dmsetup create crypt1 --table "0 `blockdev --getsize /dev/mmcblk0p2` crypt \
aes-cbc-essiv:sha256 00010203040506070809010a0b0c0d0e0f 0 /dev/mmcblk0p2 0"The long string of numbers in there is the key from the previous step, with spaces removed.
I am just interested in the second partition on the SD Card.
Now, this gave me a nice /dev/mapper/crypt1 but I was unable to mount it doe to the corruption, and also the blog (which is getting old) states that it is a ext4 volume, but it turns out that the file system is actually “f2fs”, not ext4.
# mount -t f2fs /dev/mapper/crypt1 /mnt/
mount: /mnt: mount(2) system call failed: Structure needs cleaning.Next thing I did was that I wanted to secure as much of the content from the SD card as possible. I did not have 400 GB available on the internal HD if my laptop, but I had an old external HD laying around. So I connected that HD to the laptop, mounted it on /wd and used ddrescue to dump the contents of the SD-card to the external HD.
# ddrescue -d /dev/mapper/crypt1 /wd/sd-dump.img /wd/sd-dump.logIt took a few hours, but gave me a disk-image to work with instead of touching the bad SD card more than necessary.
I then made a copy of the dump, so I even had a backup of that before going further, in case some of the later manipulations of the image should destroy it.
# cp /wd/sd-dump.img /wd/sd-dump-backup.imgThen I tried to mount the dumped image, but that still failed.
# mount -o loop /wd/sd-dump.img /mnt/
mount: /mnt: mount(2) system call failed: Structure needs cleaning.To fix this, I had to install f2fs-tools. That gave me fsck.f2fs and dump.f2fs
# fsck.f2fs sd-dump.imgAfter running for a while spewing out errors, it asked me if I wanted to fix the partition. I answered “Yes” to that question…
Then I was finally able to mount the image:
# mount -o loop sd-dump.img /mnt/
# ls /mnt/
app local media misc user user_deSuccess!
So now I needed to copy all the content from the old SD card to a new card.
First I inserted a new card in the device, formatted it as internal storage, and then connected the device to the computer using USB again.
The “adb push” command only pushes files, and does not create subdirs, so to copy all the old content you first need to find the correct path on the new SD-card, then create the entire directory tree, and finally push all the files.
Start by doing another adb root / adb shell to connect to the device.
$ adb root
restarting adbd as root
$ adb shell
X704L:/ #By using a simple “df” command while in the adb shell you can easily find the path to the mounted SD card:
X704L:/ # df(...)
/dev/block/dm-1 477G 3.9G 473G 1% /mnt/expand/6fe8c926-fda9-476d-822c-f7d5856d1847
So the SD card is mounted on /mnt/expand/6fe8c926-fda9–476d-822c-f7d5856d1847
Next, we recreate the old directory structure. Exit from the adb shell and run the following on your computer:
# cd /mnt
# find . -type d -exec adb shell mkdir \"/mnt/expand/6fe8c926-fda9-476d-822c-f7d5856d1847/{}\" \;This will recursively create all the subdirs from the old card on the new card. Make sure you have the escaped quotes around the path, so directories with spaces in their name are created correctly.
And finally push all the files:
# cd /
# adb push mnt/. /mnt/expand/6fe8c926-fda9-476d-822c-f7d5856d1847After a few hours of copying data, it finished successfully.
Then you have to to quite a lot of chown-ing. Android uses a lot of internal users. Basically every ap runs as its own user, so in your mountpoint of the old SD card, you see a lot of UIDs when doing a ls -l
drwx------. 10 1010131 1010131 3488 nov. 28 2019 com.arrowstar.FunnyFoodsLite
drwx------. 6 1010133 1010133 3488 aug. 22 2019 com.bimiboo.adventure
drwx------. 10 1010132 1010132 3488 sep. 17 2019 com.binibambini.MiniABCLite
drwx------. 10 1010144 1010144 3488 sep. 26 20:01 com.gameloft.android.ANMP.GloftDMHMAgain, you need to adb root and adb shell. Go into the directory structure you jut pushed the files to and do a whole lot of….
chown -R 1010131:1010131 com.arrowstar.FunnyFoodsLite/
chown -R 1010134:1010134 com.jingle.kidslearnabc/Etc. etc. Your paths, UIDs and GIDs will be different of course…
Finally I went into Settings — Storage on the device, removed the old SD card, and rebooted.
And we have successfully rescued all the data from the old, encrypted partition and all apps and data are available on the new SD card.
The only thing left to do now is to set up a periodic backup…
Sources:
https://nelenkov.blogspot.com/2015/06/decrypting-android-m-adopted-storage.html
https://www.technibble.com/guide-using-ddrescue-recover-data/
https://stackoverflow.com/questions/13145872/android-recursive-copy-with-adb-push