HTC Desire: Rooting, Custom ROM and A2SD

I spent a few days to flash a custom ROM to my HTC Desire. The experience was… fabulous. I thought I’d write a diary about it. So here it is.

I’m a .Net developer for many years, and have been doing Android apps development for the past year. I’m new to custom ROM thou, I could be wrong about many things below, feel free to correct me if you see any.

The Pathetic 150Mb

Just 3 months after I bought my HTC Desire I already encountered the famous low memory problem. The labeled 512Mb internal memory has a few partitions; practically there are only about 150Mb for user applications. This ridiculous 150Mb allocated for millions of applications on the Android market is just pathetic. It must be a practical joke by the guys at HTC. I had only less than 50 apps and available space was less then 15Mb. I had to keep uninstalling old apps to install new apps.

Froyo A2SD

Lucky enough HTC was pushing the Android 2.2 (Froyo) OTA update just around that time. I’ve been reading articles online about the new features in this version, including the ability to allow users to move applications to SD card instead of storing them inside internal memory. The community now called this feature “Froyo A2SD”. Cool, it sounded like a solution.

But it’s not. In many cases, an app cannot be moved. To name a few of them,

• App developers haven’t enabled their apps.
• Apps that were bundled together with the OS (called the “Stock Apps”) cannot be moved to SD card.
• App updates to the stock apps, such as Adobe Flash Player update, cannot be moved to SD card.

So I was only able to free up like 20Mb of space. The only solution I had was: try not to visit the Android Market so often. Did I mention how pathetic it was?

Force Froyo A2SD

A few months later Google released the Google Maps 5.0 update. This compass-integrated map was cool enough for me to sacrifice other apps to free up some space for it. It took 10Mb. Then there was Adobe Flash Player 10.1 update, another 10Mb. Then there was Skype, 5Mb even after Froyo A2SD. And the list goes on.

No, there must be something I can do. So I did some googling, and found a way to force-enable Froyo A2SD for all apps. It was simple enough. Connect phone to PC, fire up ADB (which I already installed before for apps development), type a line of command and done. After that I was able to move more apps to SD card including those that haven’t enabled by the developers.

Here what I did: http://myhtcdesire.com/tutorials/how-to-force-apps-to-sd-without-rooting-on-froyo

That helps, but again not much. Very soon I ran into the same problem again. I was starting to give up. Even if I can free up the entire 150Mb it’s still not quite enough for my usage. I mean, can you blame me? Apps are meant to be downloaded! We’re talking about Terabytes in desktops and suddenly I’m being calculative with Megabytes. I was this close of throwing away the phone!

Rooting

I’ve read that after “rooting” the phone I’ll have more options in managing apps. I didn’t quite understand what those “options” mean, but since I’m prepared to throw it away I might as well give it a try. Of cause, by doing so, I voided the warranty of the 6-month-old phone.

Apparently when an Android phone ships to user, user only has some sort of “guest” access to the Android system, similar to Windows’ Guest users. “Rooting” a phone refers to gaining “super user” (or administrator in Windows) access.

After some survey reading, I’ve done it using Unrevoked3. It’s not as difficult as it looks, I did it without a glitch. Credits to the Unrevoked3 team.

Here’s what I did: http://myhtcdesire.com/modding/how-to-root-your-desire-easily-method-2

Then I was able to run some apps that require root access, such as ShootMe to take screenshot (impossible before root without connecting phone to PC), Root Manager to browse previously inaccessible system files, do a much higher quality call recordings, etc. I was finally able to see the system files created by the apps I developed before, cool. These are bonus I’d consider, but the memory problem remains.

Then I bought Titanium Backup (for $5.99), which has some pretty cool features that would help me clean up internal memory. These including the Force Froyo A2SD, integrate stock app updates (merge the original copy and the update copy to save space), and uninstall unused stock apps. So I tried these, but as soon as I hit the button my phone rebooted, and nothing happened. What’s wrong?

S-Off

Google it, what else. Apparently HTC’s devices have a security feature that protects the certain portion (NAND) of the device, it cannot be written even with root access. The solution is to turn the security off, which the community called “S-Off”.

I first tried Unrevoked Forever. Looks simple. I was suppose to download a zip file, put it into SD card, then reboot phone in recovery mode, then locate the zip file… but hey the file is not shown in the list?

Here’s what I did: http://theunlockr.com/2010/09/20/how-to-root-the-htc-desire-unrevoked-method/
(Section III. It DID NOT work for me.)

Some more googling and then I found AlphaRev. The steps are more complicated. I downloaded the Linux-based ISO image, mount it into VirtualBox, then emulate my phone’s USB connection to the VirtualBox, then wait for it to reboot a few times in the next 15 minutes, and then… done. I have S-Off.

Here’s what I did: http://forum.xda-developers.com/showthread.php?t=805811

So I retried the features in Titanium Backup, and they were successful. I managed to free up 50Mb of internal memory space. Phew.

Symlink

But I’m still not satisfied. Since I have the root access I did a in-depth browsing on the system file structure. Apps are stored inside in 3 major portions:

• APK files, the actual application files. This is the portion that is moved to SD card when using Froyo A2SD.
• App data files, files generated by applications. This portion usually quite small but a small amount of apps did generate large data files, even larger than the APK file itself. This portion is not moved SD card when using Froyo A2SD.
• Dalvik Cache. Ah ha, I’ve seen this word, what does it mean? Apparently, when an app is executed, Android will generate an executable copy of the app (if not already exist), and this copy is called the “Dalvik Cache”. It’s usually similar or large than the APK file itself.

We usually read the app file size in Android Market before downloading an app. So you see, if an app labeled size is 1Mb, it may ended up taking 2Mb of internal memory space even if you use Froyo A2SD.

Then it occurred to me, I could move the app data and dalvik cache to SD card as well, by creating a Symbolic Link (Symlink). Symlink works like this: you create a symlink from /system/folderA to /sdcard/folderB, so when a program asks for path /system/folderA, it’ll be redirected to /sdcard/folderB transparently. Using this method I could tell a lie to the system, making it access my SD card instead of internal memory when it’s looking for apps.

So I did what I thought logically work. I tried it with one of the apps, symlink was successful, but the app no longer works. Now what’s wrong?

EXT2/EXT3

Some more googling. Apparently, for the above method to work correctly, the symlink target has to be formatted in EXT2/EXT3 file format, while my SD card is in FAT32 format. While FAT16 and FAT32 is the most compatible file format for various platforms, EXT2/EXT3 is a Linux-based file format. The difference between EXT2 and EXT3 is that, EXT3 has a “journaling” feature, similar to NTFS for Windows in some way. Journaling means mean bytes are read/written, additional change-info data will also be added in. This helps prevent data corruption. Back to the time of Win98, if you’d remember, if Win98 did not shut down properly, you’ll see a disk scanning progress bar during next boot. This is because Win98 runs on FAT32, which doesn’t have journaling feature.

So I formatted my SD card using GParted Live CD. It’s easy with the experience to mount AlphaRev previously. The SD card now consists of 2 partitions: 14Gb of FAT32 (this partition will be recognized as the regular SD card), and 800Mb of EXT3 (this partition should become an extended internal memory).

Here’s what I did: http://gparted.sourceforge.net/liveusb.php

Put the card back in, boot it, and OK here’s the 14Gb SD card detected correctly, and… where’s the EXT3? Apparently I have to mount it to somewhere so that the system recognizes it. Google’ed the command to do the mounting, I entered it thru ADB, works fine, in Titanium Backup I can now see 3 drives: SD card, internal memory, and extended internal memory. Yeah. The next step is to make the system run this command every time during boot. It should be simple, but turned out to be not quite so. I’m suppose to edit this /init.rc file, but seemed like every time the phone is reboot, the file will be reverted to original content. There’s just no way to insert a startup command using a Stock ROM.

I could prepare a script package and execute it manually every time I reboot my phone, but that way booting will become extremely slow since Android will rebuild dalvik cache during boot because the dalvik cache folder has not been symlink-ed to the EXT3 partition. I had to find another way around.

A2SD/A2SD+

I realize that, what I was trying to do is actually a method known as “A2SD”. The community came up with this method long before Froyo A2SD was introduced. The concept is exactly as I’ve described. Some developers argue that moving dalvik cache to SD card may slow down performance while running apps so they recommend moving only APK files and app data, thus “A2SD+” refers to configuration that moves everything including dalvik cache as well. With A2SD, one no longer needs to do Froyo A2SD, in fact, it’s better not to do Froyo A2SD.

So someone else must have done the tweak. After some search I found Darktremor. Following the instruction, I downloaded the zip file, install it using the recovery mode, then use ADB to enter the install command… and boom! My phone can’t boot anymore.

Here’s what I did: http://www.facebook.com/note.php?note_id=158809660835039
(It DID NOT work for me this time.)

Don’t panic, I told myself. The phone didn’t turn into a “brick”, as the community called it. It can still be started, but it kept restarting after the HTC boot sound effect. Google’ed but no solution found. So finally I booted into recovery mode, and did a factory reset. I should have done a ClockworkMod backup but I didn’t. The phone rebooted into a fresh copy, sure enough I lost all my apps and data. However thanks to Google contacts sync I managed to get all my contacts data back quite smoothly.

Custom ROM

I was trying to avoid flashing custom ROM because it sounds too scary, and I don’t want to take the risk to backup/restore my data. Now since things cannot be worst, I thought I might as well try out those custom ROMs.

Android phones sold by manufacturers such as HTC, Samsung, Motorola etc, are installed with a customized Android OS, i.e., a piece of software build on top of the original Android OS. For an example, HTC Sense. These are called the “Stock ROM”, something bundled together with the device when shipped. There are developers out there building various different customized Android OS, allowing users to install a different OS to their device. These are called the “Custom ROMs”.

A search in Google reveals a long list of them. I picked the Cyanogen 6.1. It turned out to be quite easy to install following the instructions on the website. Did it without a glitch. Boot it up, and it looks good. Better than the original stock HTC Sense I’d say. It works smooth without any problem so far (3 days). Credits to the Cyanogen team.

This is what I did: http://wiki.cyanogenmod.com/index.php?title=HTC_Desire_(GSM):_Full_Update_Guide
(I skipped the Rooting and ClockworkMod steps because already done so previously.)

And then I tried the Darktremor again, this time it worked like a charm.

Redo the Darktremor setup: http://www.facebook.com/note.php?note_id=158809660835039
(And worked just fine this time.)

So that’s it! I now have full 800Mb of space for apps. And if it’s not enough I can always reformat my SD card to increase the partition.

The Future

I have to admit, it sounds like (and in fact is) a big practice, just to solve a silly memory problem. HTC Desire HD now equipped with 1.5Gb internal memory, Samsung Galaxy S with 8Gb, Nexus S even pushed it to 16Gb. No any A2SD is required on these phones. So what’s the point of doing all these?

When I first got my HTC Desire, it straight away occurred to me that, it’s not a phone, it’s a Computer. It’s trivial that the industry’s model should work the same. When I buy a phone I’m only buying the hardware. Then I can choose to install any software/OS I like, and the custom ROMs serves this purpose. I can imagine developers releasing paid custom ROMs with superb features.

It’s not about solving a problem, it’s about having the freedom to choose. I’d love to see smart phone manufacturers offer customized hardware like how Dell did for desktop computers. I’d like to have a 4.0in” screen, Capacitive Super AMOLED, 2Gb internal memory, 1Gb RAM, oh I don’t need high res dpi, a medium 320x480 will do, leave it empty, I’ll flash my own ROM, and I’d prefer… this housing case, yes, the sharp red color one…