New challenge for malware detection: Virtualization Based rootkit

Several days ago, Invisible Things Lab release a new open source project which named Blue Pill, the first battle ready hardware virtualization based rootkits. Even the code is not sophisticated in this version; I also believe its impact is profound significance. It is a starting gun for new trend of rootkits and malware, which will promote the battle field from OS in-house to VM level.

Just two years ago, virtualization can only be implemented by software emulation, base on interpreter, binary translation etc. We got some software solution, includes VMware Workstation, Microsoft Virtual PC and Virtual Box etc. But they are hard to ensure enough performance and compatibility.

But as the dual core and x64 become common, hardware virtualization solution become the mainstream, includes Xen, VMware ESX Server, Microsoft Longhorn etc. These solutions base on CPU level support includes Intel VT and AMD Pacifica (AMD-V), which introduce a new isolated level beside x86’s ring 0-3.

A mini OS kernel will run in hypervisor mode (VMM), which manage multi guest OS in normal mode (VM). VMM can monitor the status of VM, and take over some operation in VM, such as IO, privileged instruction etc.

This is the common workflow as the designer expected.

But on the other hand, the world is not perfect. Some malicious guys also can use those features to bypass traditional security solutions.

First, some white hat guys from University of Michigan and MSR release a paper SubVirt: implementing malware with virtual machines in 2006. They discuss the possibility for a new type of malware, named virtual-machine based rootkit (VMBR), which installs a virtual-machine monitor underneath an existing operating system and hoists the original operating system into a virtual machine.

Second, Joanna Rutkowska presented at the Black Hat Briefings 2006, for hers hardware virtualization based rootkits, named Blue Pill. This implementation base on CPU support, don’t need any binary translation, and very hard to detect from VM in-house.

Besides, Dino Dai Zovi from matasano also presented at Black Hat USA 2006 about hardware virtualization rootkits, with their implementation base on Xen 3.0.

With my experience, a new concept from idea to malware need one or two year. Now, one year has gone, source is available, and the hardware support will be more and more popular. Everything for this new type of malware is almost ready, only two actors are still missing, a hardware virtualization based rootkit from real world, and detection and clean security solution.