Flier's Sky

Change the default zoom mode in Google Chromium

2008-12-31T00:35:00.005+08:00

As a WebKit based browser, Google Chrome use the text zoom mode as default zoom mode. It means, if you choose the Zoom -> Larger in the page context menu, or press the Ctrl + '+' accelerator, all the text font size in the page will be increased by 20%. On the other hand, if you do the same thing in FireFox, the whole page will be zoomed instead the text font size.

In my opinion, I feel the page zoom mode more comfortable, although I don't think any of those mode is better than the other. At least in my 24' LCD with 1080p, I need the page zoom width enough to fill the right space.

So, I just modified Chrome source code, add two alternative accelerators for page zoom mode; I also add a command line switch to set the default zoom mode when starting.

The changes are straight, because WebKit had provided the build-in support for page zoom mode, and Chrome also had an internal switch for zoom mode.

Following steps are the changes base on latest SVN revision.

Add two new command definition in the \app\chrome_dll_resource.h file, which define the alternative zoom command itself.

#define IDC_ZOOM_VIEW_PLUS 38004
#define IDC_ZOOM_VIEW_MINUS 38005

And add new accelerator like Ctrl + Alt + '+'/'-' in the IDR_MAINFRAME table (app\chrom_dll.rc)

VK_ADD, IDC_ZOOM_VIEW_PLUS, VIRTKEY, CONTROL, ALT
VK_SUBTRACT, IDC_ZOOM_VIEW_MINUS, VIRTKEY, CONTROL, ALT
Handle the commands in the Browser::ExecuteCommand function (\browser\browser.cc), and add a parameter for ZoomIn/ZoomOut functions which is the zoom mode

// Zoom
case IDC_ZOOM_PLUS: ZoomIn(true); break;
case IDC_ZOOM_NORMAL: ZoomReset(); break;
case IDC_ZOOM_MINUS: ZoomOut(true); break;
case IDC_ZOOM_VIEW_PLUS: ZoomIn(false); break;
case IDC_ZOOM_VIEW_MINUS: ZoomOut(false); break;
Add two new zoom mode in PageZoom (\common\page_zoom.h), which will be used for page zoom; and refactor ZoomIn/ZoomOut functions (\browser\browser.cc) for new modes.

enum Function {
VIEW_SMALLER = -2,
SMALLER = -1,
STANDARD = 0,
LARGER = 1,
VIEW_LARGER = 2, };

void Browser::ZoomIn(bool isTextOnly) {
UserMetrics::RecordAction(L"ZoomPlus", profile_);
GetSelectedTabContents()->AsWebContents()->render_view_host()->Zoom( isTextOnly ? PageZoom::LARGER : PageZoom::VIEW_LARGER);}

void Browser::ZoomOut(bool isTextOnly) {
UserMetrics::RecordAction(L"ZoomMinus", profile_);
GetSelectedTabContents()->AsWebContents()->render_view_host()->Zoom( isTextOnly ? PageZoom::SMALLER : PageZoom::VIEW_SMALLER);}

Then, the RenderViewHost::Zoom function will wrap the zoom mode as a ViewMsg_Zoom message, and send it to the render process.
Modify the render handler, which redirect ViewMsg_Zoom message to OnZoom function (\renderer\render_view.cc)

IPC_MESSAGE_HANDLER(ViewMsg_Zoom, OnZoom)

The OnZoom function has a hard code kZoomIsTextOnly constant. So, we just change its value base on the command line parameter; and set two kind of zoom mode base on its default setting.

void RenderView::OnZoom(int function) {
static const bool zoomIsTextOnly = CommandLine().HasSwitch(switches::kZoomIsTextOnly);

switch (function) {
case PageZoom::VIEW_SMALLER:
webview()->ZoomOut(!zoomIsTextOnly); break;
case PageZoom::SMALLER: webview()->ZoomOut(zoomIsTextOnly); break;
case PageZoom::STANDARD: webview()->ResetZoom(); break;
case PageZoom::LARGER: webview()->ZoomIn(zoomIsTextOnly); break;
case PageZoom::VIEW_LARGER: webview()->ZoomIn(!zoomIsTextOnly); break;
default: NOTREACHED(); }}
Besides the command workflow, there are some bookkeeping works.

We need update the command status base on the current stage in the Browser::UpdateCommandsForTabState function (browser\browser.cc)

controller_.UpdateCommandEnabled(IDC_ZOOM_VIEW_PLUS, is_web_contents);
controller_.UpdateCommandEnabled(IDC_ZOOM_VIEW_MINUS, is_web_contents);
To change the default zoom mode, I added a new switch for command line in \common\chrome_switches.h/.cc

// Turns on text only mode zoom supportconst wchar_t
kZoomIsTextOnly[] = L"zoom-is-text-only";

and add it into the switch_names array in RenderProcessHost::Init function (browser\render_process_host.cc). So, the browser process will propagate the switch to the child render process.

static const wchar_t* const switch_names[] = {
//...
switches::kZoomIsTextOnly, };

Besides, I found a similar feature request has been submited to the chrome issues tracking. So, I will add some comment as patch for this issue. Wish Google could merge it into next offical release :)

Use 3rd party font render engine in Google Chromium

2008-10-15T18:26:00.005+08:00

As you know, there are some 3rd party font engines, such as GDI++, which could hook Windows font related APIs, and render font more smoother.
In my opinion, that engine seems better than ClearType or other build-in Windows font engine.

For Chromium, it use multi-processes architecture, a standalone renderer process will render the page in sandbox. It means that we should know which process need be hooked for font smoother.

So I made some slight changes in Chromium to support GDI++

1. a new command line parameter "--use-gdipp=", which give a setting file for GDI++

I add a new key name kUseGdiPP (chrome_switches.h|.cc), which could be used to parse command line parameters.

const wchar_t kUseGdiPP[] = L"use-gdipp";

And also add it in switch_names array (render_process_host.cc), that control the main browser process propagate which switches to renderer child process.

2. the initialize and finalize code in RendererMain function, which load and free the font engine

After the renderer process started, it call RendererMain function (renderer_main.cc) to initilize process. So I add the font engine related code just after HandleRendererErrorTestParameters calling. Because that function could help us debugging the children process.

bool use_gdipp = parsed_command_line.HasSwitch(switches::kUseGdiPP);

if (use_gdipp)

{

std::wstring setting_file_name =

parsed_command_line.GetSwitchValue(switches::kUseGdiPP);

if (!LoadFontEngine(setting_file_name.c_str()))

{

LOG(WARNING) << "Fail to load GDI++ font engine.";

use_gdipp = false;

}

else

{

DLOG(INFO) << "Loaded GDI++ font engine.";

}

We can use "--renderer-startup-dialog" parameter to popup a message box before any other initialize code. And we could attach debugger to renderer process for troubeshooting. Besides, this parameter must be used with "--no-sandbox" parameter. Please check Debugging Chromium for more detail.

On the other hand, we also free the font engine before uninitialize COM with CoUninitialize function.

if (use_gdipp)

{

FreeFontEngine();

}

3. a customized GDI++ library which will be linked as static library in chrome.dll library

To compile the GDI++ library, you should download the latest FreeType version GDI++, with FreeType and Detours.

And create a static library project in chrome solution, which should add all the source files in GDI++\src folder except run.cpp.

Than, you should do some dirty works to make it compilable, remove some function definition and includes.

Two funtions LoadFontEngine and FreeFontEngine should be added in hook.cpp as wrapper, and a new header file should be created to declare those functions.

bool LoadFontEngine(LPCTSTR lpszFile)

{

hook_initinternal();

CCriticalSectionLock::Init();

if (!g_TLInfo.ProcessInit())

{

return false;

}

CGdippSettings* pSettings = CGdippSettings::CreateInstance();

if (!pSettings || !pSettings->LoadAppSettings(lpszFile))

{

CGdippSettings::DestroyInstance();

return false;

}

if (!FontLInit())

{

return false;

}

g_pFTEngine = new FreeTypeFontEngine;

if (!g_pFTEngine)

{

return false;

}

if (!InterlockedExchange(&g_bHookEnabled, TRUE))

{

hook_init();

}

return true;

}

void FreeFontEngine()

{

if (InterlockedExchange(&g_bHookEnabled, FALSE))

{

hook_term();

}

if (g_pFTEngine) delete g_pFTEngine;

FontLFree();

CGdippSettings::DestroyInstance();

g_TLInfo.ProcessTerm();

CCriticalSectionLock::Term();

}

After that, you could get a local build with GDI++, it depend on freetype6.dll, zlib1.dll and detoured.dll as cost. And you could customize your setting file for GDI++, and run your local chrome with it. For more detail, you could read its offical document (Japanese), guess it by name or google it. :)

Virtualization Detection

2008-04-02T22:39:00.005+08:00

As you know, virtualization is changing traditional IT infrastructure.
VMware ESX server, the market leading hypervisor, has been accepted by customers of all sizes, including all of the Fortune 100; Microsoft Windows 2008 also integrate Hyper-V as a core server role; several open source projects also focus on it, such as Xen (Intel), KVM (Linux) and Virtual Box (Sun) etc.

There are a lot of research papers and online discussion about Virtualization Detection

The 1st option is using vender specific public interface or backdoor, which could provide rich information about host OS and virtualization.

Microsoft public a Hypervisor Functional Specification for its Hyper-V implementation, which includes definition about not only feature and interface discovery but also Hyper-Call based management interfaces.
We could directly use one simple CPUID instruction to detect the existent of Hyper-V.

bool CHyperV::Exists(void)
{
int cpuid[4];

__cpuid(cpuid, 1);

return cpuid[2] & (1 <<>Further more, vender ID, neutral interface name, version and supported features could be fetched through similar method.

VMware also has similar private interface through a magic I/O port, which includes not only implementation information but also runtime information, such as processor speed, mouse cursor position and clipboard etc.
We could use its magic I/O port to detect the existent of VMware.

bool CVMware::Exists(void)
{
bool exists = true;

__try
{

__asm
{
push edx
push ecx
push ebx

mov eax, 'VMXh'
mov ebx, 0 // any value but not the MAGIC VALUE
mov ecx, 10 // get VMWare version

mov edx, 'VX' // port number

in eax, dx // read port

// on return EAX returns the VERSION
cmp ebx, 'VMXh' // is it a reply from VMWare?

setz [exists] // set return value

pop ebx
pop ecx
pop edx
}
}

__except(EXCEPTION_EXECUTE_HANDLER)
{
exists = false;
}
return exists;
}
For more completely commands list, you could check the following URL

VMware Backdoor I/O Port
http://chitchat.at.infoseek.co.jp/vmware/backdoor.html
VMware also provide an open source project, Open Virtual Machine Tools, which includes more detail usage of its backdoor.

Besides, Microsoft Virtual PC/Server, bochs, qemu has similar interface, you could check the following URLs for more detail.

How to detect Virtual PC or VMware from your program
http://www.codegurus.be/codegurus/Programming/virtualpc&vmware_en.htm

Attacks on Virtual Machine Emulators
http://www.symantec.com/avcenter/reference/Virtual_Machine_Threats.pdf

The 2nd option is checking VM implementation mechanism, which could detect some unknown VM vender in theory.

The mostly famous method is checking IDT register from Joanna’s Red Pill.

Red Pill... or how to detect VMM using (almost) one CPU instruction
http://invisiblethings.org/papers/redpill.html
The heart of this code is actually the SIDT instruction (encoded
as 0F010D[addr]), which stores the contents of the interrupt descriptor table register (IDTR) in the destination operand, which is actually a memory location. What is special and interesting about SIDT instruction is that, it can be executed in non privileged mode (ring3) but it returns the contents of the sensitive register, used internally by operating system.

Because there is only one IDTR register, but there are at least two OS running concurrently (i.e. the host and the guest OS), VMM needs to relocate the guest's IDTR in a safe place, so that it will not conflict with a host's one. Unfortunately, VMM cannot know if (and when) the process running in guest OS executes SIDT instruction, since it is not privileged (and it doesn't generate exception). Thus the process gets the relocated address of IDT table. It was observed that on VMWare, the relocated address of IDT is at address 0xffXXXXXX, whereas on Virtual PC it is 0xe8XXXXXX. This was tested on VMWare Workstation 4 and Virtual PC 2004, both running on Windows XP host OS.
This method only use one SIDT instruction to detect the existent of VM, because a lot of x86 software VM implementation choose to change IDT for guest OS.

bool CUnknown::CheckIDT(void)
{
unsigned char idt[6];

_asm
{
sidt idt

}
return idt[5] > 0xd0;
}
Unfortunately, this method could be spoofed by VM implementation

SubVirt: Implementing malware with virtual machines
http://www.eecs.umich.edu/~pmchen/papers/king06.pdf
The redpill [39] virtual-machine detection technique detects the
presence of a VMM by using the sidt instruction. The sidt instruction reads the address of the processor’s interrupt descriptor table. This address is different for an operating system running directly above hardware than for an operating system running above a VMM. VMMs emulate the sidt instruction when it is called from kernel-mode, but for performance reasons this instruction is not virtualized when called from user-mode. Thus, user-mode code can execute the
sidt instruction to detect the presence of a VMM. To defeat this detection technique, we use virtual-machine introspection to emulate the sidt instruction when it is called by the redpill application.

And it also doesn’t work on some hardware accelerated VM implementation and multi-core CPU system.

Introducing Blue Pill
http://theinvisiblethings.blogspot.com/2006/06/introducing-blue-pill.html
SubVirt was implemented on x86 hardware, which doesn't allow to achieve 100% virtualization, because there are number of sensitive instructions, which are not privileged, like the famous SIDT/SGDT/SLDT. This allows for trivial detection of the virtual mode - see e.g. my little Red Pill program. This however, doesn't apply to Blue Pill, as it relies on AMD SVM
technology.

Detecting the Presence of Virtual Machines
Using the Local Data Table
http://www.offensivecomputing.net/files/active/0/vm.pdf
The SIDT mechanism as implemented by Tobias Klein [1] and
separately by Joanna Rutkowska [2] is a method for detecting the presence of a virtual machine environment. While the test is by no means thorough, it is an effective test for the presence of an emulated CPU environment on a single-processor machine. There are various problems with the implementation, however. If a multi-core CPU is used, the interrupt descriptor table can change significantly when the process is run on different cores. Furthermore if two or more physical processors are present the same implementation issues apply.
To improve the issues of SIDT mechanism, a lot of attempts focus on similar register, such as GDT, LDT or TS register.

Detecting the Presence of Virtual Machines Using the Local Data Table
http://www.offensivecomputing.net/files/active/0/vm.pdf

Methods for Virtual Machine Detection
http://www.s21sec.com/descargas/vmware-eng.pdf

bool CUnknown::CheckGDT(void)
{
unsigned char gdt[6];
_asm
{
sgdt gdt
}
return gdt[5] > 0xd0;
}

bool CUnknown::CheckLDT(void)
{
unsigned char ldt[2];
_asm
{
sldt ldt
}
return ldt[0] != 0 && ldt[1] != 0;
}

bool CUnknown::CheckTR(void)
{
unsigned char tr[2];

_asm
{
str tr

}
return tr[0] == 0 && tr[1] == 0x40;
}
The major problem of these kinds of detection methods is the researcher hard to test compatibility on every VM vender.

After I made a simple test, it seems we hard to trust any of them, but we could made a hyper detection method base on more test results.

The 3rd option is using VM specific defect, which mostly tight coupling with VM version.

For example, eEye researcher published a detection method base on wrong far execution transfers behaviors.

Another VMware Detection
http://eeyeresearch.typepad.com/blog/2006/09/another_vmware_.html
So, the real world is not perfect, we hard to use one hammer to solve all the problems. We should carefully define our purpose and scope of virtualization detection, and choose a hyper detection solution for major focus market.

New challenge for malware detection: Virtualization Based rootkit

2007-08-07T18:54:00.000+08:00

Several days ago, Invisible Things Lab release a new open source project which named Blue Pill, the first battle ready hardware virtualization based rootkits. Even the code is not sophisticated in this version; I also believe its impact is profound significance. It is a starting gun for new trend of rootkits and malware, which will promote the battle field from OS in-house to VM level.

Just two years ago, virtualization can only be implemented by software emulation, base on interpreter, binary translation etc. We got some software solution, includes VMware Workstation, Microsoft Virtual PC and Virtual Box etc. But they are hard to ensure enough performance and compatibility.

But as the dual core and x64 become common, hardware virtualization solution become the mainstream, includes Xen, VMware ESX Server, Microsoft Longhorn etc. These solutions base on CPU level support includes Intel VT and AMD Pacifica (AMD-V), which introduce a new isolated level beside x86’s ring 0-3.

A mini OS kernel will run in hypervisor mode (VMM), which manage multi guest OS in normal mode (VM). VMM can monitor the status of VM, and take over some operation in VM, such as IO, privileged instruction etc.

This is the common workflow as the designer expected.

But on the other hand, the world is not perfect. Some malicious guys also can use those features to bypass traditional security solutions.

First, some white hat guys from University of Michigan and MSR release a paper SubVirt: implementing malware with virtual machines in 2006. They discuss the possibility for a new type of malware, named virtual-machine based rootkit (VMBR), which installs a virtual-machine monitor underneath an existing operating system and hoists the original operating system into a virtual machine.

Second, Joanna Rutkowska presented at the Black Hat Briefings 2006, for hers hardware virtualization based rootkits, named Blue Pill. This implementation base on CPU support, don’t need any binary translation, and very hard to detect from VM in-house.

Besides, Dino Dai Zovi from matasano also presented at Black Hat USA 2006 about hardware virtualization rootkits, with their implementation base on Xen 3.0.

With my experience, a new concept from idea to malware need one or two year. Now, one year has gone, source is available, and the hardware support will be more and more popular. Everything for this new type of malware is almost ready, only two actors are still missing, a hardware virtualization based rootkit from real world, and detection and clean security solution.

Wireshark Dissector Plugin for Look'n'Stop

2007-07-10T21:46:00.000+08:00

From a developer viewpoint, Look'n'Stop is a great personal firewall. Even their design may not very clearly for the normal user, but if you have enough background knowledge, it can be a powerful analyzer for the security threats.

After a packet be allowed or blocked by rule, Look'n'Stop record it to log and provide a dialog for detail information. But these information not enough for me, so I decide to write a plugin to got more :)

Fortunately, they provide some plugin API for log display and rule editor. Through those interfaces, I can pop up my dissector dialog to display the protocol tree of packet.

To avoid reinvent the wheel, I choose Wireshark as background dissector. Because Wireshark, or more well know name - ethereal, is the best open source network protocol analyzer, and is the standard in many industries.

Even Wireshark has encapsulate all dissector in a library, its interface not clearly and stable, so I decide use its terminal-based edition - tshark, as the major dissector, because it can read packet from stdin, and dump the protocol tree to stdou as XML format.

So the major data and control flow includes:

Look'n'Stop pass the packet data to our plugin through its API
plugin fork a tshark process for dissect
plugin dump the packet as libpcap format to tshark's stdin
tshark dissect the packet to protocol tree and output the XML to stdout
plugin fetch the XML output and parse it with expat
plugin popup a tree-based dialog and render the protocol tree
popup dialog provide more feature, for example, save the packet as libpcap format

Combine those steps, we got a new dissector plugin :)

You can select a field in protocol tree, and the corresponding data bytes will be highlight in the bottom editor. If you want to save the packet for more analyzer, just right-click the windows title, and choose "Save As" in system menu. it support save packet as libpcap, xml and text format.

But before this, you should download and install wireshark first, and configure the installation path in plugin, such as

If you input a valid path, plugin will fetch the version and copyright from tshark, and save it to registry to reuse in future.

For the china user, I integrate some location information of IP address. You should download the latest IP database from cz88.net, and configure the QQWry setting page, such as

After you choose "Use QQWry ...", a location information will be appended after some IP field in protocol tree.

If you have interest about this, please download the prebuild binary or compire it by youself.

That's all, if you have any advices or want to improve it by yourself, please contact me directly :)

Write a debugger in 5 minutes with PyDbgEng

2007-05-28T00:14:00.000+08:00

The debug mechanism of PyDbgEng is same to other Win32 debugger, just create or attach to a debuggee process and call WaitForEvent to process the debug events, such as create process, load module etc.

#!/usr/bin/env python
import sys
from PyDbgEng import *

c = DebugClient()

c.CreateProcess("ftp.exe", createFlags=[CreateFlags.ATTACH_ONLY_THIS_PROCESS,
  CreateFlags.NEW_CONSOLE])

while c.Control.WaitForEvent():
pass

The previous code is a simple debugger

create a debug session with DebugClient()
create the debuggee process with CreateProcess
attach to the new process with CreateFlags.ATTACH_ONLY_THIS_PROCESS
create a new console window for debuggee with CreateFlags.NEW_CONSOLE

To get more debug events, we must add some debug event callback

def onCreateProcess(args):
print "CreateProc: %08x-%08x %s\t%s" % (
args.BaseOffset, args.BaseOffset+args.ModuleSize,
args.ModuleName, args.ImageName)

def onExitProcess(args):
print "ExitProcess %d" % args.ExitCode

def onCreateThread(args):
print "CreateThread %x %08x %08x" % (args.Handle, args.DataOffset, args.StartOffset)

def onExitThread(args):
print "ExitThread %d" % args.ExitCode

def onLoadModule(args):
print "ModLoad: %08x-%08x %s\t%s" % (
args.BaseOffset, args.BaseOffset+args.ModuleSize,
args.ModuleName, args.ImageName)

c.EventCallbacks.CreateProcess = onCreateProcess
c.EventCallbacks.ExitProcess = onExitProcess
c.EventCallbacks.CreateThread = onCreateThread
c.EventCallbacks.ExitThread = onExitThread
c.EventCallbacks.LoadModule = onLoadModule
c.EventCallbacks.Attach()

c.CreateProcess(...)

Now, we will receive 5 kinds of debug events, which allow use show detail information, or do some action, such as add breakpoint, etc. After setting the callback and attach them to debugger, we can got events, like:

CreateProc: 01000000-01012000 ftp ftp.exe
ModLoad: 7c930000-7ca00000 ntdll ntdll.dll
ExitProcess 0

Other kinds of events are about the status or state changing for debug session, debuggee and symbol, we also can use callback to process them

def onSessionStatus(args):
print "SessionStatus: %s" % (str(args.Status))

def onChangeEngineState(args):
sys.stdout.write("EngineState: %s " % str(args.State))

if EngineState.EXECUTION_STATUS == args.State:
 print ExecutionStatus.values[args.Argument & 0xf]
else:
 print "%x" % args.Argument

c.EventCallbacks.SessionStatus = onSessionStatus
c.EventCallbacks.ChangeEngineState = onChangeEngineState

These events will allow you watch the order of state changing, like

EngineState: SYSTEMS 0
EngineState: EXECUTION_STATUS NO_CHANGE
EngineState: EXTENSIONS 0
SessionStatus: ACTIVE
EngineState: EXECUTION_STATUS BREAK
EngineState: CURRENT_THREAD 0
CreateProc: 01000000-01012000 ftp ftp.exe
EngineState: EXECUTION_STATUS BREAK
EngineState: EXECUTION_STATUS BREAK
EngineState: CURRENT_THREAD 0
ModLoad: 7c930000-7ca00000 ntdll ntdll.dll
EngineState: EXECUTION_STATUS BREAK
EngineState: EXECUTION_STATUS BREAK
EngineState: CURRENT_THREAD 0

To act as a complete debugger, we add and process breakpoint for some predefined function

def onLoadModule(args):
print "ModLoad: %08x-%08x %s\t%s" % (
  args.BaseOffset, args.BaseOffset+args.ModuleSize,
  args.ModuleName, args.ImageName)

if "WS2_32" == args.ModuleName:
  bp = c.Control.AddBreakpoint(flags=[BreakpointFlag.ENABLED],
      offset=c.Symbols.GetOffsetByName("WS2_32!socket")) 

  symbol = c.Symbols.GetNameByOffset(bp.Offset)
  print "Add Breakpoint: %s %d @ %08x %s:%d" % (str(bp.Type[0]), bp.Id, bp.Offset, symbol[0], symbol[1])

def onBreakpoint(args):
bp = args.Breakpoint

symbol = c.Symbols.GetNameByOffset(bp.Offset)
print "Hit Breakpoint: %s %d @ %08x %s:%d" % (str(bp.Type[0]), bp.Id, bp.Offset, symbol[0], symbol[1])

return ExecutionStatus.BREAK

c.EventCallbacks.Breakpoint = onBreakpoint

After the WS2_32 module was loaded, we use DebugControl.AddBreakpoint method to create a new code break for WS2_32!socket, which will be called after ftp.exe started. So we add onBreakpoint callback function to show which breakpoint was hit.

ModLoad: 71b60000-71b77000 WS2_32 C:\WINDOWS\system32\WS2_32.dll
EngineState: BREAKPOINTS 0
EngineState: BREAKPOINTS 0
EngineState: BREAKPOINTS 0
Add Breakpoint: CODE 0 @ 71b6410c WS2_32!socket:0
...
EngineState: CURRENT_THREAD 0
Hit Breakpoint: CODE 0 @ 71b6410c WS2_32!socket:0
EngineState: EXECUTION_STATUS BREAK
EngineState: EXECUTION_STATUS GO_HANDLED
Change engine state to GO
EngineState: EXECUTION_STATUS GO_HANDLED
EngineState: EXECUTION_STATUS GO

Besides these expected events, we need another callback to process the exception.

def onException(args):
 symbol = c.Symbols.GetNameByOffset(args.Address)
 sys.stdout.write("Exception: %08x %08x %s:%d" % (args.Code, args.Address, symbol[0], symbol[1]))

 if args.IsFirstChance:
   print " first"
 else:
   print " second"

 for frame in c.Control.GetStackFrames():
   symbol = c.Symbols.GetNameByOffset(frame.InstructionOffset)
   print " %04d %08x %s:%d" % (frame.FrameNumber, frame.InstructionOffset, symbol[0], symbol[1])

 print c.Control.Breakpoints

c.EventCallbacks.Exception = onException

The callback will log the exception information, and dump the caller stack with DebugControl.GetStackFrames() method, like

Exception: 000006ba 7c80bee7 kernel32!RaiseException:83 first
0000 7c80bee7 kernel32!RaiseException:83
0001 77c31e37 RPCRT4!RpcpRaiseException:36
0002 77c32042 RPCRT4!NdrGetBuffer:70
0003 77cb30e4 RPCRT4!NdrClientCall2:407
0004 76e35039 DNSAPI!R_ResolverQuery:28
0005 76e34f59 DNSAPI!Query_PrivateExW:391
0006 76e3505f DNSAPI!DnsQuery_W:58
0007 71a83f8e MSWSOCK!SaBlob_Query:45
...
0023 010045c5 ftp!main:1665
0024 01006ee0 ftp!mainCRTStartup:303
0025 7c82f23b kernel32!BaseProcessStart:35

Finally, we add a try...except to protect the WaitForEvent method, because some situation will raise exception

try:
while c.Control.WaitForEvent():
    c.Control.ExecutionStatus = ExecutionStatus.GO_HANDLED
    print "Change engine state to %s" % c.Control.ExecutionStatus
except:
if ExecutionStatus.NO_DEBUGGEE != c.Control.ExecutionStatus:
    print "Unexpected error:", sys.exc_info()[0]
    raise

Now, its work, with less than one hundred code lines, and can be expand easy :)

Access the kernel space with PyDbgEng

2007-05-26T15:19:00.000+08:00

One year ago, I wrote a Chinese article <How to use kd/windbg engine to access the kernel space>, now I port the implementation to the PyDbgExt project, so we can directly access the kernel space in python.

>>> from PyDbgEng import *
>>> c = DebugClient()
>>> c.AttachKernel()
>>> c.Control.WaitForEvent()
True
>>> c.Symbols.LoadedModules
{'nt': (Module nt @ ffffffff80800000)}
>>> c.Symbols.GetSymbols("nt!KiServiceTable")
{'KiServiceTable': ((Symbol nt!KiServiceTable), 0)}
>>> offset = c.Symbols.GetSymbols("nt!KiServiceTable").popitem()[1][0].Offset
>>> c.Symbols.GetSymbols(c.DataSpaces.Virtual.ReadPointers(offset)[0])
{'NtAcceptConnectPort': ((Symbol nt!NtAcceptConnectPort), 18446744071571636794L)}

To access the kernel mode, we must attach engine to the local kernel with AttachKernel() first, and begin wait a debug event process with WaitForEvent(). For the kernel mode, this function will return immediately. After this, we can use almost all the functions to access the kernel space, such as modules or symbols.

Under the hood, to support this feature in a standalone python module, I use some dirty hack method, because the debug engine and driver disallow it used outside kd.exe or windbg.exe.
So, before call IDebugClient::AttachKernel method to enter the kernel mode, we must first hook four system functions:

static DWORD WINAPI HookedGetModuleFileNameW(HMODULE hModule, LPWSTR lpFilename, DWORD nSize)
{
DWORD dwSize = s_fnGetModuleFileNameW(hModule, lpFilename, nSize);

if (!hModule)
{
wchar_t *pch = wcsrchr(lpFilename, L'\\');
wcscpy_s(pch ? pch+1 : lpFilename, pch ? (nSize - (pch - lpFilename)) : nSize, L"kd.exe");
dwSize = wcslen(lpFilename);
}

return dwSize;
}

GetModuleFileNameW, debug engine use it to got the current executable filename, and check whether the filename end with "kd.exe" or "windbg.exe", but our filename maybe "python.exe"
FindResourceW, SizeofResource, LoadResource: debug engine use those functions to find and export the driver, which implement some internal works. I extract it from windbg.exe and embedded into PyDbgEng.dll.

As the previous description mentioned, we embedded the driver "kldbgdrv.sys" to resource, which as type 0x7777 and id 0x4444, like

/////////////////////////////////////////////////////////////////////////////
//
// RCDATA
//

30583 17476 "kldbgdrv.sys"

This tech can be used in any program which wants to access the kernel mode :)

Dump Windows Service Table in WinDbg

2007-05-26T03:01:00.000+08:00

buri write a great article <Windows Service Table Dumper for WinDbg> show how to use the built-in script language in WinDbg to do a real job: dump the windows service table. But this script is short of readability, because the build-in script in WinDbg is very strange like its command design.
So, why we can't implement it more easy and readable, base on a friendly python script through PyDbgExt, my python extension for WinDbg :)

First we need define a script module, such as dumpServiceTable.py, which includes a function dumpServiceTable to dump that table, and import the dependence modules

from PyDbgEng import *
from struct import *

c = DebugClient.Current
s = c.Symbols
v = c.DataSpaces.Virtual

Next, we got the common base object, such as DebugClient.Current which is the current debug session in windbg; Symbols and DataSpaces.Virtual will support us query the debug symbol and read/write the virtual address space.

def getSymbol(name):
return s.GetSymbols(name).popitem()[1][0]

def getSymbol(offset):
return s.GetSymbols(offset).popitem()[1][0]

def readDWORD(offset):
return unpack_from("L", v.Read(offset, 4))[0]

To make the code more readable, we define some utility functions: getSymbol can get the symbol object with its name or offset; readDWORD read unsigned long from the offset. According to the result type of VirtualDataSpace.Read function is a buffer object, we need use unpack_from function to decode the buffer.

def dumpServiceTable():
KiServiceTable = getSymbol("nt!KiServiceTable")
KiServiceLimit = getSymbol("nt!KiServiceLimit")

idx = 0

for addr in v.ReadPointers(KiServiceTable.Offset, readDWORD(KiServiceLimit.Offset)):
try:
symbol = getSymbol(addr)

symbolName = "%s!%s" % (symbol.Module.ModuleName, symbol.Name)
except:
symbolName = ""

print "%03d %08x %s" % (idx, addr & 0xFFFFFFFF, symbolName)

idx = idx + 1

The last part of code read and dump the service table:

get the symbol object of nt!KiServiceTable and nt!KiServiceLimit
read a group of pointers from the begin of table
try to get the symbol object for every entry in table
if the symbol exists, dump it's address, module and name
if the symbol nonexists, just show warning. we can provide more information about this in future

Finally, we load the script to windbg and execute it :)

lkd> .extpath+ D:\Study\Win32\PyDbgExt\Binary\debug
Extension search path is: ...;D:\Study\Win32\PyDbgExt\Binary\debug
lkd> .load PyDbgExt
lkd> .chain
Extension DLL search Path:
...
Extension DLL chain:
PyDbgExt: API 1.0.0, built Sat May 26 02:17:49 2007
[path: D:\Study\Win32\PyDbgExt\Binary\debug\PyDbgExt.dll]
dbghelp: image 6.7.0005.0, API 6.0.6, built Fri Mar 30 02:08:09 2007
[path: D:\MS\Debugging Tools for Windows\dbghelp.dll]
...
lkd> !import dumpServiceTable
Import succeeded.
lkd> !eval dumpServiceTable.dumpServiceTable()
000 8092023a nt!NtAcceptConnectPort
001 8096b71e nt!NtAccessCheck
002 8096f9be nt!NtAccessCheckAndAuditAlarm
...
032 808b9810 nt!NtCompressKey
033 f4bed0d2
034 8088d0c8 nt!NtContinue
...

If there some wrong in script, just edit it and reload it with python build-in function

lkd> !eval reload(dumpServiceTable)

Enjoy it :)

An alternative open source virtualization solution: VirtualBox

2007-05-16T17:30:00.000+08:00

To accelerate the product development and test, we usual choose some kinds of virtualizer to simulate environment, such as VMware workstation, Virtual PC etc. With these kinds of VM, we can build the environment one time, and reuse it again and again, just take snapshot and rollback it.

But if we have some advance requirement, such as control the VM to implement some auto-test script, these commercial products maybe restrict your idea. Even they provided some SDK, the ability is very limited.

Fortunately we have some alternative virtualization solutions, such as VirtualBox. It is a general-purpose full virtualizer for x86 hardware. Targeted at server, desktop and embedded use, it is now the only professional-quality virtualization solution that is also Open Source Software.

The usage and GUI of VirtualBox is very like VMware, so we can switch to it easy. The VirtualBox can be downloaded and installed on Windows, Linux and OS X hosts; And the guest OS can support mostly common platform, such as Windows, Linux, xxxBSD, etc. A complete guest OS list can be found at Guest OSes. It also has an open source edition under the GPL license, but it isn't including some advance features, such as RDP support.

Compare with VMware or other VM vender, VirtualBox has a great openness. It's virtual machine descriptions in XML, we can easy fetch information from it; It's VM directly support RDP (Remote Desktop Protocol) protocol, so we can use remote desktop connection software to control it, the latest version of VMware - Workstation 6.0 provide similar feature which allow remotely access the console of a VM from a VNC client.

Furthermore, the most important difference is VirtualBox provided a very powerful management interface. For end-users, they can use VBoxManage in command line to control most common functions of VM; for developers, they can get some depth control with a set of COM/XPCOM control interfaces, such as control the keyboard/mouse action etc.

From the implementation viewpoint, the architecture of VirtualBox much likes a client/server solution. The controller client, such as QT-based GUI or console mode VBoxManage, connects to the backend VM implementation (VBoxVMM, etc.), through COM/XPCOM interface (VBoxSVC, VBoxC, etc).

In addition, a kernel driver (VBoxDrv.sys on Windows) or module (vboxdrv on Linux) will allocating physical memory for the VM, control the context switch and other dirty work.

But not like Xen and VMware, VirtualBox is not directly depending on some kinds of hardware virtualization technology, such as VT or AMD-V. VirtualBox run the guest OS kernel at ring 1, through some advanced code scanning, analysis and patching techniques, such as "Patch Manager" (PATM) and "Code Scanning and Analysis Manager" (CSAM).

Before executing ring 0 code, we scan it recursively to discover problematic instructions. We then perform in-situ patching, i.e. we replace the instruction with a jump to hypervisor memory where an integrated code generator has placed a more suitable implementation. In reality, this is a very complex task as there are lots of odd situations to be discovered and handled correctly. So, with its current complexity, one could argue that PATM is an advanced in-situ recompiler.

In addition, every time a fault occurs, we analyze the fault's cause to determine if it is possible to patch the offending code to prevent it from causing more expensive faults in the future. This turns out to work very well, and we can reduce the faults caused by our virtualization to a rate that performs much better than a typical recompiler, or even VT-x technology, for that matter.

This method seems specially but effective, at least on my machine, VirtualBox run some OS correctly and very fast.

If you have more detail question, please refer to their Developer FAQ or directly read the source code. :)