Paper: SPiKE: Engineering Malware Analysis Tools using Unobtrusive Binary-Instrumentation
SPiKE: Engineering Malware Analysis Tools using Unobtrusive Binary-Instrumentation is a paper by Amit Vasudevan and Ramesh Yerraballi from UT Arlington. This paper outlines several different methods to control a running process. In this case, it is used for controlling malware.
The SPiKE method uses a kernel solution to implement "drifters" or memory read/write breakpoints. These breakpoints are then used to control the execution of the malware. The breakpoints are done via setting the target memory page in the kernel to a "not-present" flag, and then through "subtle software techniques" (their quote, not mine) they are able to transfer execution to the SPiKE API. This is very similar to Joe Stewart's OllyBone method of setting region based breakpoints. The rest of the interception points are based on the Windows API hooking via CreateProcess, OpenProcess, and others.
While this method is definitely interesting, there are a couple of drawbacks. First any sort of memory region break-pointing assumes that the program does not have control at the OS level. Second, there is a limitation of not having fine-grain control of the process. Lastly, this requires the analyst to have yet another device driver loaded into their kernel. Some may not like this option.
I wish they would have tested against a much larger malware set to check for completeness. What is good about this technique and other similar ones is that it allows the executable to run on raw hardware. Certainly there are good performance advantages for these sorts of techniques.