HALF: Hollowing Analysis Framework for Binary Programs with Kernel Module Assistance

TL;DR

HALF addresses the enduring conflict between fidelity and performance in fine-grained dynamic binary analysis by using kernel-assisted process hollowing to create a hollowed container that shares the target's address layout. A lightweight instrumented target records dynamic events, while a separate analysis container performs heavyweight taint propagation under a same-address view, coordinated by a kernel monitor that uses exception-driven synchronization. Empirical results on Windows show substantial efficiency gains over in-process baselines, and HALF successfully analyzes layout-sensitive exploits and real malware where prior frameworks fail due to memory conflicts. The work demonstrates a practical, deployable approach to transparent binary analysis with strong isolation, enabling more accurate threat reproduction and analysis in security workflows.

Abstract

Binary program analysis represents a fundamental pillar of modern system security. Fine-grained methodologies like dynamic taint analysis still suffer from deployment complexity and performance overhead despite significant progress. Traditional in-process analysis tools trigger severe \textbf{address-space conflicts} that inevitably disrupt the native memory layout of the target. These conflicts frequently cause layout-sensitive exploits and evasive malware to deviate from their intended execution paths or fail entirely. This paper introduces \textbf{HALF} as a novel framework that resolves this fundamental tension while ensuring both analysis fidelity and practical performance. HALF achieves high-fidelity address-space transparency by leveraging a kernel-assisted process hollowing mechanism. This design effectively eliminates the observation artifacts that characterize traditional instrumentation tools. We further mitigate the synchronization latency of decoupled execution by implementing an exception-driven strategy via a lightweight kernel monitor. Extensive evaluation of a Windows-based prototype demonstrates that HALF maintains superior performance compared to conventional in-process baselines. HALF also provides unique capabilities for deconstructing complex, stealthy threats where existing frameworks fail to maintain execution integrity.

Figures (16)

• Figure 1: Example of process space memory allocation conflict. This figure illustrates memory usage conflicts between typical DFT tools and specific exploit programs. Left side (DFT): shows that analysis tools (such as libdft) pre-allocate large memory regions (gray area near 0x20000000) in process space to maintain shadow memory for taint analysis. Right side (Exploit): shows that certain exploit programs attempt to use heap spraying techniques to arrange memory in specific fixed address ranges (approximately 400MB - 800MB). Conflict: pre-allocation by analysis tools occupies memory addresses required by exploit programs, causing conflicts that prevent exploit code from executing as expected or lead to abnormal program behavior.
• Figure 2: A code-level example of an address-space conflict between an in-process taint tracker (libdft-style) and a layout-sensitive exploit—Adobe Reader PoC (CVE-2023-21608). The tracker pre-reserves a large shadow region (left), while the PoC attempts to spray allocations into a fixed low-address range (right). When these reservations overlap, the PoC cannot obtain the intended layout and may fail before triggering the vulnerability.
• Figure 3: HALF framework. HALF consists of three cooperative entities: (1) an instrumented process hosting the recording logic, (2) an analysis container executing heavyweight tracking and task handlers, and (3) a kernel monitor coordinating memory reconstruction and synchronization. The kernel monitor provides four mechanisms that jointly enforce transparency and efficiency: (1) process hollowing to construct a minimal container without inheriting the default user-space layout, (2) identical physical-memory mapping to share selected regions while keeping virtual addresses consistent, (3) exception-driven synchronization to move coordination off the hot path, and (4) on-demand memory management to materialize large analysis state without preemptive allocation conflicts.
• Figure 4: Runtime information recording optimization scheme diagram. (1) Fast record-pointer access via GS/TEB slots. (2) Zero-check buffer switching via guard-page faults: touching the guard page transfers control to the kernel monitor, which rotates the writer to a new free buffer.
• Figure 5: Analysis code execution switching. (1) Normal read: load a valid code address and jmp. (2) Sentinel switching: MAGIC_ADDR triggers an exception used by the kernel monitor to rotate buffers. (3) Guard-page switching: touching the buffer-end guard page triggers a page fault for rotation.