Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Securing Monolithic Kernels using Compartmentalization

Soo Yee Lim, Sidhartha Agrawal, Xueyuan Han, David Eyers, Dan O'Keeffe, Thomas Pasquier

TL;DR

This work addresses the security gaps in monolithic kernels by proposing kernel compartmentalization as a practical path forward. It introduces a two-architecture taxonomy—sandbox and safebox—and a framework for identifying boundaries and enforcing isolation, along with a synthesis of two decades of related research. The authors critique current evaluation methodologies and advocate standardized benchmarks to enable meaningful comparisons, while highlighting emerging hardware (e.g., CHERI, CET) that could substantially reduce overhead. The findings suggest that incremental, hardware-assisted compartmentalization can make existing kernels safer with manageable engineering effort, guiding future research toward more rigorous, comparable assessments and adoption in production systems.

Abstract

Monolithic operating systems, where all kernel functionality resides in a single, shared address space, are the foundation of most mainstream computer systems. However, a single flaw, even in a non-essential part of the kernel (e.g., device drivers), can cause the entire operating system to fall under an attacker's control. Kernel hardening techniques might prevent certain types of vulnerabilities, but they fail to address a fundamental weakness: the lack of intra-kernel security that safely isolates different parts of the kernel. We survey kernel compartmentalization techniques that define and enforce intra-kernel boundaries and propose a taxonomy that allows the community to compare and discuss future work. We also identify factors that complicate comparisons among compartmentalized systems, suggest new ways to compare future approaches with existing work meaningfully, and discuss emerging research directions.

Securing Monolithic Kernels using Compartmentalization

TL;DR

This work addresses the security gaps in monolithic kernels by proposing kernel compartmentalization as a practical path forward. It introduces a two-architecture taxonomy—sandbox and safebox—and a framework for identifying boundaries and enforcing isolation, along with a synthesis of two decades of related research. The authors critique current evaluation methodologies and advocate standardized benchmarks to enable meaningful comparisons, while highlighting emerging hardware (e.g., CHERI, CET) that could substantially reduce overhead. The findings suggest that incremental, hardware-assisted compartmentalization can make existing kernels safer with manageable engineering effort, guiding future research toward more rigorous, comparable assessments and adoption in production systems.

Abstract

Monolithic operating systems, where all kernel functionality resides in a single, shared address space, are the foundation of most mainstream computer systems. However, a single flaw, even in a non-essential part of the kernel (e.g., device drivers), can cause the entire operating system to fall under an attacker's control. Kernel hardening techniques might prevent certain types of vulnerabilities, but they fail to address a fundamental weakness: the lack of intra-kernel security that safely isolates different parts of the kernel. We survey kernel compartmentalization techniques that define and enforce intra-kernel boundaries and propose a taxonomy that allows the community to compare and discuss future work. We also identify factors that complicate comparisons among compartmentalized systems, suggest new ways to compare future approaches with existing work meaningfully, and discuss emerging research directions.
Paper Structure (41 sections, 7 figures, 2 tables)

This paper contains 41 sections, 7 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Kernel Hardening Techniques
  4. Alternative Kernel Designs
  5. Software Compartmentalization
  6. Overview of Kernel Compartmentalization Approaches
  7. Sandbox Architecture
  8. Fault Isolation
  9. Fault Resistance
  10. Vulnerability Isolation
  11. Safebox Architecture
  12. Security State Isolation
  13. Secure Policy Enforcement
  14. Summary
  15. Compartment Boundaries
  16. ...and 26 more sections

Figures (7)

  • Figure 1: Alternative kernel designs
  • Figure 2: Threat models of kernel compartmentalization architectures.
  • Figure 3: Different user/kernel compartmentalization approaches. In Microdriver, the user-mode driver is written in the same C language as the kernel-mode driver; in Decaf, the user-mode driver can be written in languages other than C.
  • Figure 4: Different hypervisor-based compartmentalization approaches.
  • Figure 5: An example of address space separation on ARMv7 in SKEE azab_skee_2016
  • ...and 2 more figures