Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RADAR: Exposing Unlogged NoSQL Operations

Mahfuzul I. Nissan, James Wagner

TL;DR

RADAR tackles the problem of detecting unattributed operations in NoSQL databases where logs can be tampered with or suppressed. It combines Automated NoSQL Carver (ANOC) storage carving with cross-referenced, normalized application JSONL audit logs to identify delta artifacts that lack log explanations, using two detection modes tailored to storage architecture. The authors formalize algorithms for detecting unattributed inserts, deletes, and updates, and validate RADAR across ten engines (KV and document stores) under log-evasion scenarios with throughputs from $31.7$ to $397$ MB/min. This log-independent forensic framework offers a robust defense-in-depth against insider threats, enabling trustworthy NoSQL forensics in both on-premises and cloud environments.

Abstract

The widespread adoption of NoSQL databases has made digital forensics increasingly difficult as storage formats are diverse and often opaque, and audit logs cannot be assumed trustworthy when privileged insiders, such as DevOps or administrators, can disable, suppress, or manipulate logging to conceal activity. We present RADAR (Record & Artifact Detection, Alignment & Reporting), a log-adversary-aware framework that derives forensic ground truth by cross-referencing low-level storage artifacts against high-level application logs. RADAR analyzes artifacts reconstructed by the Automated NoSQL Carver (ANOC), which infers layouts and carves records directly from raw disk bytes, bypassing database APIs and the management system entirely, thereby treating physical storage as the independent evidence source. RADAR then reconciles carved artifacts with the audit log to identify delta artifacts such as unlogged insertions, silent deletions, and field-level updates that exist on disk but are absent from the logical history. We evaluate RADAR across ten NoSQL engines, including BerkeleyDB, LMDB, MDBX, etcd, ZODB, Durus, LiteDB, Realm, RavenDB, and NitriteDB, spanning key-value and document stores and multiple storage designs, e.g., copy-on-write/MVCC, B/B+ tree, and append-only. Under log-evasion scenarios, such as log suppression and post-maintenance attacks, including cases where historical bytes are pruned, RADAR consistently exposes unattributed operations while sustaining 31.7-397 MB/min processing throughput, demonstrating the feasibility of log-independent, trustworthy NoSQL forensics.

RADAR: Exposing Unlogged NoSQL Operations

TL;DR

RADAR tackles the problem of detecting unattributed operations in NoSQL databases where logs can be tampered with or suppressed. It combines Automated NoSQL Carver (ANOC) storage carving with cross-referenced, normalized application JSONL audit logs to identify delta artifacts that lack log explanations, using two detection modes tailored to storage architecture. The authors formalize algorithms for detecting unattributed inserts, deletes, and updates, and validate RADAR across ten engines (KV and document stores) under log-evasion scenarios with throughputs from to MB/min. This log-independent forensic framework offers a robust defense-in-depth against insider threats, enabling trustworthy NoSQL forensics in both on-premises and cloud environments.

Abstract

The widespread adoption of NoSQL databases has made digital forensics increasingly difficult as storage formats are diverse and often opaque, and audit logs cannot be assumed trustworthy when privileged insiders, such as DevOps or administrators, can disable, suppress, or manipulate logging to conceal activity. We present RADAR (Record & Artifact Detection, Alignment & Reporting), a log-adversary-aware framework that derives forensic ground truth by cross-referencing low-level storage artifacts against high-level application logs. RADAR analyzes artifacts reconstructed by the Automated NoSQL Carver (ANOC), which infers layouts and carves records directly from raw disk bytes, bypassing database APIs and the management system entirely, thereby treating physical storage as the independent evidence source. RADAR then reconciles carved artifacts with the audit log to identify delta artifacts such as unlogged insertions, silent deletions, and field-level updates that exist on disk but are absent from the logical history. We evaluate RADAR across ten NoSQL engines, including BerkeleyDB, LMDB, MDBX, etcd, ZODB, Durus, LiteDB, Realm, RavenDB, and NitriteDB, spanning key-value and document stores and multiple storage designs, e.g., copy-on-write/MVCC, B/B+ tree, and append-only. Under log-evasion scenarios, such as log suppression and post-maintenance attacks, including cases where historical bytes are pruned, RADAR consistently exposes unattributed operations while sustaining 31.7-397 MB/min processing throughput, demonstrating the feasibility of log-independent, trustworthy NoSQL forensics.
Paper Structure (48 sections, 2 equations, 5 figures, 7 tables, 4 algorithms)

This paper contains 48 sections, 2 equations, 5 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Forensic Recovery from Database Storage
  4. Log Analysis, Integrity, and Tamper Detection
  5. A Unified Forensic Approach
  6. Threat Model
  7. RADAR Framework & Data Sources
  8. Overall Architecture
  9. Forensic Data Sources
  10. Application Audit Log (JSONL)
  11. Schema
  12. Canonicalization
  13. Excluded Engine-Specific Logs (WAL/Journals)
  14. Acquisition & Carving (ANOC)
  15. Threat Detection Methodology
  16. ...and 33 more sections

Figures (5)

  • Figure 1: Threat model
  • Figure 2: File-write event classification and the resulting snapshot decision.
  • Figure 3: Architecture of RADAR
  • Figure 4: Unattributed Operations (Key-Value Store)
  • Figure 5: Unattributed Operations (Document Store)