RAWDet-7: A Multi-Scenario Benchmark for Object Detection and Description on Quantized RAW Images

TL;DR

RAWDet-7 tackles the gap in evaluating object detection and object description directly on RAW sensor data by unifying four existing RAW datasets into ~32k images, annotated across seven MS-COCO/LVIS-style categories. It couples dense detection annotations with high-resolution sRGB-derived object captions and evaluates performance under realistic low-bit quantization (4, 6, 8 bits) using multiple input-scaling strategies, including linear, logarithmic, learnable gamma, and log+gamma mappings. The work demonstrates that carefully designed RAW preprocessing (notably log+$\gamma$) can approach or surpass 8-bit sRGB performance for detection and can yield high-quality object descriptions that closely track high-resolution references, even when using LVLMs in zero-shot settings. These findings support quantization-aware sensor-task co-design and provide a versatile benchmark for RAW-based vision research, while acknowledging limitations in coverage and annotation costs.

Abstract

Most vision models are trained on RGB images processed through ISP pipelines optimized for human perception, which can discard sensor-level information useful for machine reasoning. RAW images preserve unprocessed scene data, enabling models to leverage richer cues for both object detection and object description, capturing fine-grained details, spatial relationships, and contextual information often lost in processed images. To support research in this domain, we introduce RAWDet-7, a large-scale dataset of ~25k training and 7.6k test RAW images collected across diverse cameras, lighting conditions, and environments, densely annotated for seven object categories following MS-COCO and LVIS conventions. In addition, we provide object-level descriptions derived from the corresponding high-resolution sRGB images, facilitating the study of object-level information preservation under RAW image processing and low-bit quantization. The dataset allows evaluation under simulated 4-bit, 6-bit, and 8-bit quantization, reflecting realistic sensor constraints, and provides a benchmark for studying detection performance, description quality & detail, and generalization in low-bit RAW image processing. Dataset & code upon acceptance.

Figures (15)

• Figure 1: Comparing ground truth annotations provided in the original datasets and the new ones proposed in RAWDet-7. Our proposed annotations are more fine-grained, as seen for PASCAL RAW, which originally annotated only one instance of the cars in the image; we now annotate all the other instances of cars in the image (with a 20% overlap threshold i.e. at least 20% area of the bounding box should be non-overlapping). RAW NOD (-Nikon and -Sony), RAOD (-Day and -Night) original annotations contain hallucinations as seen here for RAW NOD, which hallucinates a bicycle in the center right of the frame. Original annotations even contain some misclassifications, as seen for RAOD, which misclassified a motorcycle as a person. RAWDet-7(bottom) overcomes these drawbacks.
• Figure 2: Benchmarking performance on RAWDet-7. Baselines such as logarithmic quantization and jointly learnt $\gamma$ improve results across quantization levels and architectures.
• Figure 3: Results for training on combined RAWDet-7 when evaluated on each subset, vs. training only on a subset and evaluated the subset's test set. Combining improves results on all subsets. The model used is Faster RCNN.
• Figure 4: Precision Recall Curves for RAOD. "Old" refers to the annotations as proposed in the original dataset, whereas "New" means the annotations in RAWDet-7. All the models are trained on FasterRCNN, for 8-bit quantization and jointly learnt with 1 gamma. As mentioned in the legend, the keys are "(Train Dataset - Evaluation Dataset)".
• Figure 5: Visualizing predictions made by MM-Grounding-DINO on RAWDet-7 at 6-bit quantization, using different methods for predictions on sRGB images and the expected Ground Truth predictions. Her,e for methods $\gamma$ and Log + $\gamma$, we finetune only the $\gamma$ parameter while keeping the VLM frozen; for the other methods, we evaluate zero-shot. We randomly sample three distinct scenarios to demonstrate the versatility of the proposed RAWDet-7.