Eleven Primitives and Three Gates: The Universal Structure of Computational Imaging

Abstract

Computational imaging systems -- from coded-aperture cameras to cryo-electron microscopes -- span five carrier families yet share a hidden structural simplicity. We prove that every imaging forward model decomposes into a directed acyclic graph over exactly 11 physically typed primitives (Finite Primitive Basis Theorem) -- a sufficient and minimal basis that provides a compositional language for designing any imaging modality. We further prove that every reconstruction failure has exactly three independent root causes: information deficiency, carrier noise, and operator mismatch (Triad Decomposition). The three gates map to the system lifecycle: Gates 1 and 2 guide design (sampling geometry, carrier selection); Gate 3 governs deployment-stage calibration and drift correction. Validation across 12 modalities and all five carrier families confirms both results, with +0.8 to +13.9 dB recovery on deployed instruments. Together, the 11 primitives and 3 gates establish the first universal grammar for designing, diagnosing, and correcting computational imaging systems.

Figures (5)

• Figure 1: The universal grammar of computational imaging.a, Any computational imaging system---from coded-aperture cameras to cryo-electron microscopes---serves as input. b, The system is composed as a typed directed acyclic graph (DAG) over 11 universal primitives (\ref{['fig:operatorgraph']}); shown here for CASSI as an example. The grammar enables two workflows. Top (existing instruments):c, The Triad Decomposition diagnoses the DAG through three independent gates: information deficiency (Gate 1), carrier budget (Gate 2), and operator mismatch (Gate 3). d, Based on the dominant gate, a targeted correction is applied: sampling redesign (Gate 1), source/detector improvement (Gate 2), or operator calibration (Gate 3). e, The existing solver is re-run with the corrected operator, recovering $+0.8$ to $+13.9$ dB without retraining. Bottom (new instruments):f, The same three gates guide the design of a new modality from first principles: Gate 1 determines sampling geometry, Gate 2 guides carrier and source selection, and Gate 3 predicts the calibration accuracy required for a target reconstruction quality. g, The output is an optimized new system specification ready to build. Together, the 11 primitives (alphabet) and 3 gates (rules) form a universal grammar for designing, diagnosing, and correcting any imaging system.
• Figure 2: 11 primitives: OperatorGraph IR, Physics Fidelity Ladder, and basis completeness.a, The 11 universal primitives grouped by physical role: generation (Propagate $P$), encoding (Modulate $M$, Project $\Pi$, Convolve $C$, Scatter $R$, Transform $\Lambda$), transform (Encode $F$, Accumulate $\Sigma$), and detection (Sample $S$, Disperse $W$, Detect $D$). Each primitive carries a typed signature mapping its input to its output (e.g., $P$: free-space propagation; $D$: detector response). b, Example OperatorGraph DAGs for three modalities: CASSI (photon), MRI (spin), and CT (X-ray photon). Each node wraps a primitive operator; edges define data flow. The symbols in panel b correspond directly to the primitive labels in panel a. c, Fidelity levels: forward models range from linear shift-invariant (simplest) through nonlinear and full-wave; all 11 primitives apply uniformly across every level. d, Basis-growth saturation: the number of distinct primitives $K$ required to express $N$ registered modalities saturates at $K{=}11$ for $N \geq 35$; no additional primitive has been needed for the 135 subsequent modalities across 23 physical categories.
• Figure 3: Triad Decomposition: all three gates validated with real data.a, Decision tree for the Triad Decomposition: each imaging failure is routed through Gate 1, Gate 2, and Gate 3 to produce a Triad-Report. b, Three-gate degradation and correction heatmap. G1 (blue): $\Delta$PSNR under extreme compression (Table S12; all 12 modalities). G2 (amber): $\Delta$PSNR under extreme noise (Table S13; all 12 modalities). G3 (green): correction gain $\Delta$PSNR under standard mismatch conditions (Table S1; all 12 modalities). G1/G2 degrade at extreme conditions confirming they are real failure modes; G3 dominates under standard deployment because Gates 1 and 2 were already optimized at design time. c, Recovery ratio $\rho$ distribution across all 12 validated modalities, showing that autonomous calibration recovers a substantial fraction of the oracle correction ceiling across all five carrier families.
• Figure 4: 4-Scenario Protocol across 6 representative modalities. Each panel shows PSNR (dB) for the four scenarios: Sc. I (ideal operator, green), Sc. II (mismatched operator, red), Sc. III (oracle correction, blue), and Sc. IV (autonomous calibration, purple). Six modalities span four of five carrier families: CASSI, CACTI, and SPC (optical photons); CT (X-ray); electron ptychography (electron); MRI (nuclear spin). For low-dimensional mismatch (CT, electron ptychography, MRI), Sc. IV $\approx$ Sc. III $\approx$ Sc. I, confirming near-complete recovery. For multi-parameter mismatch (CASSI), recovery reaches 85%; CACTI and SPC achieve 100% and 86%, respectively (Supplementary Table S9). Sc. I PSNR from Table S3; Sc. II/III from Table S1.
• Figure 5: Hardware validation on real CASSI and CACTI instruments.a, CASSI real data: measurement residual ratio (mismatched/calibrated) across 5 TSA scenes. GAP-TV shows $1.8\times$ mean ratio. b, CACTI real data: residual ratio across 4 scenes. GAP-TV shows $10.4\times$ mean ratio; PnP-FFDNet shows $2.0\times$. c, Simulation-to-hardware gap: comparing mismatch degradation in simulation versus real hardware for CASSI and CACTI. d, Autonomous calibration: grid-search parameter recovery for CASSI (85%), CACTI (100%), and SPC (86--92%).

Theorems & Definitions (5)

• Definition 1: Imaging Operator Class
• Definition 2: $\varepsilon$-Approximate Representation
• Theorem 1: Finite Primitive Basis
• Definition 3: Triad Decomposition
• Theorem 2: Gate 3 Dominance