A Preliminary Agentic Framework for Matrix Deflation

TL;DR

This work addresses threshold-dependent matrix deflation by introducing an agentic framework where a solver LLM proposes rank-1 SVD updates and a Vision-Language Model evaluator validates each step, guided by lightweight permutations to reveal structure and a threshold-free stopping rule. The method bypasses fixed Frobenius-norm criteria, using $A \leftarrow \max(0, A_1)$ and ICL-driven updates, and is evaluated on Digits, CIFAR-10, and synthetic matrices with and without noise. Across settings, the agentic approach yields competitive residuals relative to numerical baselines, with permutation strategies enhancing performance on natural images and more conservative updates increasing steps. The work demonstrates a viable, fully agentic alternative to classical deflation, with release of code and potential for scaling and broader permutation strategies in future research, contributing to interpretable, sequential matrix decomposition without rigid thresholds.

Abstract

Can a small team of agents peel a matrix apart, one rank-1 slice at a time? We propose an agentic approach to matrix deflation in which a solver Large Language Model (LLM) generates rank-1 Singular Value Decomposition (SVD) updates and a Vision Language Model (VLM) accepts or rejects each update and decides when to stop, eliminating fixed norm thresholds. Solver stability is improved through in-context learning (ICL) and types of row/column permutations that expose visually coherent structure. We evaluate on Digits ($8{\times}8$), CIFAR-10 ($32{\times}32$ grayscale), and synthetic ($16{\times}16$) matrices with and without Gaussian noise. In the synthetic noisy case, where the true construction rank $k$ is known, numerical deflation provides the noise target and our best agentic configuration differs by only $1.75$ RMSE of the target. For Digits and CIFAR-10, targets are defined by deflating until the Frobenius norm reaches $10\%$ of the original. Across all settings, our agent achieves competitive results, suggesting that fully agentic, threshold-free deflation is a viable alternative to classical numerical algorithms.

Figures (3)

• Figure 1: CIFAR-10 example comparing unpermuted and permuted (Sort, GroupNTeach-Block) rank-1 results.
• Figure 2: LLM-proposed rank-1 updates are VLM-validated and subtracted until deflation stops.
• Figure 3: Sort (CIFAR-10). Top row = unpermuted view, bottom row = permuted view.