OptiSQL: Executable SQL Generation from Optical TokensOptiSQL: Executable SQL Generation from Optical Tokens

TL;DR

OptiSQL tackles the problem of executable SQL generation when tables appear only as images rather than text. It introduces a vision-driven pipeline that converts a table image into a compact sequence of optical tokens via a frozen OCR-oriented visual encoder and then fine-tunes a pretrained decoder to generate executable SQL under a fixed token budget. By isolating representation sufficiency through encoder freezing and exploring robustness with token budgets and visual perturbations, OptiSQL demonstrates strong execution accuracy with order-of-magnitude input compression compared to textual encodings. On a visualized Spider 2.0-Snow dataset, OptiSQL achieves competitive executable performance while avoiding the errors and long inputs typical of OCR-based pipelines, highlighting the practical value of optical token interfaces under long-context constraints. The work suggests that optical tokenization can serve as a robust and efficient alternative to text-centric inputs for executable reasoning in document- and web-based scenarios and points to future work on multi-table queries and broader structured-generation tasks.

Abstract

Executable SQL generation is typically studied in text-to-SQL settings, where tables are provided as fully linearized textual schemas and contents. While effective, this formulation assumes access to structured text and incurs substantial token overhead, which is misaligned with many real-world scenarios where tables appear as visual artifacts in documents or webpages. We investigate whether compact optical representations can serve as an efficient interface for executable semantic parsing. We present OptiSQL, a vision-driven framework that generates executable SQL directly from table images and natural language questions using compact optical tokens. OptiSQL leverages an OCR-oriented visual encoder to compress table structure and content into a small set of optical tokens and fine-tunes a pretrained decoder for SQL generation while freezing the encoder to isolate representation sufficiency. Experiments on a visualized version of Spider 2.0-Snow show that OptiSQL retains strong execution accuracy while reducing table input tokens by an order of magnitude. Robustness analyses further demonstrate that optical tokens preserve essential structural information under visual perturbations.

Figures (5)

• Figure 1: OptiSQL generates executable SQL directly from table images using compact optical tokens. A frozen OCR-oriented visual encoder converts a table image into a fixed-size sequence of optical tokens, which are combined with a natural language question and processed by a trainable decoder. By operating under an explicit token budget, OptiSQL enables an efficient and robust alternative to text-based table encodings.
• Figure 2: An example of SQL canonicalization used for EX-Can. The procedure normalizes keyword case, whitespace, punctuation spacing, and reorders flat AND conditions within the same logical level.
• Figure 3: Different visual renderings of the same table content. Variations include changes in font style, color, layout, borders, and spacing, while preserving identical schema and cell values. These styles are used to study robustness to superficial visual changes.
• Figure 4: Visual grounding diagnostics for OptiSQL (FrozenEnc). EXAcc and EX-Can under clean and perturbed visual inputs. Bars show accuracy. Performance degrades sharply when visual grounding is disrupted.
• Figure 5: Execution accuracy and inference latency as a function of the optical token budget. Increasing the number of optical tokens improves execution accuracy with diminishing returns beyond 256 tokens, while latency grows approximately linearly, revealing a clear efficiency-accuracy trade-off.