FBS: Modeling Native Parallel Reading inside a Transformer
Tongxi Wang
TL;DR
FBS introduces a tripartite, trainable reading pipeline inside a causal Transformer: Parafovea-Attention Window (PAW) provides content-adaptive lookahead, Chunk-Head (CH) builds a chunk-level semantic channel, and Skip-Gate (SG) dynamically skips layers to allocate compute where it matters. By maintaining causality and KV-cache compatibility, FBS realizes a preview→chunk→skim loop that results in improved quality with reduced compute, demonstrated on Chinese–English benchmarks with parameter-matched baselines. The paper provides extensive ablations, RL-fine-tuning analyses, and mechanism studies showing that PAW’s foresight, CH’s chunk semantics, and SG’s gating collectively drive efficiency gains while preserving accuracy, with robust performance across tasks and hyperparameters. These findings suggest a practical path to faster, more human-like reading in LLMs without increasing model size, while highlighting deployment considerations for structure-sensitive outputs and long-context generation.
Abstract
Large language models (LLMs) excel across many tasks, yet inference is still dominated by strictly token-by-token autoregression. Existing acceleration methods largely patch this pipeline and miss core human-reading ingredients: content-adaptive foresight, chunk-structure-aware compute allocation, and train--test consistency for preview/skimming. We propose the \textbf{Fovea-Block-Skip Transformer} (FBS), which injects a causal, trainable loop into Transformers via Parafovea-Attention Window (PAW), Chunk-Head (CH), and Skip-Gate (SG). Across diverse benchmarks, FBS improves the quality-efficiency trade-off without increasing parameters, and ablations show the three modules are complementary.