BookNet: Book Image Rectification via Cross-Page Attention Network

TL;DR

BookNet addresses the challenging problem of rectifying bound book images by modeling coupled deformations across left and right pages using a dual-branch architecture with cross-page attention. The method predicts three warping flows, $\mathbf{M}_l$, $\mathbf{M}_r$, and $\mathbf{M}_f$, trained under a multi-task $L_1$ loss to ensure page-specific and global geometric consistency, with an efficient learnable upsampling scheme for high-resolution results. To support research, the authors release the Book3D synthetic dataset (56,000 samples) and Book100 real-world benchmark (100 images with scans). Empirical results on Book100 show BookNet outperforms state-of-the-art methods on both geometric and OCR-related metrics, and ablations confirm the value of cross-page attention and joint flow supervision for preserving gutter alignment and text readability in diverse books.

Abstract

Book image rectification presents unique challenges in document image processing due to complex geometric distortions from binding constraints, where left and right pages exhibit distinctly asymmetric curvature patterns. However, existing single-page document image rectification methods fail to capture the coupled geometric relationships between adjacent pages in books. In this work, we introduce BookNet, the first end-to-end deep learning framework specifically designed for dual-page book image rectification. BookNet adopts a dual-branch architecture with cross-page attention mechanisms, enabling it to estimate warping flows for both individual pages and the complete book spread, explicitly modeling how left and right pages influence each other. Moreover, to address the absence of specialized datasets, we present Book3D, a large-scale synthetic dataset for training, and Book100, a comprehensive real-world benchmark for evaluation. Extensive experiments demonstrate that BookNet outperforms existing state-of-the-art methods on book image rectification. Code and dataset will be made publicly available.

Figures (7)

• Figure 1: Rectification paradigm comparison. (a) Conventional single flow for individual pages. (b) Single flow fails on books. (c) Our multi-flow solution effectively rectifies books by predicting separate flows (left, right, full).
• Figure 2: Book3D synthetic dataset generation pipeline and representative samples. Left: Blender rendering workspace showcasing the 3D book modeling environment with parameterized deformation controls. Right: Rendered book samples from diverse arXiv academic papers, demonstrating realistic geometric deformations under varied illumination conditions and viewing angles. Top row shows the rendered synthetic book images, while bottom row displays the corresponding ground truth arXiv paper images.
• Figure 3: Representative samples from the Book100 benchmark dataset illustrating diverse capture conditions and content types. Top row: Distorted book images captured under various real-world conditions exhibiting different deformation patterns, lighting variations, and viewing angles. Bottom row: Corresponding high-quality reference scans obtained using professional overhead document cameras, providing ground truth for evaluation.
• Figure 4: Overview of the proposed BookNet architecture. Given a distorted book image $\mathbf{I}_d$ containing both left and right pages, our method extracts features through a CNN backbone and Transformer encoder. The dual-branch decoder employs a two-stage architecture with cross-page attention mechanisms to process learned queries, generating warping flows $\mathbf{M}_l$, $\mathbf{M}_r$, and $\mathbf{M}_f$ for left page, right page, and complete spread respectively. During training, all three flows are supervised, while inference uses the full flow $\mathbf{M}_f$ for final rectification.
• Figure 5: Visual comparison of flow supervision strategies. Left: left and right flows. Middle: full flow only. Right: joint supervision (ours) achieves better gutter alignment.