AesCrop: Aesthetic-driven Cropping Guided by Composition

TL;DR

AesCrop tackles aesthetic-driven image cropping by introducing a composition-aware hybrid framework that fuses a VMamba encoder with a transformer-based decoder, augmented by the Mamba Composition Attention Bias (MCAB). MCAB injects composition priors derived from Grad-CAM-based heatmaps into cross-attention, directing crops toward compositionally salient regions while generating multiple proposals with quality scores. The model is trained with a Hungarian matching strategy and a composite loss, and is pretrained on composition and object-detection datasets to compensate for limited cropping data. Quantitative results on GAICv2 show state-of-the-art performance with notable improvements in Acc$_{1/5}$ and Acc$_{1/10}$, and qualitative analyses demonstrate more compositionally faithful crops. These findings highlight the practical value of explicit composition guidance in automated cropping for thumbnails and view recommendations, with MCAB serving as a key mechanism to encode photographic rules into attention.

Abstract

Aesthetic-driven image cropping is crucial for applications like view recommendation and thumbnail generation, where visual appeal significantly impacts user engagement. A key factor in visual appeal is composition--the deliberate arrangement of elements within an image. Some methods have successfully incorporated compositional knowledge through evaluation-based and regression-based paradigms. However, evaluation-based methods lack globality while regression-based methods lack diversity. Recently, hybrid approaches that integrate both paradigms have emerged, bridging the gap between these two to achieve better diversity and globality. Notably, existing hybrid methods do not incorporate photographic composition guidance, a key attribute that defines photographic aesthetics. In this work, we introduce AesCrop, a composition-aware hybrid image-cropping model that integrates a VMamba image encoder, augmented with a novel Mamba Composition Attention Bias (MCAB) and a transformer decoder to perform end-to-end rank-based image cropping, generating multiple crops along with the corresponding quality scores. By explicitly encoding compositional cues into the attention mechanism, MCAB directs AesCrop to focus on the most compositionally salient regions. Extensive experiments demonstrate that AesCrop outperforms current state-of-the-art methods, delivering superior quantitative metrics and qualitatively more pleasing crops.

Figures (7)

• Figure 1: AesCrop is a composition-aware, hybrid image-cropping model that incorporates cues from multiple composition classes via the Mamba Composition Attention Bias (MCAB) and performs end-to-end rank-based image cropping.
• Figure 2: The architecture of AesCrop. The input image is processed by dual-stream encoders: (1) a module that generates the Mamba Composition Attention Bias (MCAB) and (2) an encoder that produces image embeddings. A decoder then refines learnable queries via self-attention, aggregates those queries with the image embeddings through MCAB-modulated cross-attention, and processes the result through feed-forward networks. The final embeddings are passed to prediction heads that output crop boxes and quality scores.
• Figure 3: Class activation maps of nine images representing distinct composition classes.
• Figure 4: Visual comparison of top three ground truth crops with predicted crops.
• Figure 5: (left) Source image with ground truth (green box), and AesCrop's best predicted crops (red box) and (right) the corresponding composition attention priors generated by MCAB.