Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Does YOLO Really Need to See Every Training Image in Every Epoch?

Xingxing Xie, Jiahua Dong, Junwei Han, Gong Cheng

Abstract

YOLO detectors are known for their fast inference speed, yet training them remains unexpectedly time-consuming due to their exhaustive pipeline that processes every training image in every epoch, even when many images have already been sufficiently learned. This stands in clear contrast to the efficiency suggested by the ``You Only Look Once'' philosophy. This naturally raises an important question: \textit{Does YOLO really need to see every training image in every epoch?} To explore this, we propose an Anti-Forgetting Sampling Strategy (AFSS) that dynamically determines which images should be used and which can be skipped during each epoch, allowing the detector to learn more effectively and efficiently. Specifically, AFSS measures the learning sufficiency of each training image as the minimum of its detection recall and precision, and dynamically categorizes training images into easy, medium, or hard levels accordingly. Easy training images are sparsely resampled during training in a continuous review manner, with priority given to those that have not been used for a long time to reduce redundancy and prevent forgetting. Moderate training images are partially selected, prioritizing recently unused ones and randomly choosing the rest from unselected images to ensure coverage and prevent forgetting. Hard training images are fully sampled in every epoch to ensure sufficient learning. The learning sufficiency of each training image is periodically updated, enabling detectors to adaptively shift its focus toward the informative training images over time while progressively discarding redundant ones. On widely used natural image detection benchmarks (MS COCO 2017 and PASCAL VOC 2007) and remote sensing detection datasets (DOTA-v1.0 and DIOR-R), AFSS achieves more than $1.43\times$ training speedup for YOLO-series detectors while also improving accuracy.

Does YOLO Really Need to See Every Training Image in Every Epoch?

Abstract

YOLO detectors are known for their fast inference speed, yet training them remains unexpectedly time-consuming due to their exhaustive pipeline that processes every training image in every epoch, even when many images have already been sufficiently learned. This stands in clear contrast to the efficiency suggested by the ``You Only Look Once'' philosophy. This naturally raises an important question: \textit{Does YOLO really need to see every training image in every epoch?} To explore this, we propose an Anti-Forgetting Sampling Strategy (AFSS) that dynamically determines which images should be used and which can be skipped during each epoch, allowing the detector to learn more effectively and efficiently. Specifically, AFSS measures the learning sufficiency of each training image as the minimum of its detection recall and precision, and dynamically categorizes training images into easy, medium, or hard levels accordingly. Easy training images are sparsely resampled during training in a continuous review manner, with priority given to those that have not been used for a long time to reduce redundancy and prevent forgetting. Moderate training images are partially selected, prioritizing recently unused ones and randomly choosing the rest from unselected images to ensure coverage and prevent forgetting. Hard training images are fully sampled in every epoch to ensure sufficient learning. The learning sufficiency of each training image is periodically updated, enabling detectors to adaptively shift its focus toward the informative training images over time while progressively discarding redundant ones. On widely used natural image detection benchmarks (MS COCO 2017 and PASCAL VOC 2007) and remote sensing detection datasets (DOTA-v1.0 and DIOR-R), AFSS achieves more than training speedup for YOLO-series detectors while also improving accuracy.
Paper Structure (18 sections, 16 equations, 4 figures, 5 tables)

This paper contains 18 sections, 16 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Learning Sufficiency Metric
  5. Continuous Review
  6. Short-Term Coverage
  7. State Update
  8. Experiments
  9. Datasets and Evaluation Protocols
  10. Implementation Details
  11. Main Results
  12. Ablation Studies
  13. Comparison with Other Training Strategies
  14. Effect of Individual Modules
  15. Comparison of Learning Sufficiency Metrics
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Comparison of YOLO11s with and without AFSS in terms of the training images used and the corresponding training efficiency and accuracy on MS COCO 2017. (a) Training data used per epoch: AFSS adaptively selects the images used for training, progressively reducing the number of images utilized over time, whereas the vanilla YOLO11s employs the full training set in every epoch; (b) Training efficiency and accuracy: AFSS accelerates YOLO11s training by 1.54 $\times$ while improving detection accuracy.
  • Figure 2: Overview of the proposed Anti-Forgetting Sampling Strategy (AFSS) at the $t$-th epoch of training.
  • Figure 3: Changes in the number of samples at easy, moderate, and hard levels during the training of YOLO11s with AFSS.
  • Figure 4: An example illustrating the learning performance of YOLO11s and YOLO11s+AFSS at the 100th, 300th, and 600th training epochs on the hard image.