Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-Tuning Without Forgetting: Adaptation of YOLOv8 Preserves COCO Performance

Vishal Gandhi, Sagar Gandhi

TL;DR

This study investigates how deeply to fine-tune a pre-trained YOLOv8n detector when adapting to a fine-grained fruit dataset. By progressively unfreezing backbone layers down to 22, 15, and 10 and evaluating on both the target task and COCO, it reveals that deeper fine-tuning yields about a 10 percentage point gain on the target task while causing negligible forgetting on the source task. The results demonstrate that mid-to-late backbone features are crucial for fine-grained discrimination and can be adapted without significant degradation of general capabilities. These findings encourage broader exploration of deeper fine-tuning in transfer learning for complex domains, though they are currently limited to a single model and dataset. Future work should test generality across architectures and tasks to validate these trends further.

Abstract

The success of large pre-trained object detectors hinges on their adaptability to diverse downstream tasks. While fine-tuning is the standard adaptation method, specializing these models for challenging fine-grained domains necessitates careful consideration of feature granularity. The critical question remains: how deeply should the pre-trained backbone be fine-tuned to optimize for the specialized task without incurring catastrophic forgetting of the original general capabilities? Addressing this, we present a systematic empirical study evaluating the impact of fine-tuning depth. We adapt a standard YOLOv8n model to a custom, fine-grained fruit detection dataset by progressively unfreezing backbone layers (freeze points at layers 22, 15, and 10) and training. Performance was rigorously evaluated on both the target fruit dataset and, using a dual-head evaluation architecture, on the original COCO validation set. Our results demonstrate unequivocally that deeper fine-tuning (unfreezing down to layer 10) yields substantial performance gains (e.g., +10\% absolute mAP50) on the fine-grained fruit task compared to only training the head. Strikingly, this significant adaptation and specialization resulted in negligible performance degradation (<0.1\% absolute mAP difference) on the COCO benchmark across all tested freeze levels. We conclude that adapting mid-to-late backbone features is highly effective for fine-grained specialization. Critically, our results demonstrate this adaptation can be achieved without the commonly expected penalty of catastrophic forgetting, presenting a compelling case for exploring deeper fine-tuning strategies, particularly when targeting complex domains or when maximizing specialized performance is paramount.

Fine-Tuning Without Forgetting: Adaptation of YOLOv8 Preserves COCO Performance

TL;DR

This study investigates how deeply to fine-tune a pre-trained YOLOv8n detector when adapting to a fine-grained fruit dataset. By progressively unfreezing backbone layers down to 22, 15, and 10 and evaluating on both the target task and COCO, it reveals that deeper fine-tuning yields about a 10 percentage point gain on the target task while causing negligible forgetting on the source task. The results demonstrate that mid-to-late backbone features are crucial for fine-grained discrimination and can be adapted without significant degradation of general capabilities. These findings encourage broader exploration of deeper fine-tuning in transfer learning for complex domains, though they are currently limited to a single model and dataset. Future work should test generality across architectures and tasks to validate these trends further.

Abstract

The success of large pre-trained object detectors hinges on their adaptability to diverse downstream tasks. While fine-tuning is the standard adaptation method, specializing these models for challenging fine-grained domains necessitates careful consideration of feature granularity. The critical question remains: how deeply should the pre-trained backbone be fine-tuned to optimize for the specialized task without incurring catastrophic forgetting of the original general capabilities? Addressing this, we present a systematic empirical study evaluating the impact of fine-tuning depth. We adapt a standard YOLOv8n model to a custom, fine-grained fruit detection dataset by progressively unfreezing backbone layers (freeze points at layers 22, 15, and 10) and training. Performance was rigorously evaluated on both the target fruit dataset and, using a dual-head evaluation architecture, on the original COCO validation set. Our results demonstrate unequivocally that deeper fine-tuning (unfreezing down to layer 10) yields substantial performance gains (e.g., +10\% absolute mAP50) on the fine-grained fruit task compared to only training the head. Strikingly, this significant adaptation and specialization resulted in negligible performance degradation (<0.1\% absolute mAP difference) on the COCO benchmark across all tested freeze levels. We conclude that adapting mid-to-late backbone features is highly effective for fine-grained specialization. Critically, our results demonstrate this adaptation can be achieved without the commonly expected penalty of catastrophic forgetting, presenting a compelling case for exploring deeper fine-tuning strategies, particularly when targeting complex domains or when maximizing specialized performance is paramount.
Paper Structure (24 sections, 2 figures, 2 tables)

This paper contains 24 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Transfer Learning in Object Detection
  4. Fine-Grained Visual Categorization (FGVC)
  5. Catastrophic Forgetting and Continual Learning
  6. YOLO Object Detection Models
  7. Methodology
  8. Base Model
  9. Target Dataset: Fine-Grained Fruits
  10. Fine-tuning Protocol
  11. Evaluation Strategy
  12. Target Task (Fruit Detection):
  13. Source Task (COCO - Catastrophic Forgetting):
  14. Experiments & Results
  15. Experimental Setup and Metrics
  16. ...and 9 more sections

Figures (2)

  • Figure 1: Target Fine-Grained Fruit Dataset Characteristics. (a) Distribution of object instances per class. (b) Correlogram showing bounding box center (x, y), width, and height distributions and relationships.
  • Figure 2: Precision-Recall curves on the fruit validation set (all classes, IoU=0.5) for different fine-tuning freeze levels. The area under the curve (mAP@0.5) increases as more backbone layers are fine-tuned (Freeze=10 > Freeze=15 > Freeze=22).