Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Task Adaptation of Reinforcement Learning-based NAS Agents through Transfer Learning

Amber Cassimon, Siegfried Mercelis, Kevin Mets

TL;DR

It is found that pretraining an agent on one task benefits the performance of the agent in another task in all but 1 task when considering final performance, and the training procedure for an agent can be shortened significantly by pretraining it on another task.

Abstract

Recently, a novel paradigm has been proposed for reinforcement learning-based NAS agents, that revolves around the incremental improvement of a given architecture. We assess the abilities of such reinforcement learning agents to transfer between different tasks. We perform our evaluation using the Trans-NASBench-101 benchmark, and consider the efficacy of the transferred agents, as well as how quickly they can be trained. We find that pretraining an agent on one task benefits the performance of the agent in another task in all but 1 task when considering final performance. We also show that the training procedure for an agent can be shortened significantly by pretraining it on another task. Our results indicate that these effects occur regardless of the source or target task, although they are more pronounced for some tasks than for others. Our results show that transfer learning can be an effective tool in mitigating the computational cost of the initial training procedure for reinforcement learning-based NAS agents.

Task Adaptation of Reinforcement Learning-based NAS Agents through Transfer Learning

TL;DR

It is found that pretraining an agent on one task benefits the performance of the agent in another task in all but 1 task when considering final performance, and the training procedure for an agent can be shortened significantly by pretraining it on another task.

Abstract

Recently, a novel paradigm has been proposed for reinforcement learning-based NAS agents, that revolves around the incremental improvement of a given architecture. We assess the abilities of such reinforcement learning agents to transfer between different tasks. We perform our evaluation using the Trans-NASBench-101 benchmark, and consider the efficacy of the transferred agents, as well as how quickly they can be trained. We find that pretraining an agent on one task benefits the performance of the agent in another task in all but 1 task when considering final performance. We also show that the training procedure for an agent can be shortened significantly by pretraining it on another task. Our results indicate that these effects occur regardless of the source or target task, although they are more pronounced for some tasks than for others. Our results show that transfer learning can be an effective tool in mitigating the computational cost of the initial training procedure for reinforcement learning-based NAS agents.

Paper Structure

This paper contains 20 sections, 13 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement Learning-Based NAS
  4. Transfer Learning in Reinforcement Learning
  5. Multi-Task NAS
  6. Methods
  7. Task Selection
  8. Reward Shaping
  9. Experiments
  10. Benefit of Pretraining
  11. Transfer Regime Comparison
  12. Required amount of training
  13. Discussion and Conclusions
  14. Task Heterogeneity
  15. Search Space Generalization
  16. ...and 5 more sections

Figures (13)

  • Figure 1: The Kendall's tau ranking correlation between the different tasks in the Trans-NASBench-101 benchmark.
  • Figure 2: A comparison of the original and gamma-transformed reward distribution for the validation set for the segmentsemantic task.
  • Figure 3: A comparison of our reward shaping function for different values of the $\gamma$ parameter.
  • Figure 4: The spread in reward values for a given value of $\gamma$ in the gamma transform. The maximum validation spread is marked with a vertical solid black line.
  • Figure 5: Performance of agents trained on one task and transferred to another under the retraining regime.
  • ...and 8 more figures