Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning

Chi-Hung Hsu, Shu-Huan Chang, Jhao-Hong Liang, Hsin-Ping Chou, Chun-Hao Liu, Shih-Chieh Chang, Jia-Yu Pan, Yu-Ting Chen, Wei Wei, Da-Cheng Juan

TL;DR

MONAS introduces a reinforcement-learning-based framework for multi-objective neural architecture search that jointly optimizes accuracy and resource metrics such as power and MACs. By using a robot network (an LSTM controller) to propose CNN hyperparameters and evaluating them on target networks, MONAS guides search toward Pareto-efficient architectures; MONAS-S extends this with weight-sharing to scale to massive search spaces. Empirical results show MONAS discovers models with competitive or superior accuracy and reduced energy consumption compared to state-of-the-art baselines, and MONAS-S achieves substantial speedups while maintaining MAC-efficient designs. The approach offers adaptable, scalable NAS suitable for resource-constrained deployments across diverse CNN families like AlexNet-like and CondenseNet-like architectures.

Abstract

Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being deployed on certain computing environment (e.g., with limited power budgets). We propose MONAS, a framework for Multi-Objective Neural Architectural Search that employs reward functions considering both prediction accuracy and other important objectives (e.g., power consumption) when searching for neural network architectures. Experimental results showed that, compared to the state-ofthe-arts, models found by MONAS achieve comparable or better classification accuracy on computer vision applications, while satisfying the additional objectives such as peak power.

MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning

TL;DR

MONAS introduces a reinforcement-learning-based framework for multi-objective neural architecture search that jointly optimizes accuracy and resource metrics such as power and MACs. By using a robot network (an LSTM controller) to propose CNN hyperparameters and evaluating them on target networks, MONAS guides search toward Pareto-efficient architectures; MONAS-S extends this with weight-sharing to scale to massive search spaces. Empirical results show MONAS discovers models with competitive or superior accuracy and reduced energy consumption compared to state-of-the-art baselines, and MONAS-S achieves substantial speedups while maintaining MAC-efficient designs. The approach offers adaptable, scalable NAS suitable for resource-constrained deployments across diverse CNN families like AlexNet-like and CondenseNet-like architectures.

Abstract

Recent studies on neural architecture search have shown that automatically designed neural networks perform as good as expert-crafted architectures. While most existing works aim at finding architectures that optimize the prediction accuracy, these architectures may have complexity and is therefore not suitable being deployed on certain computing environment (e.g., with limited power budgets). We propose MONAS, a framework for Multi-Objective Neural Architectural Search that employs reward functions considering both prediction accuracy and other important objectives (e.g., power consumption) when searching for neural network architectures. Experimental results showed that, compared to the state-ofthe-arts, models found by MONAS achieve comparable or better classification accuracy on computer vision applications, while satisfying the additional objectives such as peak power.

Paper Structure

This paper contains 27 sections, 2 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Proposed Framework: MONAS
  4. Framework Overview
  5. Implementation Details
  6. Robot Network
  7. Target Networks & Search Space
  8. Reinforcement Learning Process
  9. Policy Gradient
  10. MONAS for Scalability
  11. Reward Function
  12. Experimental Setup
  13. Hardware:
  14. GPU Profiler:
  15. Dataset:
  16. ...and 12 more sections

Figures (9)

  • Figure 1: RNN workflow for AlexNet and CondenseNet
  • Figure 2: AlexNet Random Search versus Power or Accuracy Constraint
  • Figure 3: MONAS efficiently guides the search toward models satisfying constraints, while the random search has no particular focus.
  • Figure 4: Applying different $\alpha$ when searching AlexNet
  • Figure 5: MONAS Pareto Fronts on AlexNet and CondenseNet
  • ...and 4 more figures