Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture

Zhengxin Yang, Wanling Gao, Luzhou Peng, Yunyou Huang, Fei Tang, Jianfeng Zhan

TL;DR

Younger introduces a real-world DAG-based dataset of 7,629 neural network architectures derived from ~174K models across 30+ tasks to propel Artificial Intelligence-Generated Neural Network Architecture (AIGNNA). It formalizes two research paradigms: Local, for refinement of existing components, and Global, for end-to-end design from scratch, and demonstrates their feasibility with graph neural network baselines and a diffusion-based generator. The dataset, built from ONNX-converted models and deduplicated via Weisfeiler–Lehman hashing, also serves as a rich benchmark for GNN research due to its diverse, operator-level graph structures. Public release and an extension platform aim to democratize research and accelerate innovation in automated neural architecture design and GNN benchmarking.

Abstract

Designing and optimizing neural network architectures typically requires extensive expertise, starting with handcrafted designs and then manual or automated refinement. This dependency presents a significant barrier to rapid innovation. Recognizing the complexity of automatically generating neural network architecture from scratch, we introduce Younger, a pioneering dataset to advance this ambitious goal. Derived from over 174K real-world models across more than 30 tasks from various public model hubs, Younger includes 7,629 unique architectures, and each is represented as a directed acyclic graph with detailed operator-level information. The dataset facilitates two primary design paradigms: global, for creating complete architectures from scratch, and local, for detailed architecture component refinement. By establishing these capabilities, Younger contributes to a new frontier, Artificial Intelligence-Generated Neural Network Architecture (AIGNNA). Our experiments explore the potential and effectiveness of Younger for automated architecture generation and, as a secondary benefit, demonstrate that Younger can serve as a benchmark dataset, advancing the development of graph neural networks. We release the dataset and code publicly to lower the entry barriers and encourage further research in this challenging area.

Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture

TL;DR

Younger introduces a real-world DAG-based dataset of 7,629 neural network architectures derived from ~174K models across 30+ tasks to propel Artificial Intelligence-Generated Neural Network Architecture (AIGNNA). It formalizes two research paradigms: Local, for refinement of existing components, and Global, for end-to-end design from scratch, and demonstrates their feasibility with graph neural network baselines and a diffusion-based generator. The dataset, built from ONNX-converted models and deduplicated via Weisfeiler–Lehman hashing, also serves as a rich benchmark for GNN research due to its diverse, operator-level graph structures. Public release and an extension platform aim to democratize research and accelerate innovation in automated neural architecture design and GNN benchmarking.

Abstract

Designing and optimizing neural network architectures typically requires extensive expertise, starting with handcrafted designs and then manual or automated refinement. This dependency presents a significant barrier to rapid innovation. Recognizing the complexity of automatically generating neural network architecture from scratch, we introduce Younger, a pioneering dataset to advance this ambitious goal. Derived from over 174K real-world models across more than 30 tasks from various public model hubs, Younger includes 7,629 unique architectures, and each is represented as a directed acyclic graph with detailed operator-level information. The dataset facilitates two primary design paradigms: global, for creating complete architectures from scratch, and local, for detailed architecture component refinement. By establishing these capabilities, Younger contributes to a new frontier, Artificial Intelligence-Generated Neural Network Architecture (AIGNNA). Our experiments explore the potential and effectiveness of Younger for automated architecture generation and, as a secondary benefit, demonstrate that Younger can serve as a benchmark dataset, advancing the development of graph neural networks. We release the dataset and code publicly to lower the entry barriers and encourage further research in this challenging area.
Paper Structure (58 sections, 8 equations, 29 figures, 11 tables)

This paper contains 58 sections, 8 equations, 29 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Artificial Intelligence-Generated Neural Network Architecture
  4. Benchmarking Graph Neural Network
  5. Dataset Construction
  6. Paradigms for AI-Generated Neural Network Architecture (AIGNNA)
  7. Local: Architecture Refinement In Detail
  8. Global: Architecture Design From Scratch
  9. Experiments
  10. Statistical Analysis
  11. Practical Applications
  12. Local Paradigm
  13. Global Paradigm
  14. Conclusion and Future Work
  15. Datasheet
  16. ...and 43 more sections

Figures (29)

  • Figure 1: Overview of the construction pipeline. Models are retrieved from Hugging Face Hub, ONNX Model Zoo, PyTorch Hub, and Kaggle Models. Then, all retrieved models are converted to ONNX and extracted to be DAGs.
  • Figure 2: Paradigms of the AIGNNA. From left to right are operator design for the local paradigm, data flow design for the local paradigm, and global architecture design for the global paradigm.
  • Figure 3: Distribution of #nodes and #edges and top 30 ONNX operators. (a) The distribution of the number of graph nodes and edges in Younger; (b) The top 30 ONNX operators have the highest frequency in Younger.
  • Figure 4: Node embeddings before training
  • Figure 5: Node embeddings after training
  • ...and 24 more figures