Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ShadowGenes: Leveraging Recurring Patterns within Computational Graphs for Model Genealogy

Kasimir Schulz, Kieran Evans

TL;DR

ShadowGenes tackles the challenge of verifying pretrained model architectures and lineage in public repositories by constructing a format-agnostic computational graph and deriving signature-based patterns. Through block-based analysis of repeated subgraphs and a JSON signature format, it scans models to infer architecture, family, and derivatives without external metadata, handling multimodal and derivative cases. On a diverse ONNX dataset of 1,483 models across 104 families, ShadowGenes achieves high accuracy with a mean TP rate of $97.49\%$ and precision of $99.51\%$, while addressing opset variations and architectural nuances via wildcard and combo signatures. The approach promises practical gains in model governance, transparency, and security, with future work aimed at expanding coverage and integrating complementary fingerprinting methods.

Abstract

Machine learning model genealogy enables practitioners to determine which architectural family a neural network belongs to. In this paper, we introduce ShadowGenes, a novel, signature-based method for identifying a given model's architecture, type, and family. Our method involves building a computational graph of the model that is agnostic of its serialization format, then analyzing its internal operations to identify unique patterns, and finally building and refining signatures based on these. We highlight important workings of the underlying engine and demonstrate the technique used to construct a signature and scan a given model. This approach to model genealogy can be applied to model files without the need for additional external information. We test ShadowGenes on a labeled dataset of over 1,400 models and achieve a mean true positive rate of 97.49% and a precision score of 99.51%; which validates the technique as a practical method for model genealogy. This enables practitioners to understand the use cases of a given model, the internal computational process, and identify possible security risks, such as the potential for model backdooring.

ShadowGenes: Leveraging Recurring Patterns within Computational Graphs for Model Genealogy

TL;DR

ShadowGenes tackles the challenge of verifying pretrained model architectures and lineage in public repositories by constructing a format-agnostic computational graph and deriving signature-based patterns. Through block-based analysis of repeated subgraphs and a JSON signature format, it scans models to infer architecture, family, and derivatives without external metadata, handling multimodal and derivative cases. On a diverse ONNX dataset of 1,483 models across 104 families, ShadowGenes achieves high accuracy with a mean TP rate of and precision of , while addressing opset variations and architectural nuances via wildcard and combo signatures. The approach promises practical gains in model governance, transparency, and security, with future work aimed at expanding coverage and integrating complementary fingerprinting methods.

Abstract

Machine learning model genealogy enables practitioners to determine which architectural family a neural network belongs to. In this paper, we introduce ShadowGenes, a novel, signature-based method for identifying a given model's architecture, type, and family. Our method involves building a computational graph of the model that is agnostic of its serialization format, then analyzing its internal operations to identify unique patterns, and finally building and refining signatures based on these. We highlight important workings of the underlying engine and demonstrate the technique used to construct a signature and scan a given model. This approach to model genealogy can be applied to model files without the need for additional external information. We test ShadowGenes on a labeled dataset of over 1,400 models and achieve a mean true positive rate of 97.49% and a precision score of 99.51%; which validates the technique as a practical method for model genealogy. This enables practitioners to understand the use cases of a given model, the internal computational process, and identify possible security risks, such as the potential for model backdooring.
Paper Structure (26 sections, 22 figures, 7 algorithms)

This paper contains 26 sections, 22 figures, 7 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Extracting a Model Format Agnostic Graph
  5. Extracting Repeating Subgraphs
  6. Defining Linear Blocks in a Computational Graph
  7. Extracting Blocks from the Graph
  8. Creating Signatures for Computational Graphs
  9. The Etymology of a Signature
  10. Signature Construction Through Graph Visualization
  11. Experimental Results
  12. Efficacy of Signatures for Detecting Model Families for Known Families
  13. Adjusting for Architectural Nuances within Classes
  14. Adjusting for Different ONNX Opsets
  15. Combo Signatures: Refinement through Sub-components
  16. ...and 11 more sections

Figures (22)

  • Figure 1: A figure showing the different types of execution flows.
  • Figure 2: The different types of execution flow when blocking is applied.
  • Figure 3: Blocks found by first pass.
  • Figure 4: Blocks with Constants removed.
  • Figure 5: An identifiable repeating pattern observed in ResNet50 models (L), and the corresponding constructed subgraph leveraged to build a signature for the ResNet model family (R).
  • ...and 17 more figures