Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vision Language Models for Optimization-Driven Intent Processing in Autonomous Networks

Tasnim Ahmed, Yifan Zhu, Salimur Choudhury

TL;DR

IntentOpt introduces a multimodal benchmark to evaluate vision-language models on generating provably optimal optimization code for network intents. By coupling annotated network diagrams with natural language goals and ground-truth Gurobi solutions, the study reveals a persistent vision-language gap and shows that prompting strategies and model type strongly influence performance. The results underscore the current limitations of VLMs in optimization code synthesis, especially for visual parameter extraction and open-source models, while demonstrating practical feasibility through a Model Context Protocol deployment case study. The work provides a reproducible framework and actionable insights to guide future research in vision-enabled, optimization-driven autonomous networks.

Abstract

Intent-Based Networking (IBN) allows operators to specify high-level network goals rather than low-level configurations. While recent work demonstrates that large language models can automate configuration tasks, a distinct class of intents requires generating optimization code to compute provably optimal solutions for traffic engineering, routing, and resource allocation. Current systems assume text-based intent expression, requiring operators to enumerate topologies and parameters in prose. Network practitioners naturally reason about structure through diagrams, yet whether Vision-Language Models (VLMs) can process annotated network sketches into correct optimization code remains unexplored. We present IntentOpt, a benchmark of 85 optimization problems across 17 categories, evaluating four VLMs (GPT-5-Mini, Claude-Haiku-4.5, Gemini-2.5-Flash, Llama-3.2-11B-Vision) under three prompting strategies on multimodal versus text-only inputs. Our evaluation shows that visual parameter extraction reduces execution success by 12-21 percentage points (pp), with GPT-5-Mini dropping from 93% to 72%. Program-of-thought prompting decreases performance by up to 13 pp, and open-source models lag behind closed-source ones, with Llama-3.2-11B-Vision reaching 18% compared to 75% for GPT-5-Mini. These results establish baseline capabilities and limitations of current VLMs for optimization code generation within an IBN system. We also demonstrate practical feasibility through a case study that deploys VLM-generated code to network testbed infrastructure using Model Context Protocol.

Vision Language Models for Optimization-Driven Intent Processing in Autonomous Networks

TL;DR

IntentOpt introduces a multimodal benchmark to evaluate vision-language models on generating provably optimal optimization code for network intents. By coupling annotated network diagrams with natural language goals and ground-truth Gurobi solutions, the study reveals a persistent vision-language gap and shows that prompting strategies and model type strongly influence performance. The results underscore the current limitations of VLMs in optimization code synthesis, especially for visual parameter extraction and open-source models, while demonstrating practical feasibility through a Model Context Protocol deployment case study. The work provides a reproducible framework and actionable insights to guide future research in vision-enabled, optimization-driven autonomous networks.

Abstract

Intent-Based Networking (IBN) allows operators to specify high-level network goals rather than low-level configurations. While recent work demonstrates that large language models can automate configuration tasks, a distinct class of intents requires generating optimization code to compute provably optimal solutions for traffic engineering, routing, and resource allocation. Current systems assume text-based intent expression, requiring operators to enumerate topologies and parameters in prose. Network practitioners naturally reason about structure through diagrams, yet whether Vision-Language Models (VLMs) can process annotated network sketches into correct optimization code remains unexplored. We present IntentOpt, a benchmark of 85 optimization problems across 17 categories, evaluating four VLMs (GPT-5-Mini, Claude-Haiku-4.5, Gemini-2.5-Flash, Llama-3.2-11B-Vision) under three prompting strategies on multimodal versus text-only inputs. Our evaluation shows that visual parameter extraction reduces execution success by 12-21 percentage points (pp), with GPT-5-Mini dropping from 93% to 72%. Program-of-thought prompting decreases performance by up to 13 pp, and open-source models lag behind closed-source ones, with Llama-3.2-11B-Vision reaching 18% compared to 75% for GPT-5-Mini. These results establish baseline capabilities and limitations of current VLMs for optimization code generation within an IBN system. We also demonstrate practical feasibility through a case study that deploys VLM-generated code to network testbed infrastructure using Model Context Protocol.
Paper Structure (24 sections, 3 figures, 3 tables)

This paper contains 24 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. The IntentOpt Benchmark
  3. Evaluation Framework
  4. Problem Formulation
  5. Models
  6. Prompting Strategies
  7. Direct Prompting
  8. Role Prompting
  9. Program-of-Thought
  10. Evaluation Metrics
  11. Execution Success Rate (ESR)
  12. Optimality Gap
  13. Exact Match Rate (EMR)
  14. CodeBLEU
  15. Statistical Analysis
  16. ...and 9 more sections

Figures (3)

  • Figure 1: Six stage pipeline for IntentOpt construction and evaluation. Stages 1 to 4 create the benchmark and stages 5 and 6 evaluate VLMs using different prompting strategies.
  • Figure 2: Effect of prompting strategy on model performance (multimodal). (a) PoT reduces ESR for GPT-5-Mini and Claude-Haiku-4.5 ($p < 0.05$). (b) EMR shows similar trends. (c) CodeBLEU remains stable, indicating structural quality is less impacted.
  • Figure 3: MCP deployment pipeline where the VLM generates optimization code from network intent, executes it, translates outputs to OpenFlow rules, and deploys them on the SDN.