Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LightAgent: Mobile Agentic Foundation Models

Yangqin Jiang, Chao Huang

TL;DR

LightAgent addresses the tension between on-device compute limits and cloud-model costs for mobile GUI tasks by integrating a compact $3\text{B}$ on-device GUI agent trained with a two-stage SFT+GRPO pipeline on synthetic GUI data and a device-cloud orchestration layer. It introduces an efficient long-horizon reasoning module and textual history compression to extend reasoning across steps, paired with a dynamic task-complexity–driven switch to cloud models to minimize costs without sacrificing performance. Evaluations on AndroidLab and popular apps show the approach achieves competitive task success with substantial cloud-cost reductions, validating a practical path toward capable mobile GUI automation. The work demonstrates how a carefully designed on-device model, reinforcement-aligned training, and adaptive cloud offloading can enable robust, cost-effective mobile AI assistants.

Abstract

With the advancement of multimodal large language models (MLLMs), building GUI agent systems has become an increasingly promising direction-especially for mobile platforms, given their rich app ecosystems and intuitive touch interactions. Yet mobile GUI agents face a critical dilemma: truly on-device models (4B or smaller) lack sufficient performance, while capable models (starting from 7B) are either too large for mobile deployment or prohibitively costly (e.g., cloud-only closed-source MLLMs). To resolve this, we propose LightAgent, a mobile agentic foundation model solution that leverages device-cloud collaboration to tap the cost-efficiency of on-device models and the high capability of cloud models, while avoiding their drawbacks. Specifically, LightAgent enhances Qwen2.5-VL-3B via two-stage SFT->GRPO training on synthetic GUI data for strong decision-making, integrates an efficient long-reasoning mechanism to utilize historical interactions under tight resources, and defaults to on-device execution-only escalating challenging subtasks to the cloud via real-time complexity assessment. Experiments on the online AndroidLab benchmark and diverse apps show LightAgent matches or nears larger models, with a significant reduction in cloud costs.

LightAgent: Mobile Agentic Foundation Models

TL;DR

LightAgent addresses the tension between on-device compute limits and cloud-model costs for mobile GUI tasks by integrating a compact on-device GUI agent trained with a two-stage SFT+GRPO pipeline on synthetic GUI data and a device-cloud orchestration layer. It introduces an efficient long-horizon reasoning module and textual history compression to extend reasoning across steps, paired with a dynamic task-complexity–driven switch to cloud models to minimize costs without sacrificing performance. Evaluations on AndroidLab and popular apps show the approach achieves competitive task success with substantial cloud-cost reductions, validating a practical path toward capable mobile GUI automation. The work demonstrates how a carefully designed on-device model, reinforcement-aligned training, and adaptive cloud offloading can enable robust, cost-effective mobile AI assistants.

Abstract

With the advancement of multimodal large language models (MLLMs), building GUI agent systems has become an increasingly promising direction-especially for mobile platforms, given their rich app ecosystems and intuitive touch interactions. Yet mobile GUI agents face a critical dilemma: truly on-device models (4B or smaller) lack sufficient performance, while capable models (starting from 7B) are either too large for mobile deployment or prohibitively costly (e.g., cloud-only closed-source MLLMs). To resolve this, we propose LightAgent, a mobile agentic foundation model solution that leverages device-cloud collaboration to tap the cost-efficiency of on-device models and the high capability of cloud models, while avoiding their drawbacks. Specifically, LightAgent enhances Qwen2.5-VL-3B via two-stage SFT->GRPO training on synthetic GUI data for strong decision-making, integrates an efficient long-reasoning mechanism to utilize historical interactions under tight resources, and defaults to on-device execution-only escalating challenging subtasks to the cloud via real-time complexity assessment. Experiments on the online AndroidLab benchmark and diverse apps show LightAgent matches or nears larger models, with a significant reduction in cloud costs.
Paper Structure (32 sections, 6 equations, 8 figures, 4 tables, 2 algorithms)

This paper contains 32 sections, 6 equations, 8 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Efficient Reasoning GUI Agent
  4. Long-Horizon Reasoning Enhancement
  5. Efficient Memory Management
  6. Overall Process
  7. Device-Cloud Collaborative Agent System
  8. Collaborative Control Framework
  9. Integrated Execution Flow
  10. Lightweight MLLM Tuning for On-Device Agents
  11. Synthetic Data Generation Pipeline
  12. Two-Stage Training Protocol
  13. Evaluation
  14. Experimental Setup
  15. Online Agent Evaluation
  16. ...and 17 more sections

Figures (8)

  • Figure 1: Model GUI Capability vs. Cost. Methods in the on-device (gray) region lack usable GUI capability, while basic-GUI-capability (blue) ones are too large for on-device deployment or too costly. Currently, there are no suitable approaches in the truly usable mobile GUI agent (orange) region. A promising research direction is combining gray and blue region methods, leveraging their complementary strengths to bridge the gap for practical on-device GUI agents.
  • Figure 2: Overall framework of the proposed LightAgent.
  • Figure 3: Output Template.
  • Figure 4: GUI Performance: On-Device Models vs. Cloud Models.
  • Figure 5: Impact of different GRPO training variants.
  • ...and 3 more figures