Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization

Jiashun Li, Xiaoyu Shi, Hong Xie, Mingsheng Shang, Yun Lu

Abstract

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for robotic manipulation. However, existing post-training methods face a dilemma between stability and exploration: Supervised Fine-Tuning (SFT) is constrained by demonstration quality and lacks generalization, whereas Reinforcement Learning (RL) improves exploration but often induces erratic, jittery trajectories that violate physical constraints. To bridge this gap, we propose SmoothVLA, a novel reinforcement learning fine-tuning framework that synergistically optimizes task performance and motion smoothness. The technical core is a physics-informed hybrid reward function that integrates binary sparse task rewards with a continuous dense term derived from trajectory jerk. Crucially, this reward is intrinsic, that computing directly from policy rollouts, without requiring extrinsic environment feedback or laborious reward engineering. Leveraging the Group Relative Policy Optimization (GRPO), SmoothVLA establishes trajectory smoothness as an explicit optimization prior, guiding the model toward physically feasible and stable control. Extensive experiments on the LIBERO benchmark demonstrate that SmoothVLA outperforms standard RL by 13.8\% in smoothness and significantly surpasses SFT in generalization across diverse tasks. Our work offers a scalable approach to aligning VLA models with physical-world constraints through intrinsic reward optimization.

SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization

Abstract

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for robotic manipulation. However, existing post-training methods face a dilemma between stability and exploration: Supervised Fine-Tuning (SFT) is constrained by demonstration quality and lacks generalization, whereas Reinforcement Learning (RL) improves exploration but often induces erratic, jittery trajectories that violate physical constraints. To bridge this gap, we propose SmoothVLA, a novel reinforcement learning fine-tuning framework that synergistically optimizes task performance and motion smoothness. The technical core is a physics-informed hybrid reward function that integrates binary sparse task rewards with a continuous dense term derived from trajectory jerk. Crucially, this reward is intrinsic, that computing directly from policy rollouts, without requiring extrinsic environment feedback or laborious reward engineering. Leveraging the Group Relative Policy Optimization (GRPO), SmoothVLA establishes trajectory smoothness as an explicit optimization prior, guiding the model toward physically feasible and stable control. Extensive experiments on the LIBERO benchmark demonstrate that SmoothVLA outperforms standard RL by 13.8\% in smoothness and significantly surpasses SFT in generalization across diverse tasks. Our work offers a scalable approach to aligning VLA models with physical-world constraints through intrinsic reward optimization.
Paper Structure (26 sections, 5 equations, 7 figures, 3 tables)

This paper contains 26 sections, 5 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Empirical Study on Trajectory Smoothness in SFT and RL Fine-Tuned VLA Models
  3. Qualitative Analysis: Visual Comparison of Trajectory Smoothness
  4. Quantitative Analysis: Jerk-based Metric
  5. The Functional Correlation Between Trajectory Smoothness and Task Success Rate
  6. SmoothVLA
  7. Preliminaries and Problem Formulation
  8. State Space ($\mathcal{S}$):
  9. Action Space ($\mathcal{A}$):
  10. Action Chunking:
  11. Group Relative Policy Optimization (GRPO)
  12. Physics-Informed Reward: From Jerk to Alignment
  13. The SmoothVLA Reward Function:
  14. Experiment
  15. Experimental Setups
  16. ...and 11 more sections

Figures (7)

  • Figure 1: Overview of SmoothVLA.
  • Figure 2: OpenVLA-SFT model
  • Figure 3: OpenVLA-RL model
  • Figure 4: Comparison of SmoothVLA with OpenVLA and Octo fine-tuned on the same data on the LIBERO environment.
  • Figure 5: Comparison of motion smoothness through average jerk analysis between OpenVLA and SmoothVLA fine-tuned models evaluated on the LIBERO benchmark.
  • ...and 2 more figures