Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Visual Insights into Agentic Optimization of Pervasive Stream Processing Services

Boris Sedlak, Víctor Casamayor Pujol, Schahram Dustdar

TL;DR

A platform for context-aware autoscaling of stream processing services that allows developers to monitor and adjust the service execution across multiple service-specific parameters and connects a scaling agent to these interfaces that gradually builds an understanding of the processing environment by exploring each service's action space.

Abstract

Processing sensory data close to the data source, often involving Edge devices, promises low latency for pervasive applications, like smart cities. This commonly involves a multitude of processing services, executed with limited resources; this setup faces three problems: first, the application demand and the resource availability fluctuate, so the service execution must scale dynamically to sustain processing requirements (e.g., latency); second, each service permits different actions to adjust its operation, so they require individual scaling policies; third, without a higher-level mediator, services would cannibalize any resources of services co-located on the same device. This demo first presents a platform for context-aware autoscaling of stream processing services that allows developers to monitor and adjust the service execution across multiple service-specific parameters. We then connect a scaling agent to these interfaces that gradually builds an understanding of the processing environment by exploring each service's action space; the agent then optimizes the service execution according to this knowledge. Participants can revisit the demo contents as video summary and introductory poster, or build a custom agent by extending the artifact repository.

Visual Insights into Agentic Optimization of Pervasive Stream Processing Services

TL;DR

A platform for context-aware autoscaling of stream processing services that allows developers to monitor and adjust the service execution across multiple service-specific parameters and connects a scaling agent to these interfaces that gradually builds an understanding of the processing environment by exploring each service's action space.

Abstract

Processing sensory data close to the data source, often involving Edge devices, promises low latency for pervasive applications, like smart cities. This commonly involves a multitude of processing services, executed with limited resources; this setup faces three problems: first, the application demand and the resource availability fluctuate, so the service execution must scale dynamically to sustain processing requirements (e.g., latency); second, each service permits different actions to adjust its operation, so they require individual scaling policies; third, without a higher-level mediator, services would cannibalize any resources of services co-located on the same device. This demo first presents a platform for context-aware autoscaling of stream processing services that allows developers to monitor and adjust the service execution across multiple service-specific parameters. We then connect a scaling agent to these interfaces that gradually builds an understanding of the processing environment by exploring each service's action space; the agent then optimizes the service execution according to this knowledge. Participants can revisit the demo contents as video summary and introductory poster, or build a custom agent by extending the artifact repository.
Paper Structure (13 sections, 3 figures, 1 table)

This paper contains 13 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Solution Architecture
  3. Autoscaling Platform (MUDAP)
  4. Autoscaling Agent (RASK)
  5. Demo Contents
  6. Introductory Poster
  7. Visual Demo of Operation
  8. Experimental Setup
  9. Visual Animation
  10. Results
  11. Explanatory Video
  12. Artifact Repository
  13. Conclusion

Figures (3)

  • Figure 1: Architecture of the MUDAP platform sedlak_multi-dimensional_2025_url: sensor data is 1 buffered and 2 processed by containerized services; 3 service and container states (i.e., processing metrics) are collected in a time-series DB. Lastly, 4 a scaling agent interprets these states, develops a policy, and adjusts service configurations and their containers through a REST API.
  • Figure 2: Conceptual sequence of RASK algorithm sedlak_multi-dimensional_2025_url: 1 create a tabular structure from time-series data and train regression functions; 2 supply functions, SLOs, and parameter bounds to numerical solver; 3 optimize parameter assignments for all monitored services and adjust values through MUDAP API.
  • Figure 3: Snapshot of the visual demo: For three processing services, we display their current service output, their SLO fulfillment, and the regression model that the RASK agent learns through interventions with the autoscaling platform.