Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Self-Adaptive Probabilistic Skyline Query Processing in Distributed Edge Computing via Deep Reinforcement Learning

Chuan-Chi Lai

TL;DR

The paper tackles the challenge of efficient probabilistic skyline processing for uncertain data at the edge in IoE settings, where static thresholds are ill-suited to volatile resource conditions. It formalizes threshold selection as a continuous-action Markov Decision Process and deploys a Deep Deterministic Policy Gradient agent to adaptively tune edge pruning thresholds in real time, balancing edge computation against cloud transmission. The SA-PSKY framework demonstrates up to ~60% reduction in communication and ~40% faster end-to-end response compared to baselines, with strong scalability under high uncertainty and high dimensionality. This approach enables real-time analytics in bandwidth-constrained edge ecosystems by exploiting distributed edge parallelism and intelligent threshold control, with promising avenues for decentralized architectures and federated learning in future work.

Abstract

In the era of the Internet of Everything (IoE), the exponential growth of sensor-generated data at the network edge renders efficient Probabilistic Skyline Query (PSKY) processing a critical challenge. Traditional distributed PSKY methodologies predominantly rely on pre-defined static thresholds to filter local candidates. However, these rigid approaches are fundamentally ill-suited for the highly volatile and heterogeneous nature of edge computing environments, often leading to either severe communication bottlenecks or excessive local computational latency. To resolve this resource conflict, this paper presents SA-PSKY, a novel Self-Adaptive framework designed for distributed edge-cloud collaborative systems. We formalize the dynamic threshold adjustment problem as a continuous Markov Decision Process (MDP) and leverage a Deep Deterministic Policy Gradient (DDPG) agent to autonomously optimize filtering intensities in real-time. By intelligently analyzing multi-dimensional system states, including data arrival rates, uncertainty distributions, and instantaneous resource availability, our framework effectively minimizes a joint objective function of computation and communication costs. Comprehensive experimental evaluations demonstrate that SA-PSKY consistently outperforms state-of-the-art static and heuristic baselines. Specifically, it achieves a reduction of up to 60\% in communication overhead and 40\% in total response time, while ensuring robust scalability across diverse data distributions.

Self-Adaptive Probabilistic Skyline Query Processing in Distributed Edge Computing via Deep Reinforcement Learning

TL;DR

The paper tackles the challenge of efficient probabilistic skyline processing for uncertain data at the edge in IoE settings, where static thresholds are ill-suited to volatile resource conditions. It formalizes threshold selection as a continuous-action Markov Decision Process and deploys a Deep Deterministic Policy Gradient agent to adaptively tune edge pruning thresholds in real time, balancing edge computation against cloud transmission. The SA-PSKY framework demonstrates up to ~60% reduction in communication and ~40% faster end-to-end response compared to baselines, with strong scalability under high uncertainty and high dimensionality. This approach enables real-time analytics in bandwidth-constrained edge ecosystems by exploiting distributed edge parallelism and intelligent threshold control, with promising avenues for decentralized architectures and federated learning in future work.

Abstract

In the era of the Internet of Everything (IoE), the exponential growth of sensor-generated data at the network edge renders efficient Probabilistic Skyline Query (PSKY) processing a critical challenge. Traditional distributed PSKY methodologies predominantly rely on pre-defined static thresholds to filter local candidates. However, these rigid approaches are fundamentally ill-suited for the highly volatile and heterogeneous nature of edge computing environments, often leading to either severe communication bottlenecks or excessive local computational latency. To resolve this resource conflict, this paper presents SA-PSKY, a novel Self-Adaptive framework designed for distributed edge-cloud collaborative systems. We formalize the dynamic threshold adjustment problem as a continuous Markov Decision Process (MDP) and leverage a Deep Deterministic Policy Gradient (DDPG) agent to autonomously optimize filtering intensities in real-time. By intelligently analyzing multi-dimensional system states, including data arrival rates, uncertainty distributions, and instantaneous resource availability, our framework effectively minimizes a joint objective function of computation and communication costs. Comprehensive experimental evaluations demonstrate that SA-PSKY consistently outperforms state-of-the-art static and heuristic baselines. Specifically, it achieves a reduction of up to 60\% in communication overhead and 40\% in total response time, while ensuring robust scalability across diverse data distributions.
Paper Structure (40 sections, 18 equations, 4 figures, 3 tables)

This paper contains 40 sections, 18 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Foundations of Skyline Queries over Uncertain Data
  4. Distributed and Parallel Processing Architectures
  5. Adaptive Resource Management and Edge Computing
  6. Deep Reinforcement Learning for Query Optimization
  7. System Model and Problem Definition
  8. Uncertain Data Stream and Statistical Assumptions
  9. Probabilistic Dominance and Skyline Definitions
  10. Distributed Edge-Cloud Architecture
  11. Edge Layer (Local Filtering)
  12. Cloud Layer (Global Aggregation)
  13. Computational Complexity Model
  14. Network Channel and Queuing Model
  15. Optimization Problem Formulation
  16. ...and 25 more sections

Figures (4)

  • Figure 1: Distributed Edge-Cloud Architecture for Probabilistic Skyline Query Processing. Each edge node independently filters local data streams before transmitting candidates to a central broker for final aggregation.
  • Figure 2: Comparison of different methods under default settings in terms of: \ref{['fig:default_performance-transmission_time']} Transmission Latency $T_{\rm trans}$, \ref{['fig:default_performance-computation_time']} Computation Latency $T_{\rm comp}$, and \ref{['fig:default_performance-e2e_time']} End-to-End Latency $T_{\rm total}$.
  • Figure 3: Comparisons of different methods under varying number of instances per uncertain object ($m$) in terms of: \ref{['fig:no_of_instances-transmission_time']} Transmission Latency $T_{\rm trans}$, \ref{['fig:no_of_instances-computation_time']} Computation Latency $T_{\rm comp}$, and \ref{['fig:no_of_instances-e2e_time']} End-to-End Latency $T_{\rm total}$.
  • Figure 4: Comparisons of different methods under varying data dimensionality per uncertain object ($d$) in terms of: \ref{['fig:no_of_dimensions-transmission_time']} Transmission Latency $T_{\rm trans}$, \ref{['fig:no_of_dimensions-computation_time']} Computation Latency $T_{\rm comp}$, and \ref{['fig:no_of_dimensions-e2e_time']} End-to-End Latency $T_{\rm total}$.

Theorems & Definitions (5)

  • Definition 1: Uncertain Data Stream
  • Definition 2: Sliding Window Model
  • Definition 3: Instance-Level Dominance
  • Definition 4: Object-Level Dominance Probability
  • Definition 5: Probabilistic Skyline