Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Non-Intrusive Framework for Deferred Integration of Cloud Patterns in Energy-Efficient Data-Sharing Pipelines

Sepideh Masoudi, Mark Edward Michael Daly, Jannis Kiesel, Stefan Tai

TL;DR

This paper addresses the challenge of applying cloud design patterns to Data Mesh‑style data-sharing pipelines without compromising the modularity of individual transformation stages. It proposes SnapPattern, a Kubernetes‑based tool that injects external design patterns as non‑intrusive proxies/sidecars and collects energy and performance metrics. The approach demonstrates feasibility through a proof‑of‑concept with five patterns (CB, CA, RC, GO, ARR) and a small set of transformation services, with automated Minikube deployment and monitoring via Kepler, OpenTelemetry, and Prometheus. The work enables energy‑aware, consumer‑driven pipeline design and sets groundwork for broader pattern coverage and integration with workflow engines.

Abstract

As data mesh architectures gain traction in federated environments, organizations are increasingly building consumer-specific data-sharing pipelines using modular, cloud-native transformation services. Prior work has shown that structuring these pipelines with reusable transformation stages enhances both scalability and energy efficiency. However, integrating traditional cloud design patterns into such pipelines poses a challenge: predefining and embedding patterns can compromise modularity, reduce reusability, and conflict with the pipelines dynamic, consumer-driven nature. To address this, we introduce a Kubernetes-based tool that enables the deferred and non-intrusive application of selected cloud design patterns without requiring changes to service source code. The tool supports automated pattern injection and collects energy consumption metrics, allowing developers to make energy-aware decisions while preserving the flexible, composable structure of reusable data-sharing pipelines.

A Non-Intrusive Framework for Deferred Integration of Cloud Patterns in Energy-Efficient Data-Sharing Pipelines

TL;DR

This paper addresses the challenge of applying cloud design patterns to Data Mesh‑style data-sharing pipelines without compromising the modularity of individual transformation stages. It proposes SnapPattern, a Kubernetes‑based tool that injects external design patterns as non‑intrusive proxies/sidecars and collects energy and performance metrics. The approach demonstrates feasibility through a proof‑of‑concept with five patterns (CB, CA, RC, GO, ARR) and a small set of transformation services, with automated Minikube deployment and monitoring via Kepler, OpenTelemetry, and Prometheus. The work enables energy‑aware, consumer‑driven pipeline design and sets groundwork for broader pattern coverage and integration with workflow engines.

Abstract

As data mesh architectures gain traction in federated environments, organizations are increasingly building consumer-specific data-sharing pipelines using modular, cloud-native transformation services. Prior work has shown that structuring these pipelines with reusable transformation stages enhances both scalability and energy efficiency. However, integrating traditional cloud design patterns into such pipelines poses a challenge: predefining and embedding patterns can compromise modularity, reduce reusability, and conflict with the pipelines dynamic, consumer-driven nature. To address this, we introduce a Kubernetes-based tool that enables the deferred and non-intrusive application of selected cloud design patterns without requiring changes to service source code. The tool supports automated pattern injection and collects energy consumption metrics, allowing developers to make energy-aware decisions while preserving the flexible, composable structure of reusable data-sharing pipelines.
Paper Structure (9 sections, 2 figures)

This paper contains 9 sections, 2 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Non-Intrusive Deferred Integration of Cloud Patterns
  4. Deferred Automatic Application of Cloud Design Patterns
  5. Proof of Concepts
  6. Technical Feasibility Demonstration
  7. Functionality and Integration
  8. Conclusion
  9. Acknowledgments.

Figures (2)

  • Figure 1: Logical architecture of SnapPattern for the automated application of cloud design patterns.
  • Figure 2: Energy consumption measured in joules per 10 seconds for the workload scenarios Low, Medium, and High for over all system under the test.