Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Remote Estimation of Markov Processes over Costly Channels: On the Benefits of Implicit Information

Edoardo David Santi, Touraj Soleymani, Deniz Gunduz

TL;DR

This work addresses remote estimation of a discrete-state Markov process over a costly communication channel by formulating a two-player infinite-horizon optimization that accounts for implicit information—the monitor’s knowledge gained when the sensor is silent. It introduces three algorithms: alternating policy optimization (converging to a Nash equilibrium), an occupancy-state MDP (globally optimal), and a heuristic optimal under perfect reconstruction, all designed to exploit implicit information rather than neglect it. The occupancy-state approach provides a principled global optimum via belief-based transitions, while the heuristic offers a simple, near-optimal policy that is provably optimal for perfect reconstruction. Together, these methods demonstrate substantial performance gains over traditional policies that ignore implicit information, with practical impact for designing energy-efficient, high-fidelity remote monitoring systems.

Abstract

In this paper, we study the remote estimation problem of a Markov process over a channel with a cost. We formulate this problem as an infinite horizon optimization problem with two players, i.e., a sensor and a monitor, that have distinct information, and with a reward function that takes into account both the communication cost and the estimation quality. We show that the main challenge in solving this problem is associated with the consideration of implicit information, i.e., information that the monitor can obtain about the source when the sensor is silent. Our main objective is to develop a framework for finding solutions to this problem without neglecting implicit information a priori. To that end, we propose three different algorithms. The first one is an alternating policy algorithm that converges to a Nash equilibrium. The second one is an occupancy-state algorithm that is guaranteed to find a globally optimal solution. The last one is a heuristic algorithm that is able to find a near-optimal solution.

Remote Estimation of Markov Processes over Costly Channels: On the Benefits of Implicit Information

TL;DR

This work addresses remote estimation of a discrete-state Markov process over a costly communication channel by formulating a two-player infinite-horizon optimization that accounts for implicit information—the monitor’s knowledge gained when the sensor is silent. It introduces three algorithms: alternating policy optimization (converging to a Nash equilibrium), an occupancy-state MDP (globally optimal), and a heuristic optimal under perfect reconstruction, all designed to exploit implicit information rather than neglect it. The occupancy-state approach provides a principled global optimum via belief-based transitions, while the heuristic offers a simple, near-optimal policy that is provably optimal for perfect reconstruction. Together, these methods demonstrate substantial performance gains over traditional policies that ignore implicit information, with practical impact for designing energy-efficient, high-fidelity remote monitoring systems.

Abstract

In this paper, we study the remote estimation problem of a Markov process over a channel with a cost. We formulate this problem as an infinite horizon optimization problem with two players, i.e., a sensor and a monitor, that have distinct information, and with a reward function that takes into account both the communication cost and the estimation quality. We show that the main challenge in solving this problem is associated with the consideration of implicit information, i.e., information that the monitor can obtain about the source when the sensor is silent. Our main objective is to develop a framework for finding solutions to this problem without neglecting implicit information a priori. To that end, we propose three different algorithms. The first one is an alternating policy algorithm that converges to a Nash equilibrium. The second one is an occupancy-state algorithm that is guaranteed to find a globally optimal solution. The last one is a heuristic algorithm that is able to find a near-optimal solution.
Paper Structure (13 sections, 4 theorems, 13 equations, 1 figure, 3 algorithms)

This paper contains 13 sections, 4 theorems, 13 equations, 1 figure, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions and Paper Structure
  4. Problem formulation
  5. Role of Implicit Information
  6. Alternating Policy Optimization
  7. Occupancy-State Algorithm
  8. Heuristic Algorithm
  9. Numerical results
  10. Conclusion
  11. Derivation of information state update
  12. Derivation of belief transition
  13. Derivation of monitor's best actions

Key Result

Theorem 1

The infinite horizon average reward of the system $J^k=J(\pi_1^k,\pi_2^k)$ convergences as $k \to \infty$ and the policies obtained upon convergence form a Nash equilibrium.

Figures (1)

  • Figure 1: Reconstruction performance vs communication cost trade-off for the various schemes considered in the paper.

Theorems & Definitions (7)

  • Theorem 1
  • Remark 1
  • Theorem 2
  • Remark 2
  • Theorem 3
  • Remark 3
  • Lemma 1