Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

To Optimize Human-in-the-loop Learning in Repeated Routing Games

Hongbo Li, Lingjie Duan

TL;DR

A novel user-differential probabilistic recommendation (UPR) mechanism to differentiate and randomize path recommendations for users with differential learning histories is proposed and it is proved that the UPR mechanism ensures interim individual rationality for all users and significantly reduces close-to-optimal performance.

Abstract

Today navigation applications (e.g., Waze and Google Maps) enable human users to learn and share the latest traffic observations, yet such information sharing simply aids selfish users to predict and choose the shortest paths to jam each other. Prior routing game studies focus on myopic users in oversimplified one-shot scenarios to regulate selfish routing via information hiding or pricing mechanisms. For practical human-in-the-loop learning (HILL) in repeated routing games, we face non-myopic users of differential past observations and need new mechanisms (preferably non-monetary) to persuade users to adhere to the optimal path recommendations. We model the repeated routing game in a typical parallel transportation network, which generally contains one deterministic path and $N$ stochastic paths. We first prove that no matter under the information sharing mechanism in use or the latest routing literature's hiding mechanism, the resultant price of anarchy (PoA) for measuring the efficiency loss from social optimum can approach infinity, telling arbitrarily poor exploration-exploitation tradeoff over time. Then we propose a novel user-differential probabilistic recommendation (UPR) mechanism to differentiate and randomize path recommendations for users with differential learning histories. We prove that our UPR mechanism ensures interim individual rationality for all users and significantly reduces $\text{PoA}=\infty$ to close-to-optimal $\text{PoA}=1+\frac{1}{4N+3}$, which cannot be further reduced by any other non-monetary mechanism. In addition to theoretical analysis, we conduct extensive experiments using real-world datasets to generalize our routing graphs and validate the close-to-optimal performance of UPR mechanism.

To Optimize Human-in-the-loop Learning in Repeated Routing Games

TL;DR

A novel user-differential probabilistic recommendation (UPR) mechanism to differentiate and randomize path recommendations for users with differential learning histories is proposed and it is proved that the UPR mechanism ensures interim individual rationality for all users and significantly reduces close-to-optimal performance.

Abstract

Today navigation applications (e.g., Waze and Google Maps) enable human users to learn and share the latest traffic observations, yet such information sharing simply aids selfish users to predict and choose the shortest paths to jam each other. Prior routing game studies focus on myopic users in oversimplified one-shot scenarios to regulate selfish routing via information hiding or pricing mechanisms. For practical human-in-the-loop learning (HILL) in repeated routing games, we face non-myopic users of differential past observations and need new mechanisms (preferably non-monetary) to persuade users to adhere to the optimal path recommendations. We model the repeated routing game in a typical parallel transportation network, which generally contains one deterministic path and stochastic paths. We first prove that no matter under the information sharing mechanism in use or the latest routing literature's hiding mechanism, the resultant price of anarchy (PoA) for measuring the efficiency loss from social optimum can approach infinity, telling arbitrarily poor exploration-exploitation tradeoff over time. Then we propose a novel user-differential probabilistic recommendation (UPR) mechanism to differentiate and randomize path recommendations for users with differential learning histories. We prove that our UPR mechanism ensures interim individual rationality for all users and significantly reduces to close-to-optimal , which cannot be further reduced by any other non-monetary mechanism. In addition to theoretical analysis, we conduct extensive experiments using real-world datasets to generalize our routing graphs and validate the close-to-optimal performance of UPR mechanism.

Paper Structure

This paper contains 24 sections, 9 theorems, 42 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. System model
  3. Repeated Routing Game Model
  4. Public HILL under Information Sharing
  5. Differential HILL under Information Hiding
  6. Problem Formulations and Analysis for Sharing, Hiding, and Social Optimum
  7. Formulation and Equilibrium for Information Sharing
  8. Formulation and Equilibrium for Information Hiding
  9. Formulation and Solution for Social Optimum
  10. Analytical comparison between sharing, hiding, and social optimum via PoA
  11. PoA Analysis of Information Sharing Mechanism
  12. PoA Analysis of Information Hiding Mechanism
  13. Design and Analysis of Our User-differential Informational Mechanism
  14. Experiment Validation Using Real Datasets
  15. Conclusions
  16. ...and 9 more sections

Key Result

Lemma 1

Under information sharing, given the unit flow of non-myopic user arrivals at time $t$, the recommended flow of users for choosing stochastic path 1 at equilibrium is: To realize equilibrium flow $f_1^{(s)}(x_1(t))=\epsilon$ in the second case, each user will be fairly chosen with probability $\epsi and $f_0^{(s)}(x_1(t))=1-f_1^{(s)}(x_1(t))$ to path 1, where infinitesimal $\epsilon>0$, $\mathds{1

Figures (3)

  • Figure 1: At the beginning of each $t\in\{1,2,\cdots\}$, a unit flow of users arrive at origin O to select a path among all the $N+1$ paths of the network in Fig. \ref{['fig:congestion_game']}. Here path 0 has a fixed internal travel cost $c_0$. Yet any other path $i\in\mathbb{N}$ is stochastic and its internal cost $c_i(t)$ varies based on the Markov chain illustrated in Fig. \ref{['fig:mdp']}.
  • Figure 2: A hybrid road network with two origins, World Expo Museum and Shanghai Station, and a single destination, the Bund, Shanghai's most popular shopping and sightseeing area. Users randomly leave from World Expo Museum and Shanghai Station each day, and subsequently choose a path to reach the Bund.
  • Figure 3: Average long-term social costs (in minutes) under information sharing, hiding, the optimum, and our UPR mechanism versus time horizon $T$.

Theorems & Definitions (15)

  • Definition 1: Selfish routing decision
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Proposition 1
  • Remark 1
  • Proposition 2
  • Remark 2
  • Definition 2: Informational mechanism
  • ...and 5 more