Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cooperative Patrol Routing: Optimizing Urban Crime Surveillance through Multi-Agent Reinforcement Learning

Juan Palma-Borda, Eduardo Guzmán, María-Victoria Belmonte

TL;DR

This work tackles urban crime surveillance by designing unpredictable, cooperative patrol routes using a decentralized partially observable MARL framework. Among evaluated algorithms, Value Decomposition PPO (VDPPO) best learned coordinated policies that maximize coverage of high-crime areas within a limited shift, without predefining target nodes. A novel coverage index, inspired by criminology's PAI, quantifies how well the routes prioritize the top hotspots, and experiments on three Málaga zones demonstrate effective coverage of the top $3\%$ of nodes and meaningful coverage for larger fractions under realistic constraints. The findings indicate that a moderate number of patrols and partial information are sufficient for strong hotspot coverage, with random initial placements being less effective than hotspot-informed starts, highlighting practical implications for urban policing resource allocation. Future work will extend the framework to more steps, transformer-based architectures, and dynamic adjustments to patrol counts in response to incidents.

Abstract

The effective design of patrol strategies is a difficult and complex problem, especially in medium and large areas. The objective is to plan, in a coordinated manner, the optimal routes for a set of patrols in a given area, in order to achieve maximum coverage of the area, while also trying to minimize the number of patrols. In this paper, we propose a multi-agent reinforcement learning (MARL) model, based on a decentralized partially observable Markov decision process, to plan unpredictable patrol routes within an urban environment represented as an undirected graph. The model attempts to maximize a target function that characterizes the environment within a given time frame. Our model has been tested to optimize police patrol routes in three medium-sized districts of the city of Malaga. The aim was to maximize surveillance coverage of the most crime-prone areas, based on actual crime data in the city. To address this problem, several MARL algorithms have been studied, and among these the Value Decomposition Proximal Policy Optimization (VDPPO) algorithm exhibited the best performance. We also introduce a novel metric, the coverage index, for the evaluation of the coverage performance of the routes generated by our model. This metric is inspired by the predictive accuracy index (PAI), which is commonly used in criminology to detect hotspots. Using this metric, we have evaluated the model under various scenarios in which the number of agents (or patrols), their starting positions, and the level of information they can observe in the environment have been modified. Results show that the coordinated routes generated by our model achieve a coverage of more than $90\%$ of the $3\%$ of graph nodes with the highest crime incidence, and $65\%$ for $20\%$ of these nodes; $3\%$ and $20\%$ represent the coverage standards for police resource allocation.

Cooperative Patrol Routing: Optimizing Urban Crime Surveillance through Multi-Agent Reinforcement Learning

TL;DR

This work tackles urban crime surveillance by designing unpredictable, cooperative patrol routes using a decentralized partially observable MARL framework. Among evaluated algorithms, Value Decomposition PPO (VDPPO) best learned coordinated policies that maximize coverage of high-crime areas within a limited shift, without predefining target nodes. A novel coverage index, inspired by criminology's PAI, quantifies how well the routes prioritize the top hotspots, and experiments on three Málaga zones demonstrate effective coverage of the top of nodes and meaningful coverage for larger fractions under realistic constraints. The findings indicate that a moderate number of patrols and partial information are sufficient for strong hotspot coverage, with random initial placements being less effective than hotspot-informed starts, highlighting practical implications for urban policing resource allocation. Future work will extend the framework to more steps, transformer-based architectures, and dynamic adjustments to patrol counts in response to incidents.

Abstract

The effective design of patrol strategies is a difficult and complex problem, especially in medium and large areas. The objective is to plan, in a coordinated manner, the optimal routes for a set of patrols in a given area, in order to achieve maximum coverage of the area, while also trying to minimize the number of patrols. In this paper, we propose a multi-agent reinforcement learning (MARL) model, based on a decentralized partially observable Markov decision process, to plan unpredictable patrol routes within an urban environment represented as an undirected graph. The model attempts to maximize a target function that characterizes the environment within a given time frame. Our model has been tested to optimize police patrol routes in three medium-sized districts of the city of Malaga. The aim was to maximize surveillance coverage of the most crime-prone areas, based on actual crime data in the city. To address this problem, several MARL algorithms have been studied, and among these the Value Decomposition Proximal Policy Optimization (VDPPO) algorithm exhibited the best performance. We also introduce a novel metric, the coverage index, for the evaluation of the coverage performance of the routes generated by our model. This metric is inspired by the predictive accuracy index (PAI), which is commonly used in criminology to detect hotspots. Using this metric, we have evaluated the model under various scenarios in which the number of agents (or patrols), their starting positions, and the level of information they can observe in the environment have been modified. Results show that the coordinated routes generated by our model achieve a coverage of more than of the of graph nodes with the highest crime incidence, and for of these nodes; and represent the coverage standards for police resource allocation.
Paper Structure (19 sections, 9 equations, 7 figures, 7 tables)

This paper contains 19 sections, 9 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Multi-Agent Reinforcement Learning
  4. Patrol Route Design
  5. Methodology
  6. Definition of the Problem
  7. The Environment Representation
  8. Actions
  9. Observations
  10. Rewards
  11. Model Parameters
  12. Urban Crime Surveillance Optimization
  13. The Urban Environment
  14. Assumptions
  15. Evaluation
  16. ...and 4 more sections

Figures (7)

  • Figure 1: Flowchart of the urban crime surveillance with our model.
  • Figure 2: Urban areas selected for patrol routing, Málaga, Spain: zones 3 (blue), 9 (purple), and 10 (green).
  • Figure 3: Number of crimes by typology in each studied area.
  • Figure 4: Graphs generated from the selected areas in Figure \ref{['fig:figure_zones_model']}: zone 3 (left), 9 (center), and 10 (right).
  • Figure 5: Comparison of the maximum rewards obtained with IPPO (red), MAPPO (cyan), VDPPO (purple), VD2AC (yellow), MATPRO (green), using a line of sight of 3.
  • ...and 2 more figures