Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search

Jiangyi Fang, Bowen Zhou, Haotian Wang, Xin Zhu, Leye Wang

TL;DR

The paper addresses online 3D bin packing under short-term distributional shifts by formulating it as an MPC problem and solving it with a Monte Carlo Tree Search that uses a shift-aware exploration prior. It introduces a novel dynamic prior, Shift-Aware PUCB, and a step-wise waste-space penalty to strengthen value backups, enabling robust planning over a finite lookahead horizon. The approach shows substantial gains over state-of-the-art DRL baselines under distributional shifts (exceeding $10\%$) and achieves consistent online improvements (~$4\%$) in deployment, with real-world validation at JD Logistics. The work provides theoretical bounds on performance gaps under non-stationarity and demonstrates practical impact through extensive offline experimentation and live A/B testing.

Abstract

Online 3D Bin Packing (3D-BP) with robotic arms is crucial for reducing transportation and labor costs in modern logistics. While Deep Reinforcement Learning (DRL) has shown strong performance, it often fails to adapt to real-world short-term distribution shifts, which arise as different batches of goods arrive sequentially, causing performance drops. We argue that the short-term lookahead information available in modern logistics systems is key to mitigating this issue, especially during distribution shifts. We formulate online 3D-BP with lookahead parcels as a Model Predictive Control (MPC) problem and adapt the Monte Carlo Tree Search (MCTS) framework to solve it. Our framework employs a dynamic exploration prior that automatically balances a learned RL policy and a robust random policy based on the lookahead characteristics. Additionally, we design an auxiliary reward to penalize long-term spatial waste from individual placements. Extensive experiments on real-world datasets show that our method consistently outperforms state-of-the-art baselines, achieving over 10\% gains under distributional shifts, 4\% average improvement in online deployment, and up to more than 8\% in the best case--demonstrating the effectiveness of our framework.

Effective Online 3D Bin Packing with Lookahead Parcels Using Monte Carlo Tree Search

TL;DR

The paper addresses online 3D bin packing under short-term distributional shifts by formulating it as an MPC problem and solving it with a Monte Carlo Tree Search that uses a shift-aware exploration prior. It introduces a novel dynamic prior, Shift-Aware PUCB, and a step-wise waste-space penalty to strengthen value backups, enabling robust planning over a finite lookahead horizon. The approach shows substantial gains over state-of-the-art DRL baselines under distributional shifts (exceeding ) and achieves consistent online improvements (~) in deployment, with real-world validation at JD Logistics. The work provides theoretical bounds on performance gaps under non-stationarity and demonstrates practical impact through extensive offline experimentation and live A/B testing.

Abstract

Online 3D Bin Packing (3D-BP) with robotic arms is crucial for reducing transportation and labor costs in modern logistics. While Deep Reinforcement Learning (DRL) has shown strong performance, it often fails to adapt to real-world short-term distribution shifts, which arise as different batches of goods arrive sequentially, causing performance drops. We argue that the short-term lookahead information available in modern logistics systems is key to mitigating this issue, especially during distribution shifts. We formulate online 3D-BP with lookahead parcels as a Model Predictive Control (MPC) problem and adapt the Monte Carlo Tree Search (MCTS) framework to solve it. Our framework employs a dynamic exploration prior that automatically balances a learned RL policy and a robust random policy based on the lookahead characteristics. Additionally, we design an auxiliary reward to penalize long-term spatial waste from individual placements. Extensive experiments on real-world datasets show that our method consistently outperforms state-of-the-art baselines, achieving over 10\% gains under distributional shifts, 4\% average improvement in online deployment, and up to more than 8\% in the best case--demonstrating the effectiveness of our framework.
Paper Structure (35 sections, 32 equations, 17 figures, 3 tables)

This paper contains 35 sections, 32 equations, 17 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Theoretical Analysis
  4. Preliminaries on online 3D-BP
  5. Failure in the non-stationary scenario.
  6. Methodology
  7. Tree Search Problem Modeling
  8. MPC-3D-BP
  9. Selection with Shift-Aware Polynomial Upper Confidence Trees
  10. Expansion.
  11. Simulation and Value Backup
  12. Offline Evaluation
  13. Experiment Setup
  14. Datasets, baselines, and experiment settings
  15. Evaluation metrics
  16. ...and 20 more sections

Figures (17)

  • Figure 1: The mismatch between the real-world online scenario and offline DRL training pipeline.
  • Figure 2: Illustration for brute-force tree search solution for our MPC problem of online 3D-BP with lookahead parcels.
  • Figure 3: Step by step illustration for how Shift-Aware Polynomial Upper Confidence Trees guide the selection. The thickness of edges represents the prior of action probability.
  • Figure 4: Illustration for waste space calculation.
  • Figure 5: Performance comparison under different levels.
  • ...and 12 more figures