Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeepEN: A Deep Reinforcement Learning Framework for Personalized Enteral Nutrition in Critical Care

Daniel Jason Tan, Jiayang Chen, Dilruk Perera, Kay Choong See, Mengling Feng

TL;DR

DeepEN presents a conservative offline RL framework to personalize enteral nutrition in critical care, addressing the challenge of dynamic, patient-specific metabolic needs. It employs a Duelling Double DQN with Conservative Q-Learning, trained on 11k+ ICU patients from MIMIC-IV, to generate four-hourly targets for calories, protein, and water, using a state space of 102 features and a 51-action space. The reward function blends terminal outcomes (mortality) with intermediate physiological and biomarker signals, and the evaluation via CWPDIS shows reduced estimated mortality (18.8% vs 22.5%) and higher expected returns, with a strong negative return-mortality correlation. Findings support the feasibility of conservative offline RL for individualized EN therapy and suggest data-driven personalization can outperform guideline- or clinician-based strategies, while highlighting the need for external validation and exploration of richer rewards.

Abstract

ICU enteral feeding remains sub-optimal due to limited personalization and uncertainty about appropriate calorie, protein, and fluid targets, particularly under rapidly changing metabolic demands and heterogeneous patient responses. This study introduces DeepEN, a reinforcement learning (RL)-based framework that personalizes enteral nutrition (EN) dosing for critically ill patients using electronic health record data. DeepEN was trained on over 11,000 ICU patients from the MIMIC-IV database to generate 4-hourly, patient-specific targets for caloric, protein, and fluid intake. The model's state space integrates demographics, comorbidities, vital signs, laboratory results, and prior interventions relevant to nutritional management, while its reward function balances short-term physiological and nutrition-related goals with long-term survival. A dueling double deep Q-network with Conservative Q-Learning regularization is used to ensure safe and reliable policy learning from retrospective data. DeepEN achieved a 3.7 $\pm$ 0.17 percentage-point absolute reduction in estimated mortality compared with the clinician policy (18.8% vs 22.5%) and higher expected returns compared with guideline-based dosing (11.89 vs 8.11), with improvements in key nutritional biomarkers. U-shaped associations between deviations from clinician dosing and mortality suggest that the learned policy aligns with high-value clinician actions while diverging from suboptimal ones. These findings demonstrate the feasibility of conservative offline RL for individualized EN therapy and suggest that data-driven personalization may improve outcomes beyond guideline- or heuristic-based approaches.

DeepEN: A Deep Reinforcement Learning Framework for Personalized Enteral Nutrition in Critical Care

TL;DR

DeepEN presents a conservative offline RL framework to personalize enteral nutrition in critical care, addressing the challenge of dynamic, patient-specific metabolic needs. It employs a Duelling Double DQN with Conservative Q-Learning, trained on 11k+ ICU patients from MIMIC-IV, to generate four-hourly targets for calories, protein, and water, using a state space of 102 features and a 51-action space. The reward function blends terminal outcomes (mortality) with intermediate physiological and biomarker signals, and the evaluation via CWPDIS shows reduced estimated mortality (18.8% vs 22.5%) and higher expected returns, with a strong negative return-mortality correlation. Findings support the feasibility of conservative offline RL for individualized EN therapy and suggest data-driven personalization can outperform guideline- or clinician-based strategies, while highlighting the need for external validation and exploration of richer rewards.

Abstract

ICU enteral feeding remains sub-optimal due to limited personalization and uncertainty about appropriate calorie, protein, and fluid targets, particularly under rapidly changing metabolic demands and heterogeneous patient responses. This study introduces DeepEN, a reinforcement learning (RL)-based framework that personalizes enteral nutrition (EN) dosing for critically ill patients using electronic health record data. DeepEN was trained on over 11,000 ICU patients from the MIMIC-IV database to generate 4-hourly, patient-specific targets for caloric, protein, and fluid intake. The model's state space integrates demographics, comorbidities, vital signs, laboratory results, and prior interventions relevant to nutritional management, while its reward function balances short-term physiological and nutrition-related goals with long-term survival. A dueling double deep Q-network with Conservative Q-Learning regularization is used to ensure safe and reliable policy learning from retrospective data. DeepEN achieved a 3.7 0.17 percentage-point absolute reduction in estimated mortality compared with the clinician policy (18.8% vs 22.5%) and higher expected returns compared with guideline-based dosing (11.89 vs 8.11), with improvements in key nutritional biomarkers. U-shaped associations between deviations from clinician dosing and mortality suggest that the learned policy aligns with high-value clinician actions while diverging from suboptimal ones. These findings demonstrate the feasibility of conservative offline RL for individualized EN therapy and suggest that data-driven personalization may improve outcomes beyond guideline- or heuristic-based approaches.

Paper Structure

This paper contains 34 sections, 9 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Reinforcement Learning
  4. Q-learning and Offline RL
  5. Addressing overestimation bias with D3QN and CQL in offline RL
  6. RL for Enteral Nutrition Management
  7. Methodology
  8. RL Problem Definition
  9. States
  10. Actions
  11. Rewards
  12. Terminal Reward ($R_{\text{term}}(m)$):
  13. Intermediate Rewards ($R_{\text{im}}(m)$):
  14. Experimental Setup
  15. Data Extraction and Preprocessing
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Overview of the DeepEN solution pipeline.
  • Figure 2: Mortality vs. expected returns. The shaded area represents the confidence interval.
  • Figure 3: Dosage differences (x-axis) versus mortality (y-axis) for all policies. The shaded area indicates the confidence interval.
  • Figure 4: Dosage differences (x-axis) versus biomarker deviation (y-axis) for DeepEN. (A) Glucose deviation vs. calories dosage difference and (B) Phosphate deviation vs. protein dosage.