NutriScreener: Retrieval-Augmented Multi-Pose Graph Attention Network for Malnourishment Screening

TL;DR

NutriScreener tackles scalable malnutrition screening in children by fusing retrieval-augmented learning with a multi-pose graph attention network that leverages CLIP-based visual embeddings. A FAISS-backed knowledge base provides semantically similar exemplars to boost minority-class predictions, with a context-aware fusion layer balancing retrieval and GNN outputs for both classification and regression. Across cross-continental pediatric and adult datasets, NutriScreener achieves high recall and AUC while delivering low RMSEs for anthropometrics, and ablations show its robustness to pose variability and KB density. Clinician feedback confirms deployment readiness in low-resource settings, highlighting the approach’s practical impact for early malnutrition detection and triage, with future work focusing on expanding the KB and improving interpretability and uncertainty estimation.

Abstract

Child malnutrition remains a global crisis, yet existing screening methods are laborious and poorly scalable, hindering early intervention. In this work, we present NutriScreener, a retrieval-augmented, multi-pose graph attention network that combines CLIP-based visual embeddings, class-boosted knowledge retrieval, and context awareness to enable robust malnutrition detection and anthropometric prediction from children's images, simultaneously addressing generalizability and class imbalance. In a clinical study, doctors rated it 4.3/5 for accuracy and 4.6/5 for efficiency, confirming its deployment readiness in low-resource settings. Trained and tested on 2,141 children from AnthroVision and additionally evaluated on diverse cross-continent populations, including ARAN and an in-house collected CampusPose dataset, it achieves 0.79 recall, 0.82 AUC, and significantly lower anthropometric RMSEs, demonstrating reliable measurement in unconstrained pediatric settings. Cross-dataset results show up to 25% recall gain and up to 3.5 cm RMSE reduction using demographically matched knowledge bases. NutriScreener offers a scalable and accurate solution for early malnutrition detection in low-resource environments.

Figures (9)

• Figure 1: Robustness of NutriScreener to detect malnourishment across challenging real-world malnutrition scenarios.
• Figure 2: Architectural overview of NutriScreener. The system takes a subject’s multi-pose images and age as input. It extracts pose-wise visual embeddings using a CLIP-based encoder and aggregates them, then passes the aggregated embeddings through a graph attention network to produce image-based predictions for both classification and regression tasks. In parallel, the same aggregated embeddings are used to query a FAISS-indexed knowledge base, yielding retrieval-based predictions. An adaptive fusion layer then combines the graph-based and retrieval-based outputs.
• Figure 4: Calibration and Decision-Curve Analysis for NutriScreener (Weighted). Calibration plot shows the agreement between predicted probabilities and observed outcomes (ECE = 0.06, MCE = 0.26, Brier = 0.16). Decision-curve analysis illustrates net benefit across thresholds: the model achieves maximal NB = 0.16 at threshold $\tau = 0.25$, corresponding to 16 additional correct decisions per 100 patients compared with baseline strategies. Dashed red and black lines represent the treat-all and treat-none policies; the dotted blue line marks the theoretical upper bound at the cohort prevalence (0.31).
• Figure 5: t-SNE of global embeddings showing Adult (blue), MalKB (orange), H1 (green), H2 (red), and AV test (purple). The close overlap of MalKB and AV clusters explains its superior retrieval performance.
• Figure 6: Screenshot of NutriScreener toolkit