A Comparative Analysis of Wealth Index Predictions in Africa between three Multi-Source Inference Models

Abstract

Poverty map inference has become a critical focus of research, utilizing both traditional and modern techniques, ranging from regression models to convolutional neural networks applied to tabular data, satellite imagery, and networks. While much attention has been given to validating models during the training phase, the final predictions have received less scrutiny. In this study, we analyze the International Wealth Index (IWI) predicted by Lee and Braithwaite (2022) and Espín-Noboa et al. (2023), alongside the Relative Wealth Index (RWI) inferred by Chi et al. (2022), across six Sub-Saharan African countries. Our analysis reveals trends and discrepancies in wealth predictions between these models. In particular, significant and unexpected discrepancies between the predictions of Lee and Braithwaite and Espín-Noboa et al., even after accounting for differences in training data. In contrast, the shape of the wealth distributions predicted by Espín-Noboa et al. and Chi et al. are more closely aligned, suggesting similar levels of skewness. These findings raise concerns about the validity of certain models and emphasize the importance of rigorous audits for wealth prediction algorithms used in policy-making. Continuous validation and refinement are essential to ensure the reliability of these models, particularly when they inform poverty alleviation strategies.

Figures (14)

• Figure 1: Available GDP per capita and Gini index. Top: GDP values are aligned with the years of the surveys used in the models (x-axis). Bottom: The most recent available Gini indices are shown for reference. Uganda is the only country with GDP and Gini index data available for approximately the same years as the surveys. Both metric values are provided by the World Bank world_bank_gdp_per_capitaworld_bank_gini.
• Figure 2: IWI predictions in South Africa, ZAF (M2: CNN$_a$+CB). Both models were trained on the same ground-truth data, thus, we expect similar results. (a) Poverty maps. Color bars display the average predicted IWI scores, centered at the mean of M2's predictions for comparison. Top: Prediction at the original resolution, OSM populated places. Both M1 and M2 include the Marion Island, omitted here for visualization purposes. Bottom: Aggregated mean wealth at the district level (administrative level 2). Compared to M2, M1 predicts higher wealth. (b) Predicted wealth distribution (top: PDF, bottom: CDF). ZAF is the second country with the highest GDP in our study (see \ref{['fig:preliminaries']}), and this is reflected in the prediction by both models (e.g., $\mu>48$). M2 predicts a skewed, unimodal distribution of wealth, while M1 predicts an unusual multimodal distribution. IWI poverty line ($35^{th}$ percentile, corresponding to poverty headcount ratios at $1.25 a day smits2015international) slightly differs between models.
• Figure 3: IWI predictions at overlapping places in South Africa, ZAF (M2: CNN$_a$+CB). Right: IWI differences between the predictions of M1 and M2 for each overlapping place. Left: Correlation between these predictions. Colors indicate the direction of the differences: M1 generally overestimates wealth compared to M2 (mostly blue). Moreover, the wealth distribution in these locations is equally skewed (similar Gini coefficient).
• Figure A1: IWI predictions in SLE (M2: CB). (a) Poverty maps with colors representing the average predicted IWI scores. Top: OSM populated places. Bottom: District level. M1 predicts lower wealth than M2. (b) Predicted wealth distribution. IWI poverty line ($35^{th}$ percentile, corresponding to poverty headcount ratios at $1.25 a day smits2015international) slightly differs between models.
• Figure A2: IWI predictions in overlapping places in SLE (M2: CB). Right: Prediction differences between M1 and M2. Left: Correlation between these predictions. Compared to M2, M1 underestimates wealth (mostly red) and predicts more inequality (higher Gini coefficient).