Learning from sanctioned government suppliers: A machine learning and network science approach to detecting fraud and corruption in Mexico

TL;DR

Using publicly available data on federally funded procurement in Mexico and company sanction records, this study implements positive-unlabeled (PU) learning algorithms that integrate domain-knowledge-based red flags with network-derived features to identify likely corrupt and fraudulent contracts.

Abstract

Detecting fraud and corruption in public procurement remains a major challenge for governments worldwide. Most research to-date builds on domain-knowledge-based corruption risk indicators of individual contract-level features and some also analyzes contracting network patterns. A critical barrier for supervised machine learning is the absence of confirmed non-corrupt, negative, examples, which makes conventional machine learning inappropriate for this task. Using publicly available data on federally funded procurement in Mexico and company sanction records, this study implements positive-unlabeled (PU) learning algorithms that integrate domain-knowledge-based red flags with network-derived features to identify likely corrupt and fraudulent contracts. The best-performing PU model on average captures 32 percent more known positives and performs on average 2.3 times better than random guessing, substantially outperforming approaches based solely on traditional red flags. The analysis of the Shapley Additive Explanations reveals that network-derived features, particularly those associated with contracts in the network core or suppliers with high eigenvector centrality, are the most important. Traditional red flags further enhance model performance in line with expectations, albeit mainly for contracts awarded through competitive tenders. This methodology can support law enforcement in Mexico, and it can be adapted to other national contexts too.

Figures (18)

• Figure 1: Distribution of contracts and labels across years. The height of each bar represents the percentage of contracts (y-axis) in a given year (x-axis) relative to the total number of contracts in the dataset. The dark blue in the bars represent the unlabeled contracts in the dataset, meanwhile the light blue rectangles and the percentages at the top of the distribution indicate the percentage of positive labels in that specific year.
• Figure 2: Model's performance. In panel (a) each blue line corresponds to the robust cumulative sum ($C_R(k)$) --top row-- or the lift curve ($L_c(K$) --bottom row-- of each of the 4-fold cross-validation sets for the examined models in comparison with a random classifier (red dashed line). The colored area corresponds to the gain: the difference between the robust cumulative sum and the null model. *Ranking by CRI implies to rank the contracts using only the CRI and estimate how well it captures labeled contracts without any machine learning methodology. The figures indicate that the HDSRF outperforms the PU Bagging model for every set. Moreover, the straight line at value zero up to $\sim$ 0.5 normalized ranks in the PU Bagging algorithms shows that around 50% of the observations in the test set are classified with the highest predicted probability, which makes unfeasible to work for ranking purposes. The figures on panel (b) correspond to the results of the permutation testing for our best HDSRF model. We observe that in both cases, the average gain and average lift of our model overpass all the models created with permuted labels.
• Figure 3: Top 30 most important features of the model. In (a) we observe the top 30 most important features of our HDSRF according to their mean absolute SHAP value, where the top 10 features are dominated by network features (7 out of 10). The accumulated influence of different variable types is shown in (b). The network features of the model have the highest sum of absolute mean SHAP values. The position of individual features over the entire set of features is depicted in (c). The top positions are a mixture of network and domain-knowledge features, meanwhile the bottom ones are dominated by domain-knowledge features, even though they are present in all of the ranked bins. The definition of each feature can be read in Supplementary Information \ref{['features-definition']}. * These are features that are take domain-knowledge information by leveraging the network structure
• Figure 4: Mean SHAP values of top 30 across transductive and inductive (EPN and AMLO administration) learning. We can observe the non-absolute SHAP values of each feature per instance colored by the values of the feature. All of them shows the average SHAP values of 30 features ordered by the absolute SHAP value in the transductive setting. The empty spaces show that a feature is not present in the top 30 for that setting.
• Figure 5: Selected network features' SHAP dependence plots. This figure presents the SHAP dependence plots for ten selected network features from the model and its Pearson correlation($\rho$), along with the distribution of the feature in the background (in light blue). The y-axis indicates the features' contribution to the predicted probability of being fraudulent (SHAP value) for each contract, while the x-axis indicates the original values of the feature. The plots in the left column correspond to features related to the supplier, while those on the right correspond to the buyer. Colors correspond to the values of other variables included in the model label. The $*, **, ***$ symbolize the belonging of the feature to the top 10, 20 and 30 most important features of the model,respectively. The grey band around zero represents the 10th and 90th quantiles of the SHAP value distribution across all features, and the blue histogram in the background the distribution of the feature analyzed in the given figure.