Sampling Audit Evidence Using a Naive Bayes Classifier

TL;DR

This work addresses audit data overload by integrating a Naive Bayes classifier with sampling to generate audit evidence from large and potentially unstructured datasets. It leverages Bayes' theorem with conditional independence, $P(C_i|X) \propto P(C_i)\prod_{k=1}^n P(X_k|C_i)$, to produce posterior probabilities that drive three sampling modes: user-based (symmetric around class medians), item-based (asymmetric toward riskier samples), and a hybrid of the two, evaluated by a representativeness index RI. Through three experiments on ad-click data, spam messages, and the Panama Papers graph, the approach yields unbiased samples, handles unstructured data, and concentrates on higher-risk observations, often outperforming RF and SVM baselines in large-scale contexts. However, the method’s effectiveness hinges on classifier accuracy and careful selection of priors and thresholds (e.g., $P_-$, $P_+$, $\sigma_1$, $\sigma_2$, $\sigma_3$), underscoring the need for validation and calibration before deployment in auditing practice.

Abstract

Taiwan's auditors have suffered from processing excessive audit data, including drawing audit evidence. This study advances sampling techniques by integrating machine learning with sampling. This machine learning integration helps avoid sampling bias, keep randomness and variability, and target risker samples. We first classify data using a Naive Bayes classifier into some classes. Next, a user-based, item-based, or hybrid approach is employed to draw audit evidence. The representativeness index is the primary metric for measuring its representativeness. The user-based approach samples data symmetric around the median of a class as audit evidence. It may be equivalent to a combination of monetary and variable samplings. The item-based approach represents asymmetric sampling based on posterior probabilities for obtaining risky samples as audit evidence. It may be identical to a combination of non-statistical and monetary samplings. Auditors can hybridize those user-based and item-based approaches to balance representativeness and riskiness in selecting audit evidence. Three experiments show that sampling using machine learning integration has the benefits of drawing unbiased samples, handling complex patterns, correlations, and unstructured data, and improving efficiency in sampling big data. However, the limitations are the classification accuracy output by machine learning algorithms and the range of prior probabilities.

Figures (11)

• Figure S1: Bayes' theorem
• Figure S2: Construction of a posterior probability distribution
• Figure S3: Distributions of attributes $X_{ij}, (i = 1-4, j = 1, 2\ldots,N$) values in Experiment 1
• Figure S4: ROC curves provided by different machine learning algorithms and the confusion matrix output by Equations (3)-(4) for Experiment 1
• Figure S5: Audit evidence for 50 % confidence intervals