A Unifying Bias-aware Multidisciplinary Framework for Investigating Socio-Technical Issues

TL;DR

The paper addresses the challenge of studying socio-technical issues with bias by proposing a unifying bias-aware multidisciplinary framework that fuses social science methods with machine learning. It operationalizes the framework through a PRIME project case study on minoritised ethnic communities' online harms in digitalised social housing, employing inductive coding, topic modelling (CorEx, BERTopic, BiTerm), sentiment analysis with explainable AI, SHAP visualizations, and latent class analysis for verification against EVENS and PRIME survey data. Key findings include the identification of vulnerabilities (discrimination, digital poverty, digital illiteracy, English proficiency) and the observation that Black African communities are more likely to experience these vulnerabilities in access and outcomes, with heterogeneous patterns across ME groups. The work demonstrates rigorous bias-awareness, reflexivity, and data triangulation as essential components, and offers a flexible methodological blueprint for applying the framework to diverse socio-technical inquiries with potential policy and design implications for harm-reduction in digital public services.

Abstract

This paper aims to bring together the disciplines of social science (SS) and computer science (CS) in the design and implementation of a novel multidisciplinary framework for systematic, transparent, ethically-informed, and bias-aware investigation of socio-technical issues. For this, various analysis approaches from social science and machine learning (ML) were applied in a structured sequence to arrive at an original methodology of identifying and quantifying objects of inquiry. A core feature of this framework is that it highlights where bias occurs and suggests possible steps to mitigate it. This is to improve the robustness, reliability, and explainability of the framework and its results. Such an approach also ensures that the investigation of socio-technical issues is transparent about its own limitations and potential sources of bias. To test our framework, we utilised it in the multidisciplinary investigation of the online harms encountered by minoritised ethnic (ME) communities when accessing and using digitalised social housing services in the UK. We draw our findings from 100 interviews with ME individuals in four cities across the UK to understand ME vulnerabilities when accessing and using digitalised social housing services. In our framework, a sub-sample of interviews focusing on ME individuals residing in social housing units were inductively coded. This resulted in the identification of the topics of discrimination, digital poverty, lack of digital literacy, and lack of English proficiency as key vulnerabilities of ME communities. Further ML techniques such as Topic Modelling and Sentiment Analysis were used within our framework where we found that Black African communities are more likely to experience these vulnerabilities in the access, use and outcome of digitalised social housing services.

Figures (15)

• Figure 1: The cyclic nature of data, interaction with data, and, data analysis methods/algorithms feeding into each other and building up on biases
• Figure 2: The proposed bias-aware multidisciplinary framework for systematic investigation of socio-technical issues: Part1 (top) refers to the data collection stage of the framework whereas Part2 (bottom) refers to the data analysis stage of the framework.
• Figure 3: Variant 1: Using different machine learning algorithms in parallel, with multiple possible review cycles. Results of parallel algorithms can be compared, aggregated, selected, and used for fine-tuning before a desired result is achieved. This review cycle need to done in close collaboration with all multidisciplinary research stakeholders.
• Figure 4: Variant 2: Using different machine learning algorithms in a linear fashion
• Figure 5: Variant 3: Using different machine learning algorithms in a hybrid modality