Balancing The Perception of Cheating Detection, Privacy and Fairness: A Mixed-Methods Study of Visual Data Obfuscation in Remote Proctoring

TL;DR

Tailoring remote proctoring with region-specific advanced obfuscation methods can improve the perceptions of privacy and fairness compared to the conventional methods, though it may decrease perceived information sufficiency for detecting cheating.

Abstract

Remote proctoring technology, a cheating-preventive measure, often raises privacy and fairness concerns that may affect test-takers' experiences and the validity of test results. Our study explores how selectively obfuscating information in video recordings can protect test-takers' privacy while ensuring effective and fair cheating detection. Interviews with experts (N=9) identified four key video regions indicative of potential cheating behaviors: the test-taker's face, body, background and the presence of individuals in the background. Experts recommended specific obfuscation methods for each region based on privacy significance and cheating behavior frequency, ranging from conventional blurring to advanced methods like replacement with deepfake, 3D avatars and silhouetting. We then conducted a vignette experiment with potential test-takers (N=259, non-experts) to evaluate their perceptions of cheating detection, visual privacy and fairness, using descriptions and examples of still images for each expert-recommended combination of video regions and obfuscation methods. Our results indicate that the effectiveness of obfuscation methods varies by region. Tailoring remote proctoring with region-specific advanced obfuscation methods can improve the perceptions of privacy and fairness compared to the conventional methods, though it may decrease perceived information sufficiency for detecting cheating. However, non-experts preferred conventional blurring for videos they were more willing to share, highlighting a gap between the perceived effectiveness of the advanced obfuscation methods and their practical acceptance. This study contributes to the field of user-centered privacy by suggesting promising directions to address current remote proctoring challenges and guiding future research.

Figures (5)

• Figure 1: Expert recommended pipeline for video obfuscation
• Figure 2: The survey presents an image as a visual scene, followed by hiding a region using various obfuscation methods. For instance, the test-taker's face above is highlighted to indicate where obfuscation is applied. Participants then provided their opinions using Likert scale (L) or open-ended (OE) items, with example items shown on the right side. Genders varied over the ROI in focus: for face, body and background obfuscation, test-takers' genders were varied; for background people, the genders of those people were varied
• Figure 3: Visual scenes are created as stimuli for our vignette experiment, manipulating four regions (face, body, background people in the background) with relevant obfuscation methods. Both male and female subjects are represented varied across regions in focus, resulting in a total of 8 visual scenes. When deepfake was applied to the facial region, the questionnaire was assessed for both the original and changed skin tones of the subjects
• Figure 4: Three panels address three research questions respectively: the impact of region-specific obfuscation methods on (RQ4) perceptions of information sufficiency, privacy and fairness; (RQ5) the combined perception; and (RQ6) willingness to share videos if obfuscated. PANEL 1 and PANEL 2 plot bars relative to the respective baseline values
• Figure 5: The expert interviews followed a sequence of tasks: Task 1 involved listing cheating instances for each region in the image shown; Task 2 focused on identifying and rating (1-10) visual information that could be suppressed through obfuscation without compromising cheating detection. Example only shows cues for face and body. Other regions were also discussed; Task 3 allowed experts to assess the privacy-cheating trade-off for each method in each region. They were required to drag and drop according to their assessment. Task 4 involved a discussion on the obfuscation pipeline, guided by the existing flow diagram of remote proctoring