Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LDMDroid: Leveraging LLMs for Detecting Data Manipulation Errors in Android Apps

Xiangyang Xiao, Huaxun Huang, Rongxin Wu

Abstract

Android apps rely heavily on Data Manipulation Functionalities (DMFs) for handling app-specific data through CRUDS operations, making their correctness vital for reliability. However, detecting Data Manipulation Errors (DMEs) is challenging due to their dependence on specific UI interaction sequences and manifestation as logic bugs. Existing automated UI testing tools face two primary challenges: insufficient UI path coverage for adequate DMF triggering and reliance on manually written test scripts. To address these issues, we propose an automated approach using Large Language Models (LLMs) for DME detection. We developed LDMDroid, an automated UI testing framework for Android apps. LDMDroid enhances DMF triggering success by guiding LLMs through a state-aware process for generating UI event sequences. It also uses visual features to identify changes in data states, improving DME verification accuracy. We evaluated LDMDroid on 24 real-world Android apps, demonstrating improved DMF triggering success rates compared to baselines. LDMDroid discovered 17 unique bugs, with 14 confirmed by developers and 11 fixed. The tool is publicly available at https://github.com/runnnnnner200/LDMDroid.

LDMDroid: Leveraging LLMs for Detecting Data Manipulation Errors in Android Apps

Abstract

Android apps rely heavily on Data Manipulation Functionalities (DMFs) for handling app-specific data through CRUDS operations, making their correctness vital for reliability. However, detecting Data Manipulation Errors (DMEs) is challenging due to their dependence on specific UI interaction sequences and manifestation as logic bugs. Existing automated UI testing tools face two primary challenges: insufficient UI path coverage for adequate DMF triggering and reliance on manually written test scripts. To address these issues, we propose an automated approach using Large Language Models (LLMs) for DME detection. We developed LDMDroid, an automated UI testing framework for Android apps. LDMDroid enhances DMF triggering success by guiding LLMs through a state-aware process for generating UI event sequences. It also uses visual features to identify changes in data states, improving DME verification accuracy. We evaluated LDMDroid on 24 real-world Android apps, demonstrating improved DMF triggering success rates compared to baselines. LDMDroid discovered 17 unique bugs, with 14 confirmed by developers and 11 fixed. The tool is publicly available at https://github.com/runnnnnner200/LDMDroid.

Paper Structure

This paper contains 23 sections, 2 equations, 3 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Data Manipulation
  3. Motivation
  4. Approach
  5. Identifying DUMs
  6. Collecting DMF Instances
  7. General Steps for Triggering DMFs
  8. Planning UI Actions
  9. Extracting UI Changes
  10. Tracking the Task Progress
  11. Validating the Correctness of DMFs
  12. Discover Sibling DMFs
  13. Testing with DMF Instances
  14. Evaluation
  15. Evaluation Setup
  16. ...and 8 more sections

Figures (3)

  • Figure 1: A DME in Material Files (v1.7.4) related to "Create File" DMF. The small red boxes indicate UI events. The expected outcome (f) shows the newly created file appearing in the file list, while the actual outcome (e) reveals its absence, demonstrating the DME.
  • Figure 2: The Overview of LDMDroid.
  • Figure 3: Illustrative examples of LDMDroid detections. (a,b) show a true positive: a deleted habit item remains visible in the list, revealing a logic DME. (c,d) show a false positive: a mismatch between the summary view and detail page was incorrectly flagged, though no DME occurred.