Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Socialz: Multi-Feature Social Fuzz Testing

Francisco Zanartu, Christoph Treude, Markus Wagner

TL;DR

Socialz tackles the challenge of fuzz testing online social networks by combining data-driven user characterisation with evolutionary diversification across multiple non-trivial features and in-vivo execution on a live OSN (GitLab CE). It introduces a three-stage methodology: characterize real users from GitHub data, evolve diversified interactions using a star-discrepancy objective and a self-adaptive (1+20)-EA, and execute the evolved interactions to collect performance data and reveal bugs. The study demonstrates that evolutionary diversification produces more evenly distributed and diverse user behaviors than replayed or random baselines, uncovering practical platform limitations and a sizable set of errors during fuzzing. The work provides a publicly available framework and dataset to advance social testing and suggests integrating social fuzzing with traditional code- or system-level testing for broader impact.

Abstract

Online social networks have become an integral aspect of our daily lives and play a crucial role in shaping our relationships with others. However, bugs and glitches, even minor ones, can cause anything from frustrating problems to serious data leaks that can have farreaching impacts on millions of users. To mitigate these risks, fuzz testing, a method of testing with randomised inputs, can provide increased confidence in the correct functioning of a social network. However, implementing traditional fuzz testing methods can be prohibitively difficult or impractical for programmers outside of the social network's development team. To tackle this challenge, we present Socialz, a novel approach to social fuzz testing that (1) characterises real users of a social network, (2) diversifies their interaction using evolutionary computation across multiple, non-trivial features, and (3) collects performance data as these interactions are executed. With Socialz, we aim to put social testing tools in everybody's hands, thereby improving the reliability and security of social networks used worldwide. In our study, we came across (1) one known limitation of the current GitLab CE and (2) 6,907 errors, of which 40.16% are beyond our debugging skills.

Socialz: Multi-Feature Social Fuzz Testing

TL;DR

Socialz tackles the challenge of fuzz testing online social networks by combining data-driven user characterisation with evolutionary diversification across multiple non-trivial features and in-vivo execution on a live OSN (GitLab CE). It introduces a three-stage methodology: characterize real users from GitHub data, evolve diversified interactions using a star-discrepancy objective and a self-adaptive (1+20)-EA, and execute the evolved interactions to collect performance data and reveal bugs. The study demonstrates that evolutionary diversification produces more evenly distributed and diverse user behaviors than replayed or random baselines, uncovering practical platform limitations and a sizable set of errors during fuzzing. The work provides a publicly available framework and dataset to advance social testing and suggests integrating social fuzzing with traditional code- or system-level testing for broader impact.

Abstract

Online social networks have become an integral aspect of our daily lives and play a crucial role in shaping our relationships with others. However, bugs and glitches, even minor ones, can cause anything from frustrating problems to serious data leaks that can have farreaching impacts on millions of users. To mitigate these risks, fuzz testing, a method of testing with randomised inputs, can provide increased confidence in the correct functioning of a social network. However, implementing traditional fuzz testing methods can be prohibitively difficult or impractical for programmers outside of the social network's development team. To tackle this challenge, we present Socialz, a novel approach to social fuzz testing that (1) characterises real users of a social network, (2) diversifies their interaction using evolutionary computation across multiple, non-trivial features, and (3) collects performance data as these interactions are executed. With Socialz, we aim to put social testing tools in everybody's hands, thereby improving the reliability and security of social networks used worldwide. In our study, we came across (1) one known limitation of the current GitLab CE and (2) 6,907 errors, of which 40.16% are beyond our debugging skills.
Paper Structure (28 sections, 5 figures, 3 tables)

This paper contains 28 sections, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Testing of Social Networks
  4. Evolutionary Fuzzing
  5. Diversity Optimisation
  6. The Methodology of Socialz: Overview
  7. Target platform.
  8. Characterisation of User Behaviour (Stage 1/3)
  9. Evolutionary Diversification of Community Interaction (Stage 2/3)
  10. Features of Community Interaction
  11. Solution Evaluation
  12. Evolutionary Algorithm
  13. Diversified Community Interaction
  14. Execution (Stage 3/3)
  15. Benchmarking the Evolutionary Approach
  16. ...and 13 more sections

Figures (5)

  • Figure 1: Grafana dashboard example, showing statistics of our GitLab CE server.
  • Figure 2: Evolution of the interaction graph. For 30 independent runs, red shows the average discrepancy (of user behaviour), and grey shows the number of mutations (light grey: min-max range; dark grey: 95% confidence interval).
  • Figure 3: Dataset comparison: user interaction based on interaction features. The 2d plots are projections of the 3d plots. The value ranges are always $\left[0,1\right]$ based on the minimum and maximum values across all four interaction graphs.
  • Figure 4: Processing of community interactions over time.
  • Figure 5: Correlation of user features and resource utilisation. Shown are the averages for each of the 1523 users.