Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

iSNEAK: Partial Ordering as Heuristics for Model-Based Reasoning in Software Engineering

Andre Lustosa, Tim Menzies

TL;DR

The paper tackles information overload in model-based software engineering by introducing iSNEAK, an incremental AI solver that uses partial orderings to quickly rank and prune candidate solutions with minimal human input. It combines recursive bi-clustering (via FASTMAP Nyström PCA approximation), discretization, distance-based division, and a two-pass ranking scheme (automatic and human-in-the-loop) to converge on high-quality configurations. Empirical results across diverse case studies (XOMO variants, POM3, and SPLOT SPL models) show iSNEAK achieves human-acceptable solutions with significantly fewer questions than prior state-of-the-art methods, while maintaining 100% solution validity. The approach reduces cognitive load, mitigates information overload, and preserves human preferences without sacrificing optimization quality, offering practical impact for complex, multi-goal SE problems.$

Abstract

A "partial ordering" is a way to heuristically order a set of examples (partial orderings are a set where, for certain pairs of elements, one precedes the other). While these orderings may only be approximate, they can be useful for guiding a search towards better regions of the data. To illustrate the value of that technique, this paper presents iSNEAK, an incremental human-in-the-loop AI problem solver. iSNEAK uses partial orderings and feedback from humans to prune the space of options. Further, in experiments with a dozen software models of increasing size and complexity (with up to 10,000 variables), iSNEAK only asked a handful of questions to return human-acceptable solutions that outperformed the prior state-of-the-art. We propose the use of partial orderings and tools like iSNEAK to solve the information overload problem where human experts grow fatigued and make mistakes when they are asked too many questions. iSNEAK mitigates the information overload problem since it allows humans to explore complex problem spaces in far less time, with far less effort.

iSNEAK: Partial Ordering as Heuristics for Model-Based Reasoning in Software Engineering

TL;DR

The paper tackles information overload in model-based software engineering by introducing iSNEAK, an incremental AI solver that uses partial orderings to quickly rank and prune candidate solutions with minimal human input. It combines recursive bi-clustering (via FASTMAP Nyström PCA approximation), discretization, distance-based division, and a two-pass ranking scheme (automatic and human-in-the-loop) to converge on high-quality configurations. Empirical results across diverse case studies (XOMO variants, POM3, and SPLOT SPL models) show iSNEAK achieves human-acceptable solutions with significantly fewer questions than prior state-of-the-art methods, while maintaining 100% solution validity. The approach reduces cognitive load, mitigates information overload, and preserves human preferences without sacrificing optimization quality, offering practical impact for complex, multi-goal SE problems.$

Abstract

A "partial ordering" is a way to heuristically order a set of examples (partial orderings are a set where, for certain pairs of elements, one precedes the other). While these orderings may only be approximate, they can be useful for guiding a search towards better regions of the data. To illustrate the value of that technique, this paper presents iSNEAK, an incremental human-in-the-loop AI problem solver. iSNEAK uses partial orderings and feedback from humans to prune the space of options. Further, in experiments with a dozen software models of increasing size and complexity (with up to 10,000 variables), iSNEAK only asked a handful of questions to return human-acceptable solutions that outperformed the prior state-of-the-art. We propose the use of partial orderings and tools like iSNEAK to solve the information overload problem where human experts grow fatigued and make mistakes when they are asked too many questions. iSNEAK mitigates the information overload problem since it allows humans to explore complex problem spaces in far less time, with far less effort.
Paper Structure (37 sections, 7 equations, 8 figures, 3 tables, 2 algorithms)

This paper contains 37 sections, 7 equations, 8 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Current Limitations
  3. Background
  4. Model-based Software Engineering is Widely Used
  5. Interactive Model-Based Software Engineering
  6. Models Can Confuse People
  7. Partial Orderings
  8. Why Not Explore Models Using Traditional Optimization Methods?
  9. Reasoning over Many Goals
  10. SBSE and Interactive search-based SE
  11. SMBO = Sequential Model-Based Optimization
  12. Our Proposal: The iSNEAK Algorithm
  13. Recursive Bi-Clustering
  14. Discretization
  15. Distance, Projection, and Division
  16. ...and 22 more sections

Figures (8)

  • Figure 1: SCRUM Feature Model. Such feature models define features and their dependencies, typically in the form of a feature diagram + left-over (a.k.a. cross-tree) constraints. Numbers on leaves show the numbers of constraints in that part of the model. Not shown here, for space reasons, are "cross-tree constraints" that connect choices in different subtrees.
  • Figure 2: Experimental results from altering the size of each interation on the Scrum model of Figure \ref{['fig:scrumModel']}
  • Figure 3: According to Boehm and Turner boehm2003balancing, agile is best suited for projects in the middle of this figure.
  • Figure 4: RQ1 results. Distribution of ratings given by human experts to presented solutions. The x-axis denotes human ratings and the y-axis shows the frequency counts by which humans offered those ratings.
  • Figure 5: RQ1 results: Human effort required for each solution, in minutes
  • ...and 3 more figures