Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Early Results from Teaching Modelling for Software Comprehension in New-Hire Onboarding

Mrityunjay Kumar, Venkatesh Choppella

TL;DR

The study addresses the gap between industry expectations and graduate preparation for maintenance and comprehension of large, existing software systems. It tests a short, five-session onboarding course teaching systems thinking and Labelled Transition Systems (LTS) modelling, using a structured articulation template to measure comprehension. Across 31 paired participants, the overall gain was not significant, but those below the pre-test median improved by $+15.1$ percentage points ($p=0.017$), while above-median learners showed a non-significant decline, indicating a differential effect. These findings suggest modelling-based onboarding can accelerate understanding for less-prepared hires and motivate differentiated onboarding paths, with future work to establish causal claims and scalable implementations.

Abstract

Working effectively with large, existing software systems requires strong comprehension skills, yet most graduates enter the industry with little preparation for this challenge. We report early results from a pilot intervention integrated into a SaaS company's onboarding program: a five-session course introducing systems thinking and Labelled Transition System (LTS) modelling. Participants articulated their understanding of product behaviour using a structured template and completed matched pre- and post-assessments. Of 35 new hires, 31 provided paired records for analysis. Across the full cohort, gains were small and not statistically significant. However, participants below the median on the pre-test improved by 15 percentage points on average (statistically significant), while those above the median regressed slightly (not statistically significant). Course feedback indicated high engagement and perceived applicability. These results suggest that short, modelling-focused onboarding interventions can accelerate comprehension for less-prepared new hires. At the same time, they point to the need for differentiated pathways for stronger participants, and to the potential for companies to adopt such interventions at scale as a low-cost complement to existing onboarding.

Early Results from Teaching Modelling for Software Comprehension in New-Hire Onboarding

TL;DR

The study addresses the gap between industry expectations and graduate preparation for maintenance and comprehension of large, existing software systems. It tests a short, five-session onboarding course teaching systems thinking and Labelled Transition Systems (LTS) modelling, using a structured articulation template to measure comprehension. Across 31 paired participants, the overall gain was not significant, but those below the pre-test median improved by percentage points (), while above-median learners showed a non-significant decline, indicating a differential effect. These findings suggest modelling-based onboarding can accelerate understanding for less-prepared hires and motivate differentiated onboarding paths, with future work to establish causal claims and scalable implementations.

Abstract

Working effectively with large, existing software systems requires strong comprehension skills, yet most graduates enter the industry with little preparation for this challenge. We report early results from a pilot intervention integrated into a SaaS company's onboarding program: a five-session course introducing systems thinking and Labelled Transition System (LTS) modelling. Participants articulated their understanding of product behaviour using a structured template and completed matched pre- and post-assessments. Of 35 new hires, 31 provided paired records for analysis. Across the full cohort, gains were small and not statistically significant. However, participants below the median on the pre-test improved by 15 percentage points on average (statistically significant), while those above the median regressed slightly (not statistically significant). Course feedback indicated high engagement and perceived applicability. These results suggest that short, modelling-focused onboarding interventions can accelerate comprehension for less-prepared new hires. At the same time, they point to the need for differentiated pathways for stronger participants, and to the potential for companies to adopt such interventions at scale as a low-cost complement to existing onboarding.

Paper Structure

This paper contains 15 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Experiment Design
  4. Context
  5. Intervention and Content
  6. Assessment
  7. Data Collection
  8. Results
  9. Background questionnaire
  10. Learning Outcomes
  11. Discussion
  12. Practical Implications
  13. Limitations
  14. Conclusion
  15. Acknowledgements

Figures (2)

  • Figure 1: Positioning the intervention within the broader onboarding program. The dashed box shows the study subset, which runs in parallel with technical training; the bold text highlights the key result for the below-baseline cohort (baseline=pre-test median).
  • Figure 2: Learning gains by subgroup (below-baseline vs above-baseline, baseline=pre-test median).