Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AROhI: An Interactive Tool for Estimating ROI of Data Analytics

Noopur Zambare, Jacob Idoko, Jagrit Acharya, Gouri Ginde

TL;DR

AROhI addresses the need for ROI-aware evaluation of data analytics by introducing a no-code interactive dashboard that combines conventional ML approaches with advanced methods such as Active Learning and fine-tuned BERT to estimate ROI. The tool enables decision-makers to adjust cost and benefit factors and visualize trade-offs between ML performance (e.g., F1) and ROI, demonstrated on a requirements-dependency extraction use case and deployed on AWS. Key contributions include a concrete ROI model linking benefits (true positives and penalties) to costs (data prep, labeling, and resources) and a workflow for comparing supervised and semi-supervised learners under cost constraints. The work advances practical, ROI-driven decision-making in software analytics and outlines directions for extending to broader datasets, unsupervised learning, and LLM-based ROI calculations.

Abstract

The cost of adopting new technology is rarely analyzed and discussed, while it is vital for many software companies worldwide. Thus, it is crucial to consider Return On Investment (ROI) when performing data analytics. Decisions on "How much analytics is needed"? are hard to answer. ROI could guide decision support on the What?, How?, and How Much? Analytics for a given problem. This work details a comprehensive tool that provides conventional and advanced ML approaches for demonstration using requirements dependency extraction and their ROI analysis as use case. Utilizing advanced ML techniques such as Active Learning, Transfer Learning and primitive Large language model: BERT (Bidirectional Encoder Representations from Transformers) as its various components for automating dependency extraction, the tool outcomes demonstrate a mechanism to compute the ROI of ML algorithms to present a clear picture of trade-offs between the cost and benefits of a technology investment.

AROhI: An Interactive Tool for Estimating ROI of Data Analytics

TL;DR

AROhI addresses the need for ROI-aware evaluation of data analytics by introducing a no-code interactive dashboard that combines conventional ML approaches with advanced methods such as Active Learning and fine-tuned BERT to estimate ROI. The tool enables decision-makers to adjust cost and benefit factors and visualize trade-offs between ML performance (e.g., F1) and ROI, demonstrated on a requirements-dependency extraction use case and deployed on AWS. Key contributions include a concrete ROI model linking benefits (true positives and penalties) to costs (data prep, labeling, and resources) and a workflow for comparing supervised and semi-supervised learners under cost constraints. The work advances practical, ROI-driven decision-making in software analytics and outlines directions for extending to broader datasets, unsupervised learning, and LLM-based ROI calculations.

Abstract

The cost of adopting new technology is rarely analyzed and discussed, while it is vital for many software companies worldwide. Thus, it is crucial to consider Return On Investment (ROI) when performing data analytics. Decisions on "How much analytics is needed"? are hard to answer. ROI could guide decision support on the What?, How?, and How Much? Analytics for a given problem. This work details a comprehensive tool that provides conventional and advanced ML approaches for demonstration using requirements dependency extraction and their ROI analysis as use case. Utilizing advanced ML techniques such as Active Learning, Transfer Learning and primitive Large language model: BERT (Bidirectional Encoder Representations from Transformers) as its various components for automating dependency extraction, the tool outcomes demonstrate a mechanism to compute the ROI of ML algorithms to present a clear picture of trade-offs between the cost and benefits of a technology investment.
Paper Structure (16 sections, 3 equations, 7 figures, 5 tables)

This paper contains 16 sections, 3 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Existing frameworks for measuring the business value of Data Analytics
  4. Industry applications of ROI
  5. Need for ROI estimation
  6. The accuracy fallacy
  7. Evaluating Machine Learning solutions beyond performance
  8. The AROhI Tool
  9. Evaluation Metrics
  10. Use case
  11. Dataset
  12. Execution
  13. Results
  14. ROI Sensitivity Analysis
  15. Limitations
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Conceptual design of the AROhI tool and various technology components of it
  • Figure 2: AROhI Tool's user Interface and various screens of the tool enable users to interact and draw conclusions
  • Figure 3: Overall workflow diagram of AROhI tool: Enables end user to choose the desired supervised or semi-supervised method to evaluate its usefulness for their problem and ROI analysis
  • Figure 4: The login based UI shows descriptive analysis of the uploaded data
  • Figure 5: Various graphs showing relative train set and various ML measures such as F1-Score, Precision and Recall
  • ...and 2 more figures