Application-Driven Innovation in Machine Learning

TL;DR

This paper argues that application-driven innovation is undervalued in the machine learning community and presents ADML as a paradigm that complements traditional methods-driven research by focusing on real-world tasks, data heterogeneity, and stakeholder needs. It contrasts Methods-Driven and Application-Driven paradigms, detailing how application-specific benchmarks, domain knowledge, and constrained data influence algorithm design and evaluation, while also showing how ADML can feed back into ML methodology through cross-domain insights. The authors critique current reviewing, hiring, and teaching practices that hinder ADML and offer concrete recommendations, including clearer review guidelines, expanded publication venues, data engineering support, and interdisciplinary curricula. They also acknowledge ethical considerations and advocate for stakeholder-centric design to maximize positive societal impact, while recognizing that not all applications are beneficial or aligned with values. Overall, the paper argues for integrating ADML into mainstream ML to broaden impact, diversify research directions, and accelerate deployment of effective, real-world AI solutions.

Abstract

In this position paper, we argue that application-driven research has been systemically under-valued in the machine learning community. As applications of machine learning proliferate, innovative algorithms inspired by specific real-world challenges have become increasingly important. Such work offers the potential for significant impact not merely in domains of application but also in machine learning itself. In this paper, we describe the paradigm of application-driven research in machine learning, contrasting it with the more standard paradigm of methods-driven research. We illustrate the benefits of application-driven machine learning and how this approach can productively synergize with methods-driven work. Despite these benefits, we find that reviewing, hiring, and teaching practices in machine learning often hold back application-driven innovation. We outline how these processes may be improved.

Figures (3)

• Figure 1: We distinguish between two paradigms of research in machine learning: Methods-driven innovation, in which algorithms are designed based on their performance on standardized benchmarks, and application-driven innovation, in which algorithms are designed to meet challenges faced in real-world problems. We argue that both paradigms contribute significantly to ML research.
• Figure 2: Application-driven innovation in machine learning is achieved through close integration of ML research with applications and end-users. Machine learning methods are designed to be impactful on real-world problems. In turn, applications contribute to ML research methods via novel datasets and task framing, informed by auxiliary domain information such as constraints and metadata. End-users also help define relevant criteria for measuring the success of ML methods on downstream tasks.
• Figure 3: We outline how to alleviate bottlenecks holding back application-driven ML research across reviewing, hiring, and teaching practices. In each of these settings, current practices are largely aligned with methods-driven research and ADML work is often under-valued.