Big Data and the Computational Social Science of Entrepreneurship and Innovation

TL;DR

The chapter argues that large-scale, unstructured social data can transform entrepreneurship and innovation research by enabling precise, high-resolution measurements and virtual experimentation. It advocates a dual data/model paradigm in which data are used both as inputs for observation and as sources for digital doubles created by transformers and LLMs, enabling system-level observatories and virtual laboratories. This approach addresses three core challenges—novelty identification, origins scarcity, and functional equivalence—through embeddings, multi-modal data integration, and counterfactual simulations. The resulting framework promises to raise theory development, improve education and policy relevance, and accelerate investment and technology development through data-driven insights.

Abstract

As large-scale social data explode and machine-learning methods evolve, scholars of entrepreneurship and innovation face new research opportunities but also unique challenges. This chapter discusses the difficulties of leveraging large-scale data to identify technological and commercial novelty, document new venture origins, and forecast competition between new technologies and commercial forms. It suggests how scholars can take advantage of new text, network, image, audio, and video data in two distinct ways that advance innovation and entrepreneurship research. First, machine-learning models, combined with large-scale data, enable the construction of precision measurements that function as system-level observatories of innovation and entrepreneurship across human societies. Second, new artificial intelligence models fueled by big data generate 'digital doubles' of technology and business, forming laboratories for virtual experimentation about innovation and entrepreneurship processes and policies. The chapter argues for the advancement of theory development and testing in entrepreneurship and innovation by coupling big data with big models.

Figures (1)

• Figure 1: The comparative r-squared values for linear model predictions of performance across four categories of scientific projects (performance is measured by parameter number for ML/AI models [ML]/ [AI], current density for semiconductors [SM], energy density for batteries [BA]; the two bars for each category contrast the predictive power of models using only semantic embeddings against those enhanced with both semantic and author embedding information.