Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Reality check of the benefits of LLM in business

Ming Cheung

TL;DR

The usefulness and readiness of LLMs for business processes are evaluated through experiments conducted on four accessible LLMs using real-world data and this represents the first quantified study of LLMs applied to core business operations and challenges.

Abstract

Large language models (LLMs) have achieved remarkable performance in language understanding and generation tasks by leveraging vast amounts of online texts. Unlike conventional models, LLMs can adapt to new domains through prompt engineering without the need for retraining, making them suitable for various business functions, such as strategic planning, project implementation, and data-driven decision-making. However, their limitations in terms of bias, contextual understanding, and sensitivity to prompts raise concerns about their readiness for real-world applications. This paper thoroughly examines the usefulness and readiness of LLMs for business processes. The limitations and capacities of LLMs are evaluated through experiments conducted on four accessible LLMs using real-world data. The findings have significant implications for organizations seeking to leverage generative AI and provide valuable insights into future research directions. To the best of our knowledge, this represents the first quantified study of LLMs applied to core business operations and challenges.

A Reality check of the benefits of LLM in business

TL;DR

The usefulness and readiness of LLMs for business processes are evaluated through experiments conducted on four accessible LLMs using real-world data and this represents the first quantified study of LLMs applied to core business operations and challenges.

Abstract

Large language models (LLMs) have achieved remarkable performance in language understanding and generation tasks by leveraging vast amounts of online texts. Unlike conventional models, LLMs can adapt to new domains through prompt engineering without the need for retraining, making them suitable for various business functions, such as strategic planning, project implementation, and data-driven decision-making. However, their limitations in terms of bias, contextual understanding, and sensitivity to prompts raise concerns about their readiness for real-world applications. This paper thoroughly examines the usefulness and readiness of LLMs for business processes. The limitations and capacities of LLMs are evaluated through experiments conducted on four accessible LLMs using real-world data. The findings have significant implications for organizations seeking to leverage generative AI and provide valuable insights into future research directions. To the best of our knowledge, this represents the first quantified study of LLMs applied to core business operations and challenges.
Paper Structure (27 sections, 16 figures, 1 table)

This paper contains 27 sections, 16 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Common LLMs
  4. ChatGPT 3.5
  5. Claude Instant
  6. Llama-2-70b
  7. PaLM 2
  8. LLM for Business
  9. Text Analysis
  10. Content Generation
  11. Translation
  12. Code Generation
  13. Data Analysis
  14. Chatbot with Contexts
  15. Bias
  16. ...and 12 more sections

Figures (16)

  • Figure 1: Examples of a question on LLM with different prompts: (a) the question only; (b) write for sales, (c) write for data scientists.
  • Figure 2: Example of questions on different LLM: (a) ChatGPT; (b) Claude; (c) Llama; (d) PaLM
  • Figure 3: Phrases of using LLM for business project planning, from planning, implementation, to decision making and serving
  • Figure 4: Examples of using LLMs for: (a) Text Analysis; (b) Content Generation.
  • Figure 5: Examples of translations of the same texts: (a) to Chinese; (b) to Chinese with Prompt; (c) to Chinese using Google Translate; (d) to French; (e) to French with Prompt; (f) to French using Google Translate.
  • ...and 11 more figures