Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fine-Tuned Thoughts: Leveraging Chain-of-Thought Reasoning for Industrial Asset Health Monitoring

Shuxin Lin, Dhaval Patel, Christodoulos Constantinides

TL;DR

The paper tackles the challenge of enabling reasoning in small language models for industrial asset health monitoring by transferring chain-of-thought reasoning from large models through a seed-free, KG-inspired data generation pipeline. It generates synthetic MCQA data with rationale using CoT prompts, labels via majority voting across multiple LLMs, and rigorous quality filtering to fine-tune 8B SLMs with QLoRA/LoRA. Experiments on the FailureSensorIQ benchmark show that CoT-informed SLMs outperform baselines and narrow the gap to LLMs, with notable gains in single-correct accuracy and robustness to data variations. The approach is cost-effective, scalable, and open-sourced, offering a practical route for industry to deploy reasoning-capable SLMs, while highlighting challenges around factual fidelity and perturbation robustness that warrant future work.

Abstract

Small Language Models (SLMs) are becoming increasingly popular in specialized fields, such as industrial applications, due to their efficiency, lower computational requirements, and ability to be fine-tuned for domain-specific tasks, enabling accurate and cost-effective solutions. However, performing complex reasoning using SLMs in specialized fields such as Industry 4.0 remains challenging. In this paper, we propose a knowledge distillation framework for industrial asset health, which transfers reasoning capabilities via Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) to smaller, more efficient models (SLMs). We discuss the advantages and the process of distilling LLMs using multi-choice question answering (MCQA) prompts to enhance reasoning and refine decision-making. We also perform in-context learning to verify the quality of the generated knowledge and benchmark the performance of fine-tuned SLMs with generated knowledge against widely used LLMs. The results show that the fine-tuned SLMs with CoT reasoning outperform the base models by a significant margin, narrowing the gap to their LLM counterparts. Our code is open-sourced at: https://github.com/IBM/FailureSensorIQ.

Fine-Tuned Thoughts: Leveraging Chain-of-Thought Reasoning for Industrial Asset Health Monitoring

TL;DR

The paper tackles the challenge of enabling reasoning in small language models for industrial asset health monitoring by transferring chain-of-thought reasoning from large models through a seed-free, KG-inspired data generation pipeline. It generates synthetic MCQA data with rationale using CoT prompts, labels via majority voting across multiple LLMs, and rigorous quality filtering to fine-tune 8B SLMs with QLoRA/LoRA. Experiments on the FailureSensorIQ benchmark show that CoT-informed SLMs outperform baselines and narrow the gap to LLMs, with notable gains in single-correct accuracy and robustness to data variations. The approach is cost-effective, scalable, and open-sourced, offering a practical route for industry to deploy reasoning-capable SLMs, while highlighting challenges around factual fidelity and perturbation robustness that warrant future work.

Abstract

Small Language Models (SLMs) are becoming increasingly popular in specialized fields, such as industrial applications, due to their efficiency, lower computational requirements, and ability to be fine-tuned for domain-specific tasks, enabling accurate and cost-effective solutions. However, performing complex reasoning using SLMs in specialized fields such as Industry 4.0 remains challenging. In this paper, we propose a knowledge distillation framework for industrial asset health, which transfers reasoning capabilities via Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) to smaller, more efficient models (SLMs). We discuss the advantages and the process of distilling LLMs using multi-choice question answering (MCQA) prompts to enhance reasoning and refine decision-making. We also perform in-context learning to verify the quality of the generated knowledge and benchmark the performance of fine-tuned SLMs with generated knowledge against widely used LLMs. The results show that the fine-tuned SLMs with CoT reasoning outperform the base models by a significant margin, narrowing the gap to their LLM counterparts. Our code is open-sourced at: https://github.com/IBM/FailureSensorIQ.
Paper Structure (25 sections, 2 figures, 14 tables)

This paper contains 25 sections, 2 figures, 14 tables.

Table of Contents

  1. Introduction
  2. SLM Challenges for Asset Health Monitoring
  3. Contributions
  4. Related Work
  5. Methodology
  6. KG-based Instruction Generation
  7. Options Generation
  8. Pseudo Ground Truth Labelling
  9. Rationale Generation
  10. Quality Filtering
  11. Experiments
  12. Generated Data Statistics
  13. Generated Data Factuality Analysis
  14. Benchmark Dataset
  15. Evaluation Metrics
  16. ...and 10 more sections

Figures (2)

  • Figure 1: Proposed Knowledge Distillation Framework: Workflow leading to Fine-Tune
  • Figure 2: Real knowledge capacity measurement with two knowledge-invariant perturbations: Option ID Shifting (Perturb-OIDS), and Knowledge-Invariant Question Paraphrasing (Perturb-KIQP).