Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Prompting for LLM-based Generative Internet of Things

Bin Xiao, Burak Kantarci, Jiawen Kang, Dusit Niyato, Mohsen Guizani

TL;DR

Results show that the proposed LLM-based GIoT system can achieve competitive performance compared with state-of-the-art LLMs, demonstrating that the proposed LLM-based GIoT system can provide competitive performance with tailored prompting methods and is easily extensible to new tasks without training.

Abstract

Large language models (LLMs) have demonstrated remarkable capacities on various tasks, and integrating the capacities of LLMs into the Internet of Things (IoT) applications has drawn much research attention recently. Due to security concerns, many institutions avoid accessing state-of-the-art commercial LLM services, requiring the deployment and utilization of open-source LLMs in a local network setting. However, open-source LLMs usually have more limitations regarding their performance, such as their arithmetic calculation and reasoning capacities, and practical systems of applying LLMs to IoT have yet to be well-explored. Therefore, we propose a LLM-based Generative IoT (GIoT) system deployed in the local network setting in this study. To alleviate the limitations of LLMs and provide service with competitive performance, we apply prompt engineering methods to enhance the capacities of the open-source LLMs, design a Prompt Management Module and a Post-processing Module to manage the tailored prompts for different tasks and process the results generated by the LLMs. To demonstrate the effectiveness of the proposed system, we discuss a challenging Table Question Answering (Table-QA) task as a case study of the proposed system, as tabular data is usually more challenging than plain text because of their complex structures, heterogeneous data types and sometimes huge sizes. We conduct comprehensive experiments on two popular Table-QA datasets, and the results show that our proposal can achieve competitive performance compared with state-of-the-art LLMs, demonstrating that the proposed LLM-based GIoT system can provide competitive performance with tailored prompting methods and is easily extensible to new tasks without training.

Efficient Prompting for LLM-based Generative Internet of Things

TL;DR

Results show that the proposed LLM-based GIoT system can achieve competitive performance compared with state-of-the-art LLMs, demonstrating that the proposed LLM-based GIoT system can provide competitive performance with tailored prompting methods and is easily extensible to new tasks without training.

Abstract

Large language models (LLMs) have demonstrated remarkable capacities on various tasks, and integrating the capacities of LLMs into the Internet of Things (IoT) applications has drawn much research attention recently. Due to security concerns, many institutions avoid accessing state-of-the-art commercial LLM services, requiring the deployment and utilization of open-source LLMs in a local network setting. However, open-source LLMs usually have more limitations regarding their performance, such as their arithmetic calculation and reasoning capacities, and practical systems of applying LLMs to IoT have yet to be well-explored. Therefore, we propose a LLM-based Generative IoT (GIoT) system deployed in the local network setting in this study. To alleviate the limitations of LLMs and provide service with competitive performance, we apply prompt engineering methods to enhance the capacities of the open-source LLMs, design a Prompt Management Module and a Post-processing Module to manage the tailored prompts for different tasks and process the results generated by the LLMs. To demonstrate the effectiveness of the proposed system, we discuss a challenging Table Question Answering (Table-QA) task as a case study of the proposed system, as tabular data is usually more challenging than plain text because of their complex structures, heterogeneous data types and sometimes huge sizes. We conduct comprehensive experiments on two popular Table-QA datasets, and the results show that our proposal can achieve competitive performance compared with state-of-the-art LLMs, demonstrating that the proposed LLM-based GIoT system can provide competitive performance with tailored prompting methods and is easily extensible to new tasks without training.
Paper Structure (19 sections, 1 equation, 11 figures, 8 tables)

This paper contains 19 sections, 1 equation, 11 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Generative Models in IoT
  4. Table Question Answering
  5. Prompting Methods
  6. System Model
  7. Case Study of Table-QA for the LLM-based GIoT System
  8. Problem Formulation and Analysis
  9. Proposed Prompting Method for Table-QA
  10. Overall Workflow
  11. Demonstration Crafting and Selection
  12. Experiments and Analysis
  13. Datasets and Experimental Settings
  14. Discussion and Analysis
  15. Ablation Study
  16. ...and 4 more sections

Figures (11)

  • Figure 1: Overall architecture of the proposed LLM-based GIoT system.
  • Figure 2: Detailed workflow of the proposed LLM-based GIoT system.
  • Figure 3: Comparison of CoT, PoT and the proposed method. It is worth mentioning that many details are omitted due to space limitations. The proposed method contains task-planning, task-conducting, and task-correction stages, and it uses a statistical table and sub-tables in these stages to avoid the original large tables.
  • Figure 4: An example of a semi-structured table and its failed Python code because of the heterogeneous data types and table's complex structure. The Python code is generated by Mixtral-8x7B.
  • Figure 5: The workflow of the proposed prompting solution. Notably, the question and the table are from the request of an IoT device. The Python interpreter is in the Post-processing Module, and the stages of selecting demonstrations and creating prompts are in the Prompt Management Module.
  • ...and 6 more figures