Herding LLaMaS: Using LLMs as an OS Module
Aditya K Kamath, Sujay Yadalam
TL;DR
Addressing the OS burden of heterogeneity, the paper introduces LLaMaS, an LLM-powered OS module that can adapt to new devices from plaintext descriptions. It combines a frontend LLM that derives feature embeddings from system descriptions with a backend prediction model that translates embeddings into runtime OS decisions. The feasibility study uses ChatGPT to demonstrate that LLaMaS can identify memory placement, adapt to novel memory technologies like RAT, and perform task-to-device mappings. The results suggest LLMs can reason about hardware characteristics to enable rapid integration of heterogeneous devices, potentially reducing administrator and developer effort in production systems.
Abstract
Computer systems are becoming increasingly heterogeneous with the emergence of new memory technologies and compute devices. GPUs alongside CPUs have become commonplace and CXL is poised to be a mainstay of cloud systems. The operating system is responsible for managing these hardware resources, requiring modification every time a new device is released. Years of research and development are sunk into tuning the OS for high performance with each new heterogeneous device. With the recent explosion in memory technologies and domain-specific accelerators, it would be beneficial to have an OS that could provide high performance for new devices without significant effort. We propose LLaMaS which can adapt to new devices easily. LLaMaS uses Large Language Models (LLMs) to extract the useful features of new devices from their textual description and uses these features to make operating system decisions at runtime. Adding support to LLaMaS for a new device is as simple as describing the system and new device properties in plaintext. LLaMaS reduces the burden on system administrators to enable easy integration of new devices into production systems. Preliminary evaluation using ChatGPT shows that LLMs are capable of extracting device features from text and make correct OS decisions based on those features.