Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry

TL;DR

The paper reports the outcomes of the 2024 LLM Hackathon for Applications in Materials Science and Chemistry, highlighting 34 team submissions across seven application domains and a hybrid global event format that connected physical hubs to online participation. It documents a broad spectrum of approaches, from retrieval-augmented generation and multimodal reasoning to agent-based workflows and knowledge-graph integration, demonstrating substantial advances in LLM capabilities for materials science and chemistry since the prior year. Key contributions include end-to-end LLM-driven design pipelines for peptides and MOFs, natural-language interfaces to simulation software, RAG-based data extraction and narrative generation, and several multimodal benchmarks for chemistry reasoning. The results underscore both the practical value of LLMs as multipurpose ML tools and their potential as rapid prototyping platforms for domain-specific scientific tasks, with implications for education, data management, and collaborative research.

Abstract

Here, we present the outcomes from the second Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistry, which engaged participants across global hybrid locations, resulting in 34 team submissions. The submissions spanned seven key application areas and demonstrated the diverse utility of LLMs for applications in (1) molecular and material property prediction; (2) molecular and material design; (3) automation and novel interfaces; (4) scientific communication and education; (5) research data management and automation; (6) hypothesis generation and evaluation; and (7) knowledge extraction and reasoning from scientific literature. Each team submission is presented in a summary table with links to the code and as brief papers in the appendix. Beyond team results, we discuss the hackathon event and its hybrid format, which included physical hubs in Toronto, Montreal, San Francisco, Berlin, Lausanne, and Tokyo, alongside a global online hub to enable local and virtual collaboration. Overall, the event highlighted significant improvements in LLM capabilities since the previous year's hackathon, suggesting continued expansion of LLMs for applications in materials science and chemistry research. These outcomes demonstrate the dual utility of LLMs as both multipurpose models for diverse machine learning tasks and platforms for rapid prototyping custom applications in scientific research.

Figures (40)

• Figure 1: LLM Hackathon for Applications in Materials and Chemistry hybrid hackathon. Researchers were able to participate from both remote and in-person locations (purple pins).
• Figure 2: Schematic depicting the prompt for fine-tuning the LLM with Alpaca prompt format.
• Figure 3: Model architecture and the schema of the second experiment. Material composition is encoded with Roost encoder, additional information extracted from cited paper with Llama3, and encoded with Mat(Sci)Bert. Composition and LLM embeddings are aggregated and passed through the Residual Net projection head to predict the property. At the inference stage, the average LLM embedding from 5 nearest neighbors in composition space is taken. Results show MAE for adding different types of context (top to bottom): adding random context; not adding context; adding consistently structured context for chemical and structural family (data extracted by humans); adding automatically extracted context for experimental conditions and structure-property relationship.
• Figure 4:
• Figure 5: