EPFL · Wetenschap

AI System Learns to Doubt for Optimized Scientific Discovery

Researchers at EPFL have developed GOLLuM, a framework that trains Large Language Models (LLMs) to create AI systems capable of optimizing scientific experiments. This method combines LLMs with a 'doubt detector' to improve the efficiency and reliability of scientific discovery.

Modern computational tools allow scientists to explore vast numbers of potential molecules, materials, and chemical reactions, but laboratory testing of every combination is time-consuming and expensive. Bayesian optimization is one method that learns from previous results to predict promising options and estimate uncertainty, guiding researchers to focus on the most useful experiments. However, this method often needs to be restarted for different scientific fields.

The emergence of LLMs offers a new approach, as they already possess broad scientific knowledge and process information as text. A key challenge with LLMs is 'hallucination,' where their apparent confidence doesn't always correlate with accuracy, potentially leading scientists down unproductive paths. To address this, Bojana Ranković and Philippe Schwaller at EPFL's Laboratory of Artificial Chemical Intelligence created GOLLuM (Gaussian Process Optimized LLMs), which integrates an LLM with a Gaussian process, a probabilistic model used for uncertainty estimation in Bayesian optimization.

GOLLuM trains LLMs by using the Gaussian process's scoring of uncertainty and performance for each option, rather than direct prompting. As the LLM learns from past experiments, it reorganizes the 'search space' of possible experimental choices. Conditions leading to similar results are grouped together, while dissimilar ones are separated. This process allows the AI to learn from its uncertainty, which is a known weakness of standard LLMs.

The researchers tested GOLLuM on 23 benchmark tasks across various scientific domains, including organic synthesis, chemistry, materials science, and molecular property optimization. Starting with ten initial observations, GOLLuM consistently outperformed traditional Bayesian optimization with expert-designed descriptors, ranking first on average. Within a budget of 50 experiments, GOLLuM identified high-performing conditions in the top 5% of all possible outcomes 36.3% of the time, compared to 29.7% for the traditional method. Overall, GOLLuM achieved comparable results to traditional methods using over 40% fewer experiments.

When LLMs were used to choose experiments directly without the 'doubt detector,' their performance was inconsistent, with failure rates ranging from 10% to 80%. Issues included generating fabricated chemical structures, repeating conditions, and suggesting options outside the defined search space. GOLLuM offers a new role for LLMs in research, combining their broad scientific information representation with probabilistic models that explicitly handle uncertainty, enabling optimization campaigns to begin immediately using plain English descriptions of experimental procedures.

AI-samenvatting op basis van de bron.