Building, EnergyLLM: A Domain-Specific Large Language Model Trained on SPE Content

Abstract This work describes how we built and evaluated "EnergyLLM," a large language model like OpenAI's GPT-4o for the energy domain. The project is a collaboration among Aramco, SPE, and i2k Connect. The model was trained on SPE's content including OnePetro papers, SPE-published books, and PetroWiki. Our goal was to produce a language model that knows more about the energy domain, specifically petroleum engineering, than other available models. We fine-tuned Llama 3, Meta's open source foundation model, with training data we generated from SPE's content. We also developed an embedding model for search applications that we built from EnergyLLM's training data. We evaluated the performance of EnergyLLM with subject matter experts in a head-to-head comparison with the Llama 3 model. Results showed that SMEs significantly preferred EnergyLLM. We also evaluated the embedding model on retrieval tasks, and asked ChatGPT how it thinks EnergyLLM compares to its own responses.