March 3, 2026Open Access

Using LLMs to Extract OCL Specifications from Java and Python Programs:An Empirical Study

Puntos clave

Mistral and DeepSeek significantly outperform other LLMs in generating OCL specifications.
Open-source LLMs were tested for both Java and Python programming languages.
The study provides insights into the capabilities of different LLMs in abstraction tasks.
Limitations of the LLMs in extracting OCL specifications were also discussed.

Resumen

This paper presents a comprehensive study of the application of several open-source Large Language Models (LLMs) for abstracting Object Constraint Language (OCL) specifications from source code. We aim to provide researchers and developers with insights into the capabilities and limitations of using different LLMs to abstract OCL specifications from code.We evaluate a collection of open-source LLMs of comparable size (StarCoder2, LLaMA, CodeLlama, Mistral, and DeepSeek) by prompting them to generate OCL specifications for both Java and Python programs. The results show that both Mistral and DeepSeek outperform other LLMs in abstracting OCL specifications from both languages.

Leer artículo completoexternamente

Me gusta

Guardar

Ver artículo completo