Benchmarking the Trade-offs between Performance and Energy Consumption in NLP Model Fine-Tuning

This paper presents a detailed empirical study and experience report on software-level optimization techniques aimed at helping practitioners balance model performance with resource consumption during natural language processing model fine-tuning. We conduct a comparative analysis of dynamic batching and selective encoder layer freezing on three Transformer models (BART-base, Flan-T5-Small, and T5-Small), measuring the trade-offs between performance (ROUGE, BERTScore) and resource efficiency (energy, CO2 emissions). Our key finding is that the T5-Small model, when combined with a hybrid optimization approach, offers a near-optimal trade-off: it reduces energy consumption by 80.37% while retaining 99.01% of its baseline ROUGE-1 performance. Across all experiments, these software techniques achieved energy reductions of up to 87.35%. We also report an unexpected finding: dynamic batching, a practical programming technique, can improve model performance, increasing the ROUGE-1 score by up to 3.32% for Flan-T5-Small. Ultimately, this work provides a replicable benchmarking framework and a set of practical guidelines to assist software engineers in designing and implementing more efficient and cost-effective artificial intelligence systems.