Empirical study shows alignment techniques improve code quality in LLMs, suggesting key trade-offs for developers.
Key Points
This research investigates the effectiveness of LLM alignment techniques for code generation tasks, focusing on functional and non-functional requirements.
Examined five state-of-the-art LLMs using Direct Preference Optimization and BoNBoN techniques.
Evaluated functional requirements with four benchmarks and non-functional requirements with the CODAL benchmark.
Conducted analyses on both pretrained and finetuned versions of the models.
Pretrained-to-aligned pathways show larger improvements (+75% non-functional, +42% functional).
Pretrained variant generally less accurate than finetuned variant, but alignment narrows the performance gap.
Non-functional requirements consistently improved more than functional through alignment.