What does this research mean for the field?

FeatureSHAP, a model-agnostic explainability framework based on Shapley values, provides higher fidelity explanations for LLM outputs in software engineering tasks at a lower computational cost than token-level alternatives, significantly improving practitioner interpretation. Novelty: ClaimNovelty.METHODOLOGICAL. Consensus alignment: ConsensusAlignment.NEUTRAL.

What question did this study set out to answer?

To introduce and evaluate FeatureSHAP, a model-agnostic explainability framework designed to explain LLM decisions in software engineering tasks using Shapley values.

August 24, 2026ACM Transactions on Software Engineering and MethodologyOpen Access

Toward Explaining Large Language Models in Software Engineering Tasks

View Full Paper

Ask AI

Bookmark

Authors

AVAntonio VitalePolytechnic University of Turin KNKhai-Nguyen NguyenWilliam & Mary DPDenys PoshyvanykWilliam & Mary

Discussion

Member takes

Overview

Evaluation study reveals that FeatureSHAP improves explanation fidelity and reduces computational cost in software engineering LLMs, suggesting a path to transparent AI development.

Key Points

To introduce and evaluate FeatureSHAP, a model-agnostic explainability framework designed to explain LLM decisions in software engineering tasks using Shapley values.
Designed FeatureSHAP to attribute outputs to high-level input features via systematic input perturbation and task-specific similarity metrics across open-source and proprietary LLMs.
Evaluated the framework on code generation and code summarization tasks alongside a practitioner survey with 37 participants.
FeatureSHAP assigned less importance to irrelevant features and achieved higher fidelity explanations compared with baseline methods.
The framework operated at substantially lower computational cost than token-level alternatives and improved decision-making among 37 surveyed software practitioners.

Cite This Study

Vitale et al. (2026) studied this question.

synapsesocial.com/papers/6a8c0010bca056c88e6dee70 https://doi.org/10.1145/3837084

View Full Paper

Ask AI

Bookmark

August 24, 2026ACM Transactions on Software Engineering and MethodologyOpen Access

Toward Explaining Large Language Models in Software Engineering Tasks

View Full Paper

Ask AI

Bookmark

Authors

AVAntonio VitalePolytechnic University of Turin KNKhai-Nguyen NguyenWilliam & Mary DPDenys PoshyvanykWilliam & Mary

Discussion

Member takes

Overview

Evaluation study reveals that FeatureSHAP improves explanation fidelity and reduces computational cost in software engineering LLMs, suggesting a path to transparent AI development.

Key Points

To introduce and evaluate FeatureSHAP, a model-agnostic explainability framework designed to explain LLM decisions in software engineering tasks using Shapley values.
Designed FeatureSHAP to attribute outputs to high-level input features via systematic input perturbation and task-specific similarity metrics across open-source and proprietary LLMs.
Evaluated the framework on code generation and code summarization tasks alongside a practitioner survey with 37 participants.
FeatureSHAP assigned less importance to irrelevant features and achieved higher fidelity explanations compared with baseline methods.
The framework operated at substantially lower computational cost than token-level alternatives and improved decision-making among 37 surveyed software practitioners.

Cite This Study

Vitale et al. (2026) studied this question.

synapsesocial.com/papers/6a8c0010bca056c88e6dee70 https://doi.org/10.1145/3837084

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

View Full Paper

Ask AI

Bookmark