Fusion of fuzzy rule-based factual and counterfactual explanations with large language models

In this paper, we explore various strategies for generating trustworthy, natural narrative explanations through the combination of fuzzy rule-based systems with large language models. Specifically, we aimed to leverage contrastive, counterfactual explanations to supplement the factual pieces of information provided by the fuzzy systems to a template-based explainer; and to leverage Chain-of-Thought prompting schemes to improve the generation from language models. We have investigated the validity of the following two research hypotheses: (1) the addition of counterfactuals, which makes for a more complete explanation overall, does not significantly penalize trustworthiness; and (2) Chain-of-Thought prompting induces better discourse structuring and content planning on the language model side, so that it can effectively rewrite the template-based explanations into a higher-quality narrative, yet remaining a truthful explanation. We validated these research hypotheses in a use case in the medical field, comparing several metrics related to predictive performance, trust, brevity and clarity in explanation for several experimental setups. After experimentation, we found that the addition of counterfactual information is useful because it neither increases hallucinatory tendency nor decreases accuracy, hence not harming trustworthiness. However, Chain-of-Thought prompting does not induce the desired effects. On the contrary, it increases verbosity, and in consequence, it harms readability in the final narratives, with no apparent benefits.