Towards trustworthy early warning systems: Human-centered explainable AI for food recall classification

Foodborne illnesses and product recalls pose growing public health threats, yet automatic classifiers that label food-recall texts with hazards and products remain opaque “black boxes.” This paper addresses the transparency gap by systematically comparing model-specific (Integrated Gradients, DeepLift, Input×Gradient, GradientSHAP) and model-agnostic (LIME, KernelSHAP, Shapley Value Sampling, Feature Ablation) explainability methods for multi-class food hazard and product classification, using expert-annotated word-level rationales as the gold standard for plausibility. We trained transformer models (BERT, RoBERTa, ModernBERT, and Llama 3.2) on Food Recall Incidents under Weighted Cross-Entropy, Focal, Dice, and a custom Hybrid loss, finding that Weighted Cross-Entropy consistently yields the best performance. Explainability is evaluated by measuring the overlap between each method’s top-5 attributed words and human rationales with lexical (BLEU, ROUGE, IoU, Token-F1, Token-Recall, Token-Precision) and semantic (BERTScore, COMET) metrics. Results show that Integrated Gradients and LIME produce the most expert-aligned explanations across both hazard and product tasks, while KernelSHAP and DeepLift show the lowest agreement. Class-level analysis reveals specific hazard and product categories where model explanations diverge from human reasoning, exposing potential dataset ambiguities and labelling challenges. These findings provide practical guidance for selecting explainability techniques in food-safety AI systems and contribute toward more transparent, human-centered early-warning tools. The code implementation is available at: https://github.com/foteinipapadopoulou/xAIForFoodHazard .