Answers are grounded in this paper and cite their sources.
The requirement classification technique helps with planning, quality assurance, architecture decisions, and risk management. However, traditional metrics regard all misclassifications as equally costly. In this work, we evaluate Term Frequency-Inverse Document Frequency (TF-IDF) with Logistic Regression, TF-IDF with LinearSVC, Qwen3.5-4B, and Gemma 4 E2B IT on PROMISE_exp in two tasks: binary functional/non-functional classification and 11-class NFR subtype classification. In addition to common metrics, we provide four risk metrics: Risk-Weighted Error (RWE), Criticality Miss Rate (CMR) by triage level, Critical-Alert Precision (CAP), and Critical Subtype Error Rate (CSER). Uncertainty is measured with 2,000 paired class-stratified bootstrap replicates. LinearSVC achieves the highest accuracy both in Task A (0.907) and Task B (0.752), and the lowest Task B RWE (0.235). On gold-labelled NFRs, Qwen3.5-4B shows the best criticality-triage point estimates: CMR=0.073, CAP=0.911, and CSER=0.127. Its paired CMR and CSER differences against Gemma exclude zero, whereas its comparisons with the two classical baselines remain uncertain. Sensitivity analysis reveals that this advantage of Qwen3.5-4B does not persist when the six critical classes are considered equally important. Its differences against Gemma lose support when Gemma's five invalid outputs are removed. When Task A errors are included in triage, Qwen misses 19/55 critical requirements compared with 7/55 for LinearSVC. Thus, risk metrics give the most meaningful results in combination with class-specific, output validity, and pipeline-level analysis.
No takes yet. Share an insight, caveat, or question.
Ekiz et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: