Distilling structural reasoning: efficient semantic parsing via chain-of-thought rationalization and contrastive demonstration selection

In order to use hierarchical semantic parsing, models must be able to generalize to complex, compositional logic structures in addition to comprehending natural language. Although chain-of-thought prompting has produced state-of-the-art results for large language models, two major obstacles to their deployment are structural hallucinations resulting from surface-level demonstration retrieval and prohibitive inference latency. This paper introduces a neuro-symbolic framework called distilled structural reasoning, which uses semantic fragment decoding to overcome these bottlenecks. We present three new contributions: (1) contrastive fragment selection, a retrieval mechanism trained to separate lexical noise from structural intent, allowing zero-shot schema adaptation; (2) trie-based grammar constraints, which remove syntax errors during decoding; and (3) a parser-critic feedback loop for semantic self-correction. Most importantly, we suggest a rationale distillation protocol that converts a 70B-parameter teacher’s structural reasoning traces into a compact 8B-parameter student. Our framework achieves a new state-of-the-art for open-source models (87.42% exact match on TOP), outperforming standard supervised baselines and reducing inference latency by 9.4× when compared to the teacher model, according to extensive experiments on TOP, TOPv2, ATIS, and SNIPS.