Semantic Topic Discovery for Curriculum Analysis of NCERT Chemistry Textbooks Using Transformer-Based Language Models

Objectives: This study aims to develop and compare an AI-based approach for analyzing the conceptual structure of NCERT Chemistry curricula through BERTopic and Latent Dirichlet Allocation (LDA). Method: The textbooks from Classes XI and XII (19 chapters) were gathered from the official NCERT website repository and converted to machine-readable text through OCR. Texts were normalized, filtered, tokenized, lemmatized, and classified into approximately 500-token chunks. Further BERTopic used Sentence Transformer embeddings, HDBSCAN, UMAP, and c-TF-IDF. LDA served as a comparative model. Models were evaluated using Cᵥ topic coherence and qualitative evaluation of topic interpretability and alignment between curriculum and domains. Findings: BERTopic obtained a Cᵥ coherence score of 0.64, contrasted with 0.54 for LDA, indicating an approximately 19% relative optimization. BERTopic also generated more distinct and domain-aligned topics, reflecting major areas including “organic”, “inorganic”, and “physical chemistry”, as well as concepts such as “chemical bonding”, “equilibrium”, “thermodynamics”, “electrochemistry”, and “atomic structure”. Alternatively, LDA produced overlapping and lower interpretable topics, including combinations of “thermodynamics”, “equilibrium,” and “reaction kinetics”. The outcomes are consistent with the expected strength of contextual embeddings over bag-of-words interpretation for capturing semantic relationships. Conversely evaluation was based on coherence and qualitative analysis rather than statistical significance testing. Novelty: The analysis benchmarks BERTopic against LDA on school based NCERT Chemistry textbooks and validates its potential for fine-grained, domain-aligned curriculum assessment.