PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
January 1, 2000591 citationsOpen Access

Comparing corpora using frequency profiling

PRPaul RaysonRGRoger Garside

Key Points

  • To introduce and evaluate a frequency profiling methodology for comparing text corpora and extracting distinctive lexical, grammatical, and semantic characteristics.
  • Implemented frequency profiling techniques on standard and annotated text corpora.
  • Evaluated the method across distinct linguistic applications, including sociolinguistic variation, language learner profiling, and software engineering documentation.
  • Identified distinguishing keywords, grammatical patterns, and word-sense categories that systematically differentiate paired corpora.
  • Established the framework as an efficient, rapid tool for document analysis across sociolinguistic research, second-language acquisition, and technical software workflows.

Abstract

This paper describes a method of comparing corpora which uses frequency profiling. The method can be used to discover key words in the corpora which differentiate one corpus from another. Using annotated corpora, it can be applied to discover key grammatical or word-sense categories. This can be used as a quick way in to find the differences between the corpora and is shown to have applications in the study of social differentiation in the use of English vocabulary, profiling of learner English and document analysis in the software engineering process.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Rayson et al. (2000) studied this question.

synapsesocial.com/papers/6a00702b413f0c047f2d7744https://doi.org/10.3115/1117729.1117730
Ask AI
Helpful
Bookmark
Share
View Full Paper