PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
March 8, 2010PLoS ONE162 citationsOpen Access

Random Texts Do Not Exhibit the Real Zipf's Law-Like Rank Distribution

View Full Paper
RFRamon Ferrer‐i‐CanchoUniversitat Politècnica de CatalunyaBEBrita ElvevågUniversity Hospital of North Norway

Key Points

Key points are not available for this paper at this time.

Abstract

BACKGROUND: Zipf's law states that the relationship between the frequency of a word in a text and its rank (the most frequent word has rank , the 2nd most frequent word has rank ,...) is approximately linear when plotted on a double logarithmic scale. It has been argued that the law is not a relevant or useful property of language because simple random texts - constructed by concatenating random characters including blanks behaving as word delimiters - exhibit a Zipf's law-like word rank distribution. METHODOLOGY/PRINCIPAL FINDINGS: In this article, we examine the flaws of such putative good fits of random texts. We demonstrate - by means of three different statistical tests - that ranks derived from random texts and ranks derived from real texts are statistically inconsistent with the parameters employed to argue for such a good fit, even when the parameters are inferred from the target real text. Our findings are valid for both the simplest random texts composed of equally likely characters as well as more elaborate and realistic versions where character probabilities are borrowed from a real text. CONCLUSIONS/SIGNIFICANCE: The good fit of random texts to real Zipf's law-like rank distributions has not yet been established. Therefore, we suggest that Zipf's law might in fact be a fundamental law in natural languages.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Ferrer‐i‐Cancho et al. (2010) studied this question.

synapsesocial.com/papers/6a108c16d13714ec96000540https://doi.org/10.1371/journal.pone.0009411
Ask AI
Helpful
Bookmark
Share
View Full Paper

Also Consider

Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context:

  1. 1Text Compression1990 · 987 citations
  2. 2Quantitative Linguistics.1966 · 129 citations
  3. 3Proceedings of the National Academy of Sciences, USA1996 · 5,272 citations
  4. 4Some Effects of Intermittent Silence1957 · 333 citations
  5. 5The Psycho-Biology of Language1936 · 738 citations