The Alchemy of Language

This presentation offers a visually rich and conceptually rigorous exploration of how meaning in natural language emerges from statistical structure. It begins by situating GloVe within the evolution of word embedding techniques, contrasting it with LSA and Word2Vec to highlight its hybrid strength in combining global co-occurrence statistics with local contextual learning. The presentation then develops the central intuition that raw probabilities are noisy, but their ratios reveal meaningful semantic relationships, leading to the formulation of embeddings where vector differences encode logarithmic co-occurrence patterns. Through clear architectural breakdowns, it explains the end-to-end pipeline—from constructing co-occurrence matrices to optimising embeddings via weighted regression—supported by intuitive metaphors such as inverse-distance weighting and signal extraction. The training dynamics are illustrated through loss convergence behaviour, emphasising how embeddings stabilise into a meaningful geometric space over time. A distinctive contribution of the presentation is its application of the trained GloVe model to a thematic corpus inspired by Jane Austen, demonstrating how semantic clusters emerge at scale, forming what is described as an “Austen Galaxy” in which narrative, social, and thematic relationships become spatially interpretable. The visualisation of embedding spaces using dimensionality reduction techniques further reinforces the idea that language organises itself into coherent geometric structures. The presentation concludes by reflecting on the philosophical implication that meaning is inferred rather than explicitly defined, while also acknowledging limitations such as GloVe's inability to capture subword information or handle unseen vocabulary, thereby motivating the transition toward more advanced models like FastText.