The ever-expanding high dimensional data in both scientific and technology domains has posed fundamental difficulty in analysis, interpretation, and visualization, described collectively as the "complexity crisis." In this paper, we argue for a geometric perspective of how dimensionality reduction is a principled response to these challenges. It opens with the discussion on the pathological effects of high dimensionality such as distance concentration, volume distortion, and exponential sampling complexity, which undermine traditional statistical and distance analysis methods. Based on this premise, this method contributes to providing the ground for the manifold hypothesis, which states that high-dimensional data are usually present on or close to a low-dimensional, smoothly embedded manifold. And it moves the emphasis from the ambient Euclidean structure into the geometry of relationships; this means that the difference that can accurately interpret meaningful data structure lies between Euclidean and geodesic distances. The paper also provides a general comparison of global and local dimensionality reduction methods, highlighting an inherent trade-off in obtaining large-scale variance without losing local neighborhood structure. It shows that reducing the dimension is inherently limited by the information loss — which is formally described in the Johnson–Lindenstrauss lemma and rate-distortion theory as limiting the fidelity of low-dimensional representations. One of the main contributions of this paper is the description and critique of geometric hallucinations—artificial structures (for example those created by dimensionality reduction algorithms) that may misrepresent the true high-dimensional geometry. This is a warning of 'over-interpreting' good looking embeddings without sound validation. More generally, we recast dimensionality reduction in the paper as a geometric and information-theoretic compromise between interpretability and fidelity. It also underscores the importance of methodological selection, critique of visualizations and approaches that focus on distortion, and designing processes that address distortion (to provide more robust and theory-derived applications of high-dimensional data analytics), providing a more trustworthy and theoretically informed approach to the application of high-dimensional data analysis.
Jitin Yadav (Wed,) studied this question.