Tracing the historical, theoretical, and mathematical foundations of factor analysis and principal component analysis

This paper explores the foundational distinctions between Factor Analysis (FA) and Principal Component Analysis (PCA), focusing on their historical origins, theoretical motivations, and mathematical structures. FA emerged from early psychological research as a model-driven technique to reveal latent traits underlying observed behaviors, particularly in cognitive assessments. PCA, by contrast, developed from a mathematical need to reduce data dimensionality and summarize variance without presupposing hidden constructs. This study begins by tracing the intellectual lineage of both methods, showing how FA evolved from psychological theory and PCA from statistical geometry. A central contribution of the manuscript is a side-by-side explanation of their mathematical formulations: FA models observed variables as linear combinations of latent factors and error, whereas PCA transforms data into uncorrelated principal components by maximizing total variance explained. Practical examples are discussed to illustrate how the two methods serve different research goals: FA for uncovering underlying psychological or social constructs, and PCA for simplifying complex datasets in exploratory analysis. This work clarifies frequent misconceptions between the two and emphasizes their respective roles in multivariate analysis, offering guidance for researchers selecting the appropriate method based on analytical purpose and data structure.