Answers are grounded in this paper and cite their sources.
The generalized linear model (GLM) is widely used in econometrics and statistics. In a GLM, a location parameter of the dependent variable—typically the expected value or mean—is linked to a linear combination of independent variables through a link function. The choice of link function is determined by the distribution of the error term. In contrast, activation functions are used in the convolutional layers of deep learning (DL) models. Common activation functions include the indicator (step) function, sigmoid, rectified linear unit (ReLU), and leaky ReLU. Although the universal approximation theorem shows that DL models using these activation functions can approximate a wide class of functions, major challenges remain, including overfitting, local minima, and substantial computational requirements. It is therefore essential to select an appropriate activation function. Link functions in GLMs and activation functions in DL are closely related and often coincide. In this note, we examine the similarities between these functions and propose an activation function derived from the truncated mean. We focus on the theoretical and methodological aspects rather than practical applications. The relationship between link functions and activation functions remains conjectural and warrants further research.
No takes yet. Share an insight, caveat, or question.
Kazumitsu Nawata (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: