HFG LASSO: A method for models with hierarchical categorical predictors

Hierarchical structures of categorical predictors are characterized by the levels of some predictors being nested within levels of others, forming a hierarchical tree. These structures are commonly encountered in disciplines like biology, business and public health. Such sets of categorical predictors are referred to as Hierarchical Categorical Predictors (HCPs). Given HCPs often have high cardinality at the most granular layer, their inclusion within linear regression models can increase model complexity substantially. A new penalty term for linear regression models, called the “Hierarchical Fused Group LASSO” (HFG LASSO) penalty, is introduced. This penalty fuses together an HCP’s coefficients based on the groups defined by its underlying hierarchical tree, both promoting model sparsity and fusing coefficients according to their hierarchical groups. A fitting algorithm and its implementation are presented for linear regression modelling with an HFG LASSO penalty. The performance of HFG LASSO with respect to prediction, coefficient estimation and group recovery is assessed on both simulated data and an applied dataset. A novel group recovery measure is also proposed that accounts for the hierarchy among groups. The results of HFG LASSO in comparison to other candidate methods demonstrate that HFG LASSO excels at group recovery and coefficient estimation, while maintaining comparable predictive performance. Although HFG LASSO is developed for HCPs, the methodology is generalizable to other predictors with hierarchical structures.