Low Separation Rank in Tensor Generalized Linear Models: An Asymptotic Analysis

Several works have investigated tensor decompositions in the context of estimation and prediction problems. Specifically, a recent study explored Tensor Generalized Linear Models (TGLM), in which regression coefficients are assumed to have a tensor structure known as the Low Separation Rank (LSR) decomposition. This structure offers a more favourable bias-variance trade-off than other tensor structures of the same sized core tensor. While the study explored various aspects of the Low Separation Rank Tensor Generalized Linear Model (LSR-TGLM), some aspects of said model and the LSR structure are not well understood. In our work, we discuss some theoretical properties of the LSR-TGLM problem and the set of LSR-structured tensors. We establish an asymptotic consistency result for the LSR-TGLM problem, showing that its MLE estimates converge in probability to the originating LSR-structured coefficient tensor as the sample size increases. This involves proving new properties of LSR-structured tensors, particularly compactness of the set of LSR-structured tensors. We also present conditions for local identifiability of the LSR-structured tensor in an LSR-TGLM problem. Our findings clarify the relationship between the intrinsic degrees of freedom of the LSR-TGLM problem and the rank of the Fisher Information Matrix. Lastly, we provide asymptotic normality results of the MLE estimates, which can be useful for likelihood ratio tests in replication studies.