Abstract Bender, Coley, Robbins, and Rumsey posed the problem of counting the number of subspaces which have a given profile with respect to a linear endomorphism defined on a finite vector space. We settle this problem in full generality by giving an explicit counting formula involving symmetric functions. This formula can be expressed compactly in terms of a Hall scalar product involving dual q -Whittaker functions and another symmetric function that is determined by conjugacy class invariants of the endomorphism. As corollaries, we obtain new combinatorial interpretations for the coefficients in the q -Whittaker expansions of several symmetric functions. These include the power sum, complete homogeneous, products of modified Hall–Littlewood functions, and certain products of q -Whittaker functions. These results are used to derive a formula for the number of anti-invariant subspaces (as defined by Barría and Halmos) with respect to an arbitrary operator. We also give an application to an open problem in Krylov subspace theory.
Samrith Ram (Thu,) studied this question.