Induced Representations of Locally Compact Groups II. The Frobenius Reciprocity Theorem

In the study of the relationship between the representation theory of a group and those of its various subgroups an important role is played by Frobenius's notion of induced representation. To every representation L of the subgroup G of the finite group 65 there is assigned a well defined representation UL of ( called the representation of M induced by L. In I of this series 10 the author began a systematic study of a generalization of this notion in which (5 is a separable locally compact topological group and the spaces of L and UL are possibly infinite dimensional Hilbert spaces. In particular 10 contains a generalization of the Frobenius reciprocity theorem; that is the theorem asserting that UL contains the irreducible representation M of (D just as many times as the restriction of M to G contains the irreducible representation L of G. The generalization contained in 10 is unsatisfactory in that it deals only with the discrete finite dimensional irreducible components of the representations concerned and becomes vacuous when these representations decompose in a continuous fashion or have no finite dimensional irreducible components. A more satisfactory generalization has been obtained by Mautner 13, 14. It deals in a consequent fashion with continuously decomposable representations of 5 but is restricted by the requirement that G be compact. This means in particular that only discretely decomposable representations of G need be dealt with. The principal result of the present article is a generalization of the Frobenius reciprocity theorem which deals effectively with continuously decomposable representations of both M and G. Since in compensation for the hypothesis that G be compact, we require that both G and 5 have regular representations which are of type I, our theorem does not quite include that of Mautner. However we show in addition that whenever the regular representation of G is not only of type I but also discretely decomposable then the requirement that the regular representation of (M be of type I can be eliminated. Thus our methods also yield a result which includes that of Mautner. In neither of our results is it necessary to assume that the groups concerned are unimodular. A noteworthy feature of our principal theorem is that it includes a reciprocity not only for the multiplicities but for the measures involved in the continuous decompositions as well. The basic idea in our approach is a new proof of the Frobenius reciprocity theorem in the finite case which has the advantage of generalizing significantly