Transforms associated to square integrable group representations. I. General results

Let G be a locally compact group, which need not be unimodular. Let x→U(x) (x∈G) be an irreducible unitary representation of G in a Hilbert space ℋ(U). Assume that U is square integrable, i.e., that there exists in ℋ(U) at least one nonzero vector g such that ∫‖(U(x)g,g)‖2 dx<∞. We give here a reasonably self-contained analysis of the correspondence associating to every vector f∈ℋ(U) the function (U(x)g,f) on G, discussing its isometry, characterization of the range, inversion, and simplest interpolation properties. This correspondence underlies many properties of generalized coherent states.