Sample-Based Classification Procedures Derived from Density Estimators

Parametric or nonparametric density estimators from a mixture of K distributions can be used to estimate the non-error rate of an arbitrary classification rule—and to construct a rule which maximizes estimated probability of correct classification. (For two multivariate normal distributions with common covariance matrix, this general criterion yields the usual linear discriminant.) Such a sample-based rule is asymptotically optimal under very general conditions. Often its “apparent” non-error rate exceeds its true rate and is even optimistically biased as an estimator of the (unknown) optimal rule's non-error rate; but the apparent rate converges to this optimum.