Fast approximation of Kullback-Leibler distance for dependence trees and hidden Markov models

We present a fast algorithm to approximate the Kullback-Leibler distance (KLD) between two dependence tree models. The algorithm uses the "upward" (or "forward") procedure to compute an upper bound for the KLD. For hidden Markov models, this algorithm is reduced to a simple expression. Numerical experiments show that for a similar accuracy, the proposed algorithm offers a saving of hundreds of times in computational complexity compared to the commonly used Monte Carlo method. This makes the proposed algorithm important for real-time applications, such as image retrieval.