Deep Multilingual Correlation for Improved Word Embeddings

Word embeddings have been found useful for many NLP tasks, including part-of-speech tagging, named entity recognition, and pars-ing. Adding multilingual context when learn-ing embeddings can improve their quality, for example via canonical correlation analysis (CCA) on embeddings from two languages. In this paper, we extend this idea to learn deep non-linear transformations of word embed-dings of the two languages, using the recently proposed deep canonical correlation analy-sis. The resulting embeddings, when eval-uated on multiple word and bigram similar-ity tasks, consistently improve over monolin-gual embeddings and over embeddings trans-formed with linear CCA. 1