Inverted indexing for cross-lingual NLP

We present a novel, count-based approach to obtaining inter-lingual word represen-tations based on inverted indexing of Wikipedia. We present experiments ap-plying these representations to 17 datasets in document classification, POS tagging, dependency parsing, and word alignment. Our approach has the advantage that it is simple, computationally efficient and almost parameter-free, and, more im-portantly, it enables multi-source cross-lingual learning. In 14/17 cases, we im-prove over using state-of-the-art bilingual embeddings. 1