On Achieving and Evaluating Language-Independence in NLP

Language independence is commonly presented as one of the advantages of modern, machine-learning approaches to NLP, and it is an important type of scalability. In this position paper, I critically review the widespread approaches to achieving and evaluating language independence in the field of computational linguistics and argue that, on the one hand, we are not truly evaluating language independence with any systematicity and on the other hand, that truly language-independent technology requires more linguistic sophistication than is the norm.