Enhancing the LexVec Distributed Word Representation Model Using Positional Contexts and External Memory

In this paper we take a state-of-the-art model for distributed word that explicitly factorizes the positive pointwise mutual (PPMI) matrix using window sampling and negative sampling and two of its shortcomings. We improve syntactic performance by using contexts, and solve the need to store the PPMI matrix in memory by on aggregate data in external memory. The effectiveness of both is shown using word similarity and analogy tasks.