Effect of dimension size and window size on word embedding in classification tasks

Abstract In natural language processing, there are several approaches to transform text into multi-dimensional word vectors, such as TF-IDF (term frequency - inverse document frequency), Word2Vec, GloVe (Global Vectors), which are widely used to this day. The meaning of a word in Word2Vec and GloVe models represents its context. Syntactic or semantic relationships between words are preserved, and the vector distances between individual words correspond to human perception of the relationship between words. Word2Vec and GloVe generate a vector for each word, which can be further utilized. Unlike GPT, ELMo, or BERT, we don't need a model trained on a corpus for further text processing. It's important to know how to set the size of the context window and the dimension size for Word2Vec and GloVe models, as an improper combination of these parameters can lead to low-quality word vectors. In our article, we experimented with these parameters. The results show that it's necessary to choose an appropriate window size based on the embedding method used. In terms of dimension size, according to our results, dimensions smaller than 50 are no longer suitable. On the other hand, with dimensions larger than 150, the results did not significantly improve.