Answers are grounded in this paper and cite their sources.
The article is devoted to the urgent problem of representing and classifying long text documents usingtransformers. Transformers-based text representation methods cannot effectively process long sequencesdue to their self-attention process, which scales quadratically with the sequence length. This limitationleads to high computational complexity and the inability to apply such models for processing longdocuments. To eliminate this drawback, the article developed an algorithm based on the SBERT transformer,which allows building a vector representation of long text documents. The key idea of the algorithmis the application of two different procedures for creating a vector representation: the first is basedon text segmentation and averaging of segment vectors, and the second is based on concatenation of segmentvectors. This combination of procedures allows preserving important information from long documents.To verify the effectiveness of the algorithm, a computational experiment was conducted on a groupof classifiers built on the basis of the proposed algorithm and a group of well-known text vectorizationmethods, such as TF-IDF, LSA, and BoWC. The results of the computational experiment showed thattransformer-based classifiers generally achieve better classification accuracy results compared to classicalmethods. However, this advantage is achieved at the cost of higher computational complexity and,accordingly, longer training and application times for such models. On the other hand, classical textvectorization methods, such as TF-IDF, LSA, and BoWC, demonstrated higher speed, making them morepreferable in cases where pre-encoding is not allowed and real-time operation is required. The proposedalgorithm has proven its high efficiency and led and led to an increase in the classification accuracy of theBBC dataset by 0.5% according to the F1 criterion.
No takes yet. Share an insight, caveat, or question.
Ali Mansour (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: