Answers are grounded in this paper and cite their sources.
In recent years, the Natural Language Processing (NLP) field has experienced a revolution, where numerous models –based on the Transformer architecture– have emerged to process the ever-growing volume of online text-generated data. This architecture has been the basis for the rise of Large Language Models (LLMs). Enabling their application to many diverse tasks in which they excel with just a fine-tuning process that comes right after a vast pre-training phase. However, their sustainability can often be overlooked, especially regarding computational and environmental costs. Our research aims to compare various BERT derivatives in the context of a dyadic data task while also drawing attention to the growing need for sustainable AI solutions. To this end, we utilize a selection of transformer models in an explainable recommendation setting, modeled as a multi-label classification task originating from a social network context, where users, restaurants, and reviews interact. • A detailed review of the pre-training process for each of the selected BERT variants. • A comparison of diverse transformers within an explainable recommendation setting, where dyadic data is modeled under a multilabel classification task using reviews as explanations. • A comparative result assessment considering environmental impact and performance of the models.
No takes yet. Share an insight, caveat, or question.
Escarda et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: