In our paper, we explore the limitations of byte pair encoding (BPE) in Arabic language models, revealing significant token inflation and a failure to preserve morphological relationships. We introduce root pattern tensor subspaces (RPTS) as a more efficient and linguistically informed alternative, achieving a remarkable reduction in embedding parameters while maintaining structural integrity.
Salhi arselen (Sat,) studied this question.