The aim is to develop Roken, a deep learning model that incorporates state tracking for better language processing.
Designed a recurrent Noken model focusing on attention mechanisms.
Implemented dynamic implicit learning for both next-token and sequence-to-sequence tasks.
Demonstrates improved attention integration in language tasks.
Shows enhanced performance in state tracking during learning processes.
Abstract
Roken - recurrent Noken, a dynamic implicit deep learning model of attention, designed to integrate state tracking into large language model next-token and sequence-to-sequence learning.