Roken - a recurrent implicit dynamic LLM

Key Points

• The aim is to develop Roken, a deep learning model that incorporates state tracking for better language processing.
• Designed a recurrent Noken model focusing on attention mechanisms.
• Implemented dynamic implicit learning for both next-token and sequence-to-sequence tasks.
• Demonstrates improved attention integration in language tasks.
• Shows enhanced performance in state tracking during learning processes.