Noken Rademacher Descent Jan 20, 2026

Key Points

• The aim is to refine Noken attention through better alignment of query and key similarities using Rademacher descent.
• Applied Rademacher descent to optimize hyperparameters of the implicit model.
• Evaluated changes in query-key similarity alignment post-optimization.
• Compared performance metrics before and after refinement.
• Achieved improved alignment of query-key similarities after Rademacher descent.
• Enhanced overall attention performance in tasks using Noken attention.
• Results indicate a significant reduction in alignment errors.