What type of study is this?

This is a Quantitative Study study.

October 12, 2025Open Access

HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models

View Full Paper

Ask AI

Bookmark

Authors

Geometric reformulation improves positional encodings in transformers, suggesting potential for reliable long-range context.

HoPE enhances the modeling of long-range dependencies, addressing issues found with traditional positional encodings.
Through a geometric approach inspired by Lorentz transformations, it enables more stable attention weights across distances.
Comparative experiments show HoPE outperforms existing methods, particularly in extended sequence benchmarks.
This new encoding method emphasizes monotonic decay of attention, potentially allowing better generalization.

Dai et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark

October 12, 2025Open Access

View Full Paper

Ask AI

Bookmark

Authors

Geometric reformulation improves positional encodings in transformers, suggesting potential for reliable long-range context.

HoPE enhances the modeling of long-range dependencies, addressing issues found with traditional positional encodings.
Through a geometric approach inspired by Lorentz transformations, it enables more stable attention weights across distances.
Comparative experiments show HoPE outperforms existing methods, particularly in extended sequence benchmarks.
This new encoding method emphasizes monotonic decay of attention, potentially allowing better generalization.

Dai et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark