#rope
1 paper
-
inspiration
Position Is a Rotation
Every modern open-weight LLM encodes token position using rotation. Not addition. The shift from additive to multiplicative position encoding is not cosmetic — it changes what the model can generalize to, how far its context can be extended, and what hardware-efficient tricks remain available.