Verified record · from the answer index

long-context

Extend context windows of transformer models using RoPE, YaRN, ALiBi, and position interpolation techniques. Use when processing long documents (32k-128k+ tokens), extending pre-trained models beyond original context limits, or implementing

skill:orchestra-research/ai-research-skills/long-context · published by orchestra-research · source repository