Sliding-window attention beats the retrofit
A new paper tests plain sliding-window attention with attention sinks against models that were post-trained into linear attention to cut inference cost. On long-context benchmarks (Needle-in-a-Haystack, BABILong), the sliding-window version wins by 2 to 10x. No retraining needed either. If you've been budgeting for a linear-attention conversion, check whether you actually need it.