LSA speeds up single-layer prefill 22.7-fold in a ten-million-token context test

Prefrontal AI and ETH Zurich propose Latent Sparse Attention, using clustering and hierarchical indexing to select past tokens. The paper separates single-layer computation from whole-model time to first token; 22.7x refers to a specified GLM attention-layer test.
Editorial correction (November 18, 2026): The 22.7x and 72.4x speedups are prefill results for particular attention layers. Statements about whole-model TTFT and the proportion of keys have been corrected.
Million-token context has gone from a selling point to a standard feature. In April, DeepSeek-V4 made 1M context the default for all official services. In August, Zhipu's GLM-5.3-Flash also supported 1M context, introducing a sparse/linear attention hybrid to the GLM family for the first time. These models share a choice: instead of having each query inspect every historical token, select only a small set of the most relevant ones.
This approach has been repeatedly validated at the 1M scale. But as context grows toward ten million tokens, a previously minor expense starts spiraling. For one global-attention layer of GLM-5.3-Flash, for example, prefill at 10M tokens spends 98% of its time not on attention computation, but on the indexer deciding which tokens to inspect.
The nature of the problem has changed. Sparse attention has already shown that inspecting a small subset can preserve accuracy. What truly constrains ultra-long context is that finding that subset still scales quadratically with context length.
London startup Prefrontal AI and ETH Zurich have now jointly published a paper proposing a new method called Latent Sparse Attention (LSA).

At 10M tokens, it accelerates prefill for a single GLM-5.3-Flash attention layer by 22.7x and for Qwen3.8-27B by 72.4x, while maintaining accuracy close to the original models across a range of tasks.

- Paper title: Latent Sparse Attention: Towards Subquadratic Ultra-Long Context Attention
- Paper: https://arxiv.org/abs/2610.42147
- Code: https://github.com/Prefrontal-AI/latent-sparse-attention
Sparse attention's hidden bill
First, revisit attention's cost structure. In standard full attention, every query computes relevance against all previous keys. With context length T, prefill costs O(T²), and each decoded token costs O(T). Growing context from one million to ten million therefore multiplies prefill attention computation by one hundred.
Sparse attention divides the work: a lightweight indexer first chooses the k most relevant keys for each query, and actual attention is calculated only over those keys. This reduces attention's own cost to O(Tk).
The appendix tests sparsity at 128k tokens. Allowing each Qwen3.8-27B query to access only the 512 most relevant keys retains about 98% of full attention's accuracy on RULER. The proportion represented by 512 keys depends on the visible context range; the original draft's 0.8%, which did not specify a denominator, is not carried forward.
But a paradox remains. To know which 512 keys are most relevant, the indexer must score every key first. DeepSeek's DSA, MiniMax's sparse attention and block-sparse schemes in GLM-5.3-Flash, Qwen and other models reduce this burden through lower precision, fewer dimensions or compressing four keys into one. These only reduce constant factors, however: indexing work still grows linearly with context, and prefill remains quadratic.
Imagine borrowing 4,096 books from a library with ten million volumes. Borrowing is fast; the trouble is inspecting every spine to decide which 4,096 to borrow. As the library grows, spine inspection dominates. The paper estimates that if indexing cost fell to zero at 10M tokens, sparse-attention prefill could become another 58 times faster.

Runtime breakdown for conventional sparse attention at ultra-long context: one GLM-5.3-Flash global-attention layer on a single H200. At 10M tokens, indexing accounts for 98% of total prefill time.
Organize by meaning, not time
LSA's answer is to have the indexer inspect groups of keys instead of individual keys.

D-LSA and N-LSA overview. Top: Within one attention layer, keys are pooled and clustered into buckets; queries score only bucket centers, then inspect selected distant keys plus sink tokens and a sliding window. Bottom: Accuracy and speedups of the two variants.
The question is how to group them. Most existing block-sparse methods group by position, packing neighboring keys into blocks. This resembles boxing library books by arrival date and placing a summary on each box. One important book can be drowned out by summaries of the unrelated books beside it.
LSA groups by content. It projects keys into a learned low-dimensional latent space, then uses hierarchical Ward clustering to place semantically similar keys in the same bucket. Each bucket retains only a centroid and an index of its keys' positions. A query scores bucket centers, selects the most relevant buckets and expands them into individual keys until k keys are available for attention.
Returning to the library analogy, books are shelved by topic rather than boxed by arrival date. To find quantum-error-correction books, go directly to that row of shelves. Books on the topic may have arrived over ten years, but they sit together.
This step offers a constant-factor gain and is called Flat search. The Tree variant changes complexity: buckets are clustered into larger node buckets, continuing upward into a hierarchy. Queries start at the top and retain only the highest-scoring branches at each level through beam search. Like choosing a floor, then an area, then a shelf, each query checks only O(log T) centroids. Indexer prefill drops from quadratic complexity to O(T log T), explaining the paper title's term subquadratic complexity.

D-LSA design. Left, Flat search: Keys are projected, pooled and clustered into a bucket cache; queries score centers and expand them into top-k indices. Right, Tree search: Buckets form a multi-level hierarchy, traversed with beam search.
Two uses: retrofit an existing model or train natively
LSA offers two implementation paths.
The first is D-LSA (Distilled-LSA), for pretrained models. It freezes all original parameters and adds just two small projection matrices per layer, projecting keys and queries into a 64-dimensional indexing space. Distillation then makes LSA scores approximate the original attention distribution. The entire distillation used 352 million tokens; by comparison, DeepSeek DSA's dense indexer warmup alone used 2.1 billion. The paper also presents a fully training-free version using keys' principal-component directions as projections, with accuracy close to the trained version on Qwen3.8-27B.
The second is N-LSA (Native-LSA), which uses LSA natively during pretraining with only standard next-token prediction loss. It directly clusters attention's own keys, requiring no additional indexing projections, teacher model or auxiliary loss, and adds no model parameters.

N-LSA design. The attention layer directly clusters its own keys and can train from scratch using only next-token prediction loss.
On the engineering side, the team wrote GPU kernels for clustering, scoring, selection and expansion targeting H200 and B300, plus a Triton version for any GPU. LSA is integrated into SGLang with tensor, expert and pipeline parallelism, as well as context parallelism during prefill and decoding.
Results at ten million tokens
First, accuracy. The team tested D-LSA on Qwen3.8-27B, whose attention layers use full attention, and GLM-5.3-Flash, which has native sparse attention, extending context to 10M tokens, far beyond both models' official 1M limits. On knowledge, reasoning and long-context tasks including MMLU-Pro, GPQA-Diamond, AIME25, MATH500 and LongBench v2, D-LSA fell within the baselines' 95% confidence intervals without noticeably changing reasoning length.
Across RULER from 4k to 10M tokens, Qwen3.8-27B with D-LSA remained within five points of full attention through 1M, then overtook it at 10M, 63.9 to 59.9. GLM-5.3-Flash with D-LSA stayed within three points of native sparse attention through 1M, trailed by up to eight points between 2M and 5M, and returned within four points at 10M. Note that GLM's baseline sparse attention was trained with the model, while D-LSA was only distilled onto a frozen model, making the comparison favorable to the baseline.
To test ultra-long context in a more realistic setting, the team also built a benchmark called PaperTrail.
It embeds fictional research threads in large collections of real 2026 arXiv papers. One fictional paper introduces a result; passages citing it are inserted into real papers. Weeks later, another fictional paper revises the value and renames the method, while some later papers still cite the old value, resembling real academic literature. The model must find and connect these facts as in a literature review. Samples at 10M tokens contain roughly 400–465 papers. Here, Qwen3.8-27B with D-LSA scored 29.1 at 10M tokens versus 19.8 for full attention.

D-LSA matches full- and sparse-attention baselines on knowledge, reasoning and long-context tasks up to 10M tokens.
Next, speed, the most striking part of the work. For one attention layer's prefill at 10M tokens, D-LSA is 22.7x faster than GLM-5.3-Flash's native sparse attention and 72.4x faster than Qwen3.8-27B's full attention. During decoding, the respective speedups are 4.6x and 42.5x. Speedups grow with context. For GLM-5.3-Flash, prefill is 3.5x faster at 1M tokens but only 2.1x at 512k, because block scanning is inexpensive at shorter contexts. Extrapolating the trend, the Tree variant is projected to exceed a 90x speedup over baseline sparse attention at 100 million tokens.
Single-layer acceleration does not translate proportionally to the whole model. In the listed SGLang deployment settings at 10M tokens, GLM-5.3-Flash's time to first token, TTFT, falls from about 7.5 to 3.4 minutes, roughly 1/2.2 of the original. Qwen3.8-27B falls from about 76 to 8 minutes. The 22.7x and 72.4x figures describe attention-layer prefill, not whole-model speedups.
Index memory is also smaller. At 10M tokens, D-LSA uses 246 MiB per layer, versus 331 MiB for GLM-5.3-Flash's native scheme and 642 MiB for Qwen-style block sparsity.

D-LSA speedups for a single global-attention layer on a logarithmic axis, indexing memory at 10M tokens, and prefill runtime extrapolated to 100M tokens.
N-LSA experiments are much smaller. The team trained a roughly 500-million-parameter hybrid from scratch, with one attention layer in every four of its twenty-four layers and Gated DeltaNet linear-attention layers elsewhere, annealing at 128k context. N-LSA exactly matched full attention's validation loss at 1.085, while PISA, NSA and sliding-window attention at the same sparse budget scored 1.100, 1.117 and 1.154. On 128k RULER, N-LSA scored 32.2 versus full attention's 34.7, with NSA and PISA dropping to 19.2 and 14.6. For training speed, one attention layer's forward-plus-backward pass at 128k was 1.6x faster than full attention implemented with FlashAttention-3.

N-LSA training acceleration. Left: Forward-plus-backward speedup for a single attention layer. Right: Whole-model validation loss over training time during annealing.
Conclusion
For several years, long-context competition has centered on making attention inspect less. From fixed sliding windows to DeepSeek's DSA and various block-sparse schemes, the industry has largely shown that looking at less than 1% of tokens can preserve accuracy. LSA identifies the next bottleneck: once looking is cheap, finding becomes the dominant cost.
Turning attention into retrieval with a hierarchical index is not new. Early work such as Reformer and Routing Transformer also organized keys by content, but few approaches previously validated both accuracy and speed beyond 1M context. As agents take on research, programming and other long-running tasks lasting days, context windows will continue growing. Indexer complexity may ultimately determine the cost curve.
This article was originally published by Synced. Author: Panda.
Related topics
After reading
Join the AINRK community
Read how to join the community and connect with people working in AI.




