Subquadratic's central claim is that you can attend to 12 million tokens while comparing almost none of them. The mechanism it credits, SSA, is also the part of SubQ the company has disclosed the least about, which makes understanding how it picks positions the first thing worth scrutinizing.
How SSA selects which positions to process
SSA (Subquadratic Sparse Attention) is a content-dependent attention mechanism: for each query token it selects a small subset of positions deemed relevant, then computes exact attention only over that subset, with no fixed stride and no sliding window . Instead of comparing every token to every other token, the O(n²) cost of dense Transformer attention, SubQ claims roughly linear O(n) scaling in compute and memory per forward pass, because the count of compared pairs grows with the sequence rather than with its square .
The headline number quantifies how aggressive the pruning is. At 12M tokens, SubQ's model card states the model attends to only 0.13% of token pairs, close to a 1,000× reduction versus dense attention, and notably at a length six times the 2M maximum it was trained on . The donor model launched as SubQ 1M-Preview on May 5, 2026; the 1.1 Small model card describing this behavior was published June 16, 2026 .
What separates SSA from prior sparse approaches is where the sparsity comes from:
- Fixed-pattern sparse attention (Longformer, BigBird) uses predetermined windows and global tokens, and the pattern is the same regardless of content .
- State-space models (Mamba, RWKV) replace attention with a recurrent/compressed state, trading exactness for a fixed-size memory .
- SSA claims content-aware routing (the selected positions change per query) while preserving exact attention over whatever subset it picks, rather than approximating across the full sequence .
That distinction is also where the unverified question lives. Exact attention over a chosen subset is only as good as the choice. If the router reliably surfaces the right positions across diverse task types (multi-hop tracing, aggregation, code navigation, long-running agent state), then linear-cost frontier reasoning is plausible. If the gains concentrate on retrieval-shaped tasks like needle-in-a-haystack, the architecture could look excellent on benchmarks designed around locating a known span and weaker where relevance is diffuse. VentureBeat framed the researcher consensus as exactly this: demands for independent proof that the efficiency holds without sacrificing understanding . The sections that follow test that question tier by tier.
The donor SubQ won't name
SubQ 1.1 Small was not trained from scratch. According to the model card, the team converted an existing open-weight frontier model by replacing its dense attention with SSA, then applied staged context extension and continued pretraining . That detail reframes the whole evaluation: the architecture being tested is a graft onto a pre-existing model, not a clean-room design. The donor began with a 262K-token context window, and Subquadratic used YaRN positional scaling to stretch it through four stages before any of the multi-million-token claims apply .
The scaling schedule is specific. YaRN ran through 262K → 512K → 1M → 2M, with most continued-pretraining tokens spent at the 1M stage and a final extension/training stage at 2M . Training data emphasized naturally long artifacts (books, long documents, repository-scale code) and post-training mixed synthetic retrieval, long-context reasoning, coding, and general instruction following . The company reports more than 100 long-context experiments across six model generations to arrive at this recipe . Note what the numbers describe: 2M is the maximum length the model was ever trained on, yet the headline retrieval claims extend to 12M, six times beyond training.
What the materials do not say is which model was converted. The donor identity is undisclosed, and independent researchers have filled the gap with speculation. One suggested SubQ could be a sparse-attention finetune of an open model such as Kimi or DeepSeek , a guess consistent with a 262K starting context and with the model card's own donor-conversion disclosure. Subquadratic has not confirmed any candidate.
The omission matters more than a typical non-disclosure, because it makes the central question unanswerable from public data. Without knowing the donor, there is no way to separate SSA's architectural contribution from the donor model's pre-existing long-context ability or from the continued-pretraining recipe layered on top. The model card concedes this directly: the team has not run controlled ablations pairing the same continued-pretraining recipe with architecturally distinct backbones, so it cannot rule out architecture-specific effects . VentureBeat captured the same reservation, noting the historical pattern where subquadratic architectures match dense attention at small and medium scale but underperform at frontier scale, with researchers demanding independent proof of the efficiency gains . Until the donor is named or a backbone-controlled ablation appears, every benchmark that follows is measuring the graft and the host together, with no way to credit either alone.
RULER at 128K, NIAH at 12M: the tier-by-tier summary
SubQ 1.1 Small's retrieval scores are its strongest evidence, and they hold up across more than a single-needle test. The model card reports 99.12% on RULER at a 128K context, and clean needle-in-a-haystack exact match of 100% at 1M and 2M tokens, slipping to 98% at 6M and 12M . RULER matters more than a bare needle test because it spans 13 tasks (NIAH variants plus multi-hop tracing and aggregation) and was built precisely because simple needle retrieval can overstate long-context understanding . A high RULER number signals the model is tracking and combining facts, not just locating one.
Appen, commissioned by Subquadratic, ran an independent pass on RULER's niah_single_1 task and reproduced the same shape: 100% exact match at 1M and 2M, 98% at 6M and 12M, with 50 samples per tier, temperature 0, and zero execution errors . The agreement between the model card and the commissioned brief is reassuring on reproducibility, but both trace back to work Subquadratic paid for, not an open public leaderboard.
| Context tier | NIAH exact match | Relative to 2M training max | Source |
|---|---|---|---|
| 128K (RULER, 13 tasks) | 99.12% | Within training distribution | Model card |
| 1M | 100% | Within training distribution | Card + Appen [1][5] |
| 2M | 100% | At training maximum | Card + Appen [1][5] |
| 6M | 98% | 3× beyond training max | Card + Appen [1][5] |
| 12M | 98% | 6× beyond training max | Card + Appen [1][5] |
The distinction that deserves the most weight is interpolation versus extrapolation. The model's longest continued-pretraining stage topped out at 2M tokens, so 12M is 6× past anything it saw in training . The 6M and 12M scores are extrapolation beyond the training distribution, not interpolation inside it. That the curve only drops two points there is the genuinely interesting result, and also the one most in need of scrutiny. At 12M the model attends to roughly 0.13% of token pairs, a near 1,000× reduction , so the question is whether sparse routing still finds the right needle when the haystack is six times larger than training ever showed it.
The 50-sample-per-tier design is the practical caveat. Fifty held-out UUID lookups is a narrow slice, enough to detect catastrophic failure but not enough to estimate a robust pass rate at the highest lengths or across varied task types. A 98% on 50 samples is two misses; the confidence interval is wide. Treat the 6M and 12M figures as provisional until they are reproduced on larger sample sizes and the full 13-task RULER suite, not just niah_single_1.
What Appen's brief captures and what it leaves out
Appen's data is the closest thing to third-party validation SubQ has, but it is commissioned, not independent: Appen was engaged by Subquadratic to benchmark the model, so its briefs verify the company's setup rather than audit it adversarially . The headline number is real and reproducible under the stated conditions, but it is also narrower than the marketing framing suggests. Read the briefs for what they actually measured: a single attention layer, not a production pipeline.
Quick Answer: Appen, paid by Subquadratic, clocked SSA at 381 ms versus FlashAttention-2's 21.4 seconds on a B200 at 1M tokens, a 56.2× speedup . But that is one isolated attention-layer kernel measurement, not full-prefill, batched, end-to-end production latency.
Two separate briefs exist, on different hardware. One covers an H100 (CUDA 12.x); the second covers a B200 running CUDA 13.0, PyTorch 2.11.0, and bfloat16, with five timed runs after three warmups . On the B200, the 1M-token pass took 381 ms for SSA against 21.4 seconds for FlashAttention-2, the 56.2× wall-clock figure that anchors SubQ's efficiency pitch . The FLOP reduction scales with sequence length rather than staying flat: 7.9× at 128K rising to 62.8× at 1M, where SSA computes 144.9 TFLOP against dense attention's 9,095.2 TFLOP . That gradient matters: the advantage is small at short contexts and only becomes dramatic at the multi-million-token lengths SubQ is built for.
| Context length | FLOP reduction (SSA vs FA2) | Note |
|---|---|---|
| 128K | 7.9× | Modest at short context |
| 1M | 62.8× | 144.9 vs 9,095.2 TFLOP |
| 1M (wall-clock) | 56.2× | 381 ms vs 21.4 s, B200 |
Here is what the briefs do not capture. The comparison is a single attention layer or isolated kernel measurement: not a full prefill across all layers, not batched concurrent requests, and not end-to-end latency on realistic mixed-length inputs . In a real serving stack, attention is one cost among many: MLP blocks, KV-cache movement, memory bandwidth, and batch scheduling all dilute a layer-level speedup. The model card's own H100 figure (966 ms vs 54,164 ms, a 56× single-layer speedup) lines up directionally with Appen's B200 result, which is reassuring for the scaling claim but says nothing about throughput under load .
The reporting itself flags the gap: exact speedups "vary by hardware, implementation scope, and whether the comparison is a full prefill or isolated layer/kernel measurement" . VentureBeat captured the broader reception bluntly: independent researchers are "demanding independent proof of the efficiency gains" rather than accepting vendor-and-commissioned numbers at face value . Treat 56× as an upper-bound kernel claim, a clean signal that SSA's asymptotics hold in practice, not a production throughput guarantee. The number a developer cares about, tokens per second per dollar on a loaded server, is not in either brief.
AutomationBench Finance: 13% vs. Opus 4.8 and GPT-5.5
On agentic and reasoning tasks, SubQ 1.1 Small posts mixed but legible numbers: 85.4% on GPQA Diamond pass@1, 89.7% on LiveCodeBench v6 pass@4, and 13% on AutomationBench Finance . The first two are standard capability checks; the third is the one Subquadratic leans on to argue the model holds up beyond retrieval. AutomationBench Finance measures cross-application workflow orchestration: chaining REST API calls across tools and grading the programmatic end state rather than the chat transcript . That 13% is the number to scrutinize, because it is reported against named competitors that never submitted under the same conditions.
The LiveCodeBench result is the most defensible of the three. The 89.7% pass@4 covers 1,055 problems and 4,220 completions, and the benchmark is contamination-resistant by design: its authors source problems continuously from LeetCode, AtCoder, and Codeforces and date-filter to exclude items predating a model's training cutoff . A coding score that survives contamination controls is a stronger signal than a needle test.
AutomationBench Finance is where the framing gets aggressive. Subquadratic positions its 13% as close to frontier agentic models and well ahead of a mid-tier one:
| Model | AutomationBench Finance |
|---|---|
| GPT-5.5 | 18% |
| Opus 4.8 | 16% |
| SubQ 1.1 Small | 13% |
| Sonnet 4.6 | 8% |
On this single task, 13% does put SubQ closer to Opus 4.8 than to Sonnet 4.6 . The problem is provenance: every competitor figure in that table comes from Subquadratic and Appen material, not a public leaderboard where Opus, GPT-5.5, and Sonnet were run by their own vendors under disclosed conditions . When a vendor benchmarks its rivals on a harness it controls, the relative ordering is a claim, not a result.
Two caveats follow. First, AutomationBench Finance is one workflow domain; a 13% on finance orchestration says nothing about diverse multi-application tasks, where the gap between SubQ and the actual frontier leaders (o3, GPT-5.5) on varied workflows is simply uncharted. Second, all three scores sit in absolute terms that are low: an 18% ceiling means even the strongest model fails most finance-orchestration runs, so a 5-point spread separates models that all struggle. For developers, the takeaway is narrower than the table suggests: SubQ is competitive on a hard agentic slice when graded by its own commissioned harness, and that is a hypothesis awaiting independent leaderboard confirmation, not a settled ranking.
What SubQ is keeping private
As of June 2026, SubQ remains a private-preview, design-partner-only system, and the materials behind its headline numbers withhold most of what would let an outside engineer reproduce or stress-test them. The public site and the SubQ 1.1 Small model card do not disclose the parameter count, the donor model's identity, the full SSA mechanism, the total continued-pretraining token budget, or the exact dataset composition . The model card explicitly places the SSA mechanism outside its scope, so the content-aware routing that the entire efficiency story depends on is described in principle but not in detail .
The operational gaps matter as much as the architectural ones. There are no published safety evaluations, no pricing, no rate limits, no downloadable weights, and no general-availability date . For a developer, that means you cannot yet budget a deployment, audit refusal behavior, or self-host: the OpenAI-compatible endpoint lowers switching cost in theory, but only design partners can exercise it in practice.
Two disclosure gaps are worth tracking specifically. First, Subquadratic announced a LayerLens/Stratix evaluation partnership on May 14, 2026 to produce standardized public benchmarks, yet no results from it were published as of the June 16, 2026 model card release . The independent third party that could de-risk the vendor-reported numbers has not yet reported. Second, the model card concedes a methodological limit that bears directly on the SSA thesis:
"The team has not run controlled ablations pairing the same continued-pretraining recipe with architecturally distinct backbones, so it cannot rule out architecture-specific effects," SubQ 1.1 Small model card, Subquadratic Inc. (source: subq.ai).
That is the key methodological gap. Without an ablation that swaps SSA onto a dense-attention backbone under an identical recipe, the reported gains cannot be cleanly attributed to the sparse-attention design rather than the donor model or the continued-pretraining data. It is consistent with researchers' broader skepticism: VentureBeat framed the consensus as a demand for independent proof of the efficiency claims . The practical signal: treat SubQ as an interesting architecture hypothesis with credentialed backers, not as a benchmarked product you can adopt today.
What external audits need to address
Independent verification of SubQ should target four specific gaps that the company- and vendor-reported numbers leave open. The published results lean heavily on isolated kernel measurements and retrieval-style tasks, so an external audit needs full-prefill timings, routing quality across task types, a controlled architecture ablation, and public leaderboard submissions before the headline claims can be treated as product behavior rather than directional evidence.
- Full prefill at production batch sizes. The standout efficiency figures, including a 56× speedup over FlashAttention-2 at 1M tokens on an H100 (966 ms vs 54,164 ms) in the model card and Appen's 56.2× B200 result (381 ms vs 21.4 s) , are single-attention-layer tests. Audits should report end-to-end prefill and decode latency at realistic batch sizes across representative workloads, not one kernel against FA2.
- Routing quality beyond retrieval. SubQ's strongest scores are RULER at 128K (99.12%) and Needle-in-a-Haystack at 1M-2M (100%) . The open question is whether content-dependent SSA selection finds the right positions for multi-hop tracing, aggregation, and instruction following (not just single-needle lookup) at the 6M-12M tiers where retrieval already slips to 98%.
- A controlled ablation. The model card itself concedes the team has not paired its continued-pretraining recipe with an architecturally distinct backbone . Running the same recipe on a dense-attention backbone versus the SSA backbone is the only way to isolate what the architecture contributes versus the donor model and training data.
- Public leaderboard submissions. The competitor comparisons, 85.4% GPQA Diamond, 89.7% LiveCodeBench v6 pass@4, and 13% on AutomationBench Finance against Opus 4.8 (16%) and GPT-5.5 (18%), appear in Subquadratic/Appen material, not an independent leaderboard . Submitting under standard conditions would let the community validate them.
The May 14, 2026 LayerLens/Stratix partnership for standardized public benchmarks is the right venue for several of these, but no results were visible in the static pages reviewed . Until those land, the bottom line holds: SubQ is a credibly backed architecture hypothesis with verification debt, and VentureBeat's framing, that researchers should demand independent proof , is the correct posture. Watch the LayerLens results; that is where the 56× becomes auditable or not.
Frequently asked questions
What is SSA (Subquadratic Sparse Attention)?
SSA is the content-dependent sparse-attention mechanism at the core of SubQ. For each query, it selects a small subset of token positions deemed relevant and computes exact attention only over that subset, which Subquadratic says scales roughly linearly (O(n)) in compute and memory instead of quadratically (O(n²)) like dense Transformer attention . The company distinguishes it from fixed-pattern sparse attention (Longformer, BigBird) and state-space models (Mamba, RWKV) by emphasizing content-aware routing while keeping attention exact over the selected positions . The full mechanism is not publicly described; the model card explicitly places it out of scope .
Why does the unnamed donor model matter when evaluating SubQ?
It matters because SubQ 1.1 Small was not trained from scratch. The model card states the team converted an existing open-weight frontier model by replacing dense attention with SSA, then applied staged context extension and continued pretraining from a 262K-token base out to 2M . Without knowing the donor's identity, you cannot attribute the benchmark results to SSA itself versus the donor's pre-existing capabilities and training recipe. The team also acknowledges it has not run controlled ablations pairing the same recipe with architecturally distinct backbones, so it cannot rule out architecture-specific effects . Independent researchers flagged this as a critical disclosure gap, with one suggesting the base could be a model such as Kimi or DeepSeek .
Is 99% NIAH at 12M tokens a meaningful result?
It is a positive signal, but a narrow one. Subquadratic reports Needle-in-a-Haystack retrieval of 100% at 1M and 2M tokens, falling to 98% at 6M and 12M on held-out 50-sample UUID tests . The key caveat is extrapolation: 12M is six times the 2M maximum training length, and each tier used only 50 samples at temperature 0, a thin statistical base . The RULER score of 99.12% at 128K is a stronger multi-task indicator, since RULER spans 13 tasks including multi-hop tracing and aggregation rather than simple needle retrieval , but it too was measured by Appen under contract .
Can I use SubQ in production today?
No. As of June 2026, SubQ 1.1 Small is in private preview and a design-partner phase only, with the company saying it is deploying with select partners ahead of a broader 2M-to-12M lineup later in 2026 . OpenAI-compatible endpoints exist in principle and lower switching cost , but the public materials do not disclose a public API, documented pricing, rate limits, downloadable weights, or a general-availability date . Treat it as evaluation-stage technology, not a deployable dependency.
How reliable is the 56× speedup claim?
Treat it as an upper-bound kernel claim. The 56× figure comes from an isolated single-attention-layer measurement at 1M tokens: the model card reports 966 ms for SSA versus 54,164 ms for FlashAttention-2 on an H100 , and Appen's separate B200 brief reports 381 ms versus 21.4 seconds, a 56.2× wall-clock speedup . Both were run by Appen, commissioned by Subquadratic, and neither covers full prefill, batching, or end-to-end production latency . The scaling direction is plausible, but exact gains vary by hardware, implementation scope, and whether the test is an isolated layer or a full pass; independent audits via the announced LayerLens partnership are where it becomes verifiable .
