A0 (winner map) and hybrid exact+manifold
Parents: 17 Thm1; 18 hybrid unification; HARVESTED_COHERENT_SPEC; filter empirics.
A0 — piecewise constant attention
For tope τ(q) with argmax index i\*(q):
\[
A_0(q) := v_{i^*(q)}
\]
Error bound (sketch): controlled by score gap Δ(q) between top and second:
\[
\|A(q)-A_0(q)\| \lesssim (m-1)\, e^{-\Delta(q)} \cdot \mathrm{diam}(\{v_i\})
\]
When Δ large (peaky attention), A0 is excellent. Empirics on some heads: near-zero error.
Product mode: if top-2 gap high → use A0 (one value) instead of soft anchor.
Hybrid construction (data-forced)
A_hat(q) = A_exact(q; local_window) + A_cover(q; anchors for far past)
| Piece | Stores | Role |
|---|---|---|
| Local window W | exact K,V | Recent tokens; quality critical |
| Far cover | r\* anchors + protos | Old context; size ~ r\* not m |
Why hybrid: pure cover from prefix only → non-trivial error mid layers / longer seq (brutal analysis + 18).
Online graduate rule
append token → local
if |local| > W:
oldest chunk → update anchors (recluster / barycentric)
drop raw K,V of that chunk (if head in AOC mode)
Link to Track A
Hybrid window ≈ “recent turns fuller; old turns codes/anchors only” — same time split, different medium (text codes vs float anchors).