New AI Language docs · static / Cloudflare Pages

A0 (winner map) and hybrid exact+manifold

Parents: 17 Thm1; 18 hybrid unification; HARVESTED_COHERENT_SPEC; filter empirics.


A0 — piecewise constant attention

For tope τ(q) with argmax index i\*(q):

\[

A_0(q) := v_{i^*(q)}

\]

Error bound (sketch): controlled by score gap Δ(q) between top and second:

\[

\|A(q)-A_0(q)\| \lesssim (m-1)\, e^{-\Delta(q)} \cdot \mathrm{diam}(\{v_i\})

\]

When Δ large (peaky attention), A0 is excellent. Empirics on some heads: near-zero error.

Product mode: if top-2 gap high → use A0 (one value) instead of soft anchor.


Hybrid construction (data-forced)


A_hat(q) = A_exact(q; local_window) + A_cover(q; anchors for far past)

PieceStoresRole
Local window Wexact K,VRecent tokens; quality critical
Far coverr\* anchors + protosOld context; size ~ r\* not m

Why hybrid: pure cover from prefix only → non-trivial error mid layers / longer seq (brutal analysis + 18).


Online graduate rule


append token → local

if |local| > W:

  oldest chunk → update anchors (recluster / barycentric)

  drop raw K,V of that chunk (if head in AOC mode)


Link to Track A

Hybrid window ≈ “recent turns fuller; old turns codes/anchors only” — same time split, different medium (text codes vs float anchors).