Finding: AOC MVP wired in-tree (Track B)
Date: 2026-07-15
Status: keep (with open e2e gap)
Track: B — engine only (not text)
What shipped
| Item | Path |
|---|---|
| Hybrid AOC object | new_ai_language/aoc/hybrid_aoc.py |
| CLI harness | new_ai_language/aoc/run_mvp.py |
| Runs | new_ai_language/aoc/runs/*.json |
Implements harvested Tier 0.5/1 without the oracle bloat in harvested_practical_approx.py:
- Sample A(q) cloud (key-perturb + isotropic μ proxy)
- r_Σ = PCA 95%
- Gate → dense if r_Σ ≥ thresh
- r* ≈ 4 · r_Σ; kmeans anchors on A
- q-probes for locate (inducing qs; harvest surrogate)
- key_protos + local_window payload retained
- Optional hybrid Z-blend path; primary fidelity = pure anchors
Sources: maths/05_practical_mvp_from_harvest.md, HARVESTED_COHERENT_SPEC.md, math_kv/attention_output_approx.py.
Kill tests (synthetic, 2026-07-15)
| Run | m | r_Σ | r* | compress | pure mean L2 | gate |
|---|---|---|---|---|---|---|
| clustered seed42 | 256 | 7 | 28 | 3.46× | 0.163 | AOC open |
| clustered seed7 | 512 | 7 | 28 | 4.83× | 0.047 | AOC open |
| isotropic seed1 | 256 | 53 | dense | 1.0× | 0.0 (exact) | dense fallback |
JSON: aoc/runs/latest.json, synth_m512.json, synth_iso.json.
Pass: mem win when r_Σ low; finite A-error; iso high-r_Σ → dense (no false “always constant”).
Fail / open: e2e logits, needle, real Qwen/GPT2 raw_kv in this environment.
Mechanism notes
- q-probe locate is mandatory for low A-error. Key-proto-only locate gave pure mean L2 ~7 on the same clustered synth; probes dropped it to ~0.05–0.16. Matches harvest
ApproxOutputO.probespath. - A0 still strong baseline (mean L2 ~1.5 on clustered synth) but pure anchors beat A0 here once locate is correct.
- Hybrid Z-blend is secondary; pure full-cover anchors are the Thm3 fidelity metric. Hybrid needs engine-side mass estimates for production.
- Backend: NumPy. Local torch blocked (Win Application Control on
c10_cuda.dll); real.ptload deferred.
Non-claims
- Not plugged into HF attention / FlashAttn.
- Not “constant memory for 7B long-ctx.”
- Not Track A. Glyph remains separate.
Next kill tests
| Test | How |
|---|---|
| Real head KV | Export raw_kv/l0_h0.pt → .npz or fix torch; run --raw_kv |
| Layer sweep | Same build on deep-layer heads; expect gate → dense often |
| E2E | Hook one small model; needle + PPL vs dense |
open_questions
- Soft-k over anchors vs hard argmax for locate
- Better Z_f estimator for hybrid without full far K
- Whether auto-A0 switch threshold generalizes beyond synth