New AI Language docs · static / Cloudflare Pages

Track A — Glyph I/O codec (still text)

Code: repo root glyph_codec.py

Role: denser text working memory for agents — not a neural non-text representation.


1. Idea

Split meaning into:

  1. Palette — long phrases once (a0=postgres users need tx)
  2. Frame — short codes on role rows (F|a0|a1)

Like image palette + pixel indices, but serialized as ASCII for LLMs.


2. Alphabet (v1)

Roles: G F D O N X R S L P

(goal, facts, decisions, open, next, constraints, refs, scores, laws, packet)

Ops: + - ~ > = ? ! @ ^ * / & 0 1

Empty: .

Cells: palette codes a0, d1, l3 or score atoms S0.71


3. Run


python glyph_codec.py alphabet

python glyph_codec.py demo

python glyph_codec.py encode memory/packets.jsonl memory/hybrids.jsonl --out memory/glyph_runs/latest_mix

Outputs: palette.txt, frame.glf, stream.glf, metrics.json, sample_before_after.txt


4. Practical meaning

WithoutWith
Resend full chat every turnResend palette (cacheable) + small frame
Repeat long entity namesReuse a1
Absorb near-duplicatesJaccard shingle absorb

Measured on project traces (approx token proxy): large reduction of frame stream vs raw event text. Still BPE text.


5. Prompt pattern


SYSTEM (cached):

  You read GLYPH frames. Legend:

  a0=...

  a1=...

  Answer with Δ rows or short user prose; do not rewrite full history.



USER:

  #frame

  F|a0|a1

  N|a2

  X|a3

  ---

  <new user message>


6. Kill test

Same as Track A global: ≥20–25% tokens_in multi-turn; quality hold; measure with real tokenizer when shipping.