Backoff slot selection
Slot selection (Step 0) added B=4 extra K/V slots to attention for each query. The slots
hold the suffix-backoff trie nodes (the node's path minus leading characters, as in
Kneser-Ney backoff), with gradient through the existing --anc-grad closed-form path.
It was built in the v1 CUDA trainer and sign-checked: the K/V gradient helps, and the harm
comes from diluting the attention softmax. At d=16, L=2, pd=0, 25 ep rmsprop, both variants
raised byte PPL over the 17.10 baseline (same-as-k 19.25, sentinel 18.71; 3 seeds). At L=4
and in a 100-epoch Adam run the effect was null. Closed 2026-05-31: the mechanism needs
pd=0 because h_subtree is fire-scoped, so it cannot run at pd=1, where AGPT reaches
useful PPL.
Code: branch slot-selection, tag exp/slot-selection-final (includes the 23 run directories with their result.json; the older tag exp/slot-selection predates them). Key files are
notes/seq-len-extension/slot-selection.md (branch version), src/cuda/agpt_backoff_kernels.cuh,
src/cuda/agpt_backoff_table.cuh, src/cuda/agpt_train.cu, src/cuda/kernels.cu and
src/tools/agpt_build_backoff_table.cr (YAML experimental.backoff_slots,
backoff_position). Run configs are in rnd/slot-selection-step0/configs/.