Progressive growth vs SGD

Progressive Growth vs SGD Baseline

Status: initial run set complete

Question

Compare CUDAX progressive growth at several division/epoch schedules against a standard Crystal µGPT sliding-window baseline at the same model size and context length.

Protocol

Each run is a subdirectory under this directory. The copied config.yml, resolved_config.json, meta.json, eval_raw.json, and result.json in each run directory are the committed canonical record for that run. Raw logs and checkpoints remain local debugging artifacts.

Runs

Caveats

• train (s) is the primary timing column. For CUDAX runs it is the sum of trainer-reported growth-stage-timing totals. For the µGPT run it is approximated from the train log close time because the old µGPT binary did not report elapsed training time.
• total (s) is full harness time: split, training, HF conversion, rolling evaluation, fixed-token evaluation, and aggregation.
• Some runs overlapped with other GPU/CPU work, so timing is noisy. Treat PPL as the primary result from this batch.
• The SGD baseline is preliminary. A fair baseline needs explicit agreement on matching criterion: wall time, optimizer steps, target-token exposures, or some combination of these.

Results

<!-- agpt-experiment-table:start -->