I'll start by pulling the relevant method guidance and checking what compute and packages are available.
Sizing the ablation study before any training run
Torch 2.13 CPU with 2 cores is available locally, and a headless GPU is not. Before committing to a training budget I'll fetch the standard char-level corpus and time a forward/backward pass to size the run.
Sizing the ablation study before any training run
Measured per-step cost so the run budget can be set
Generated · 1
Timing is in: 0.69 s/step at batch 64, 0.34 s/step at batch 32 (block 128, 4 layers, d=128, 0.83 M params). That sizes a full sweep at a couple of hours on the 2 CPU cores. This is long enough compute that I'll lay out the design and budget for sign-off before launching.
Getting sign-off on design and compute budget before training
Plan awaits the researcher's approval
Plan approved. Building the harness: a switchable model and a training script that logs to CSV.
Building the harness for the ablation sweep
Smoke-testing all four configurations for 30 steps to verify they run and to record parameter/FLOP counts.
Building the harness for the ablation sweep
All four switch settings run; parameter and FLOP counts recorded
