Autonomous overnight LLM research setup where an AI agent edits train.py, trains 5 minutes, and keeps improvements.
Field note
Give an AI agent a small but real single-GPU LLM training setup and let it experiment autonomously overnight, editing train.py and keeping or discarding results.
Capabilities
Tags
Ways to use it
python / uv
python / https://docs.astral.sh/uv/
Product features
Fixed constants, one-time data prep (downloads training data, trains a BPE tokenizer) and runtime utilities.
Baseline instructions for one agent, edited and iterated on by the human.
The single file the agent edits, containing the GPT model, optimizer (Muon + AdamW) and training loop.
Validation bits per byte; lower is better and vocab-size-independent so architectural changes are fairly compared.