nanochat

nanochat is an experimental framework designed for training large language models (LLMs) on a single GPU node, emphasizing simplicity and customizability. It covers all major phases of LLM development—tokenization, pretraining, fine-tuning, evaluation, and inference—allowing users to efficiently train models like GPT-2 at significantly reduced costs. Key features include automatic hyperparameter optimization based on a single complexity setting, a leaderboard for tracking training performance, and streamlined CLI interaction for testing the trained models.