torchtune

torchtune is a PyTorch library for authoring, post-training and experimenting with LLMs. It supports single and multi-GPU setups and enables distributed training with FSDP2. The library provides YAML-based recipes for tasks like fine-tuning, inference, evaluation and quantization-aware training. Each recipe offers a focused set of features, avoiding complex flag-based configurations. It prioritizes simplicity, favoring code clarity over excessive abstractions. It also includes a CLI for downloading models, managing recipes and running experiments efficiently.