petals

Petals is a distributed framework that facilitates running and fine-tuning large language models like Llama 3.1 and BLOOM from personal computers or Google Colab environments. The tool supports BitTorrent-style model layer sharing, enabling users to achieve inference and fine-tuning speeds up to 10 times faster than traditional offloading methods. Notable features include support for various cutting-edge models, a community-driven GPU sharing system, and the ability to configure private swarms for enhanced privacy.