exo

exo enables users to connect multiple devices into an AI cluster for running large-scale AI models locally, utilizing advanced features like automatic device discovery, RDMA over Thunderbolt for reduced latency, and topology-aware parallelization. Notably, it supports tensor parallelism for performance optimization, compatibility with popular AI APIs, and loading custom models from sources like HuggingFace. The built-in dashboard facilitates cluster management and interaction with AI models.