Distributed training frameworks
Distributed training frameworks covers frameworks for training one model across many GPUs or many machines.
In brief
Distributed training frameworks, in brief.
What it isDistributed training frameworks covers frameworks for training one model across many GPUs or many machines.
Where it sitsOne of 12 categories in the Training and fine-tuning layer. Training and fine-tuning covers the frameworks, libraries and platforms used to train a model from scratch or adapt an existing one to a task, and the tools that track those experiments.
What we track24 products, including Amazon SageMaker distributed training libraries (SMDDP / SMP), Colossal-AI, Composer, DeepSpeed and DLRover.
The products
24 products in Distributed training frameworks.
Each name opens a search for that product in our index.
ProductMakerIts own site
From our census of 2026-11-08. A product listed here is one that belongs in this category; the search answer says whether we have checked it.
Questions
Distributed training frameworks, answered.
What is Distributed training frameworks in AI infrastructure?Distributed training frameworks covers frameworks for training one model across many GPUs or many machines.
Which layer of AI infrastructure is Distributed training frameworks in?Distributed training frameworks is in the Training and fine-tuning layer. Training and fine-tuning covers the frameworks, libraries and platforms used to train a model from scratch or adapt an existing one to a task, and the tools that track those experiments.
Which products are in Distributed training frameworks?We track 24 products in Distributed training frameworks, including Amazon SageMaker distributed training libraries (SMDDP / SMP), Colossal-AI, Composer, DeepSpeed, DLRover, FairScale (legacy; largely superseded by FSDP), GPT-NeoX and Hivemind (decentralized training). The full list is on this page.
Training and fine-tuning
The other categories in Training and fine-tuning.
AutoML & hyperparameter optimizationComputer vision toolkits & platformsDeep learning & ML frameworksExperiment trackingFeature storesFederated & privacy-preserving learningFine-tuning platformsML workflow / MLOps platformsOpen-source fine-tuning librariesRL / post-training librariesRL environments & gyms