Verified record · from the answer index

pytorch-fsdp2

Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMes

skill:orchestra-research/ai-research-skills/pytorch-fsdp2 · published by orchestra-research · source repository