Spmd Parallelism With Automatic Axis Annotation And Sharding

Neural network library for JAX with functional patterns.

Unique: Integrates JAX's pmap with Flax's variable system to automatically handle parameter sharding and gradient synchronization across devices, with optional axis annotations for model parallelism, eliminating manual collective operation code

vs others: More flexible than PyTorch DDP because it supports model parallelism and fine-grained sharding control; more explicit than TensorFlow's distribution strategies because sharding decisions are visible in code