Functional Transformations Composition With Jaxpr Intermediate Representation

via “custom-primitive-operation-registration”

Google's numerical computing library — autodiff, JIT, vectorization, NumPy API for ML research.

Unique: JAX's primitive system is unified — a single primitive definition automatically works with grad, jit, vmap, and pmap through a tracer-based architecture. Users define gradient rules once, and the system automatically applies them in any transformation context. This is achieved through a rule-based system where each transformation queries the primitive for its behavior (gradient, batching, compilation).

vs others: More composable than PyTorch's custom autograd functions because custom operations automatically work with jit, vmap, and pmap; more flexible than TensorFlow's custom ops because gradient rules are defined in Python rather than C++

Differentiate, compile, and transform Numpy code.

Unique: JAX's transformations (grad, jit, vmap, pmap) operate on a pure functional intermediate representation (Jaxpr) that enables arbitrary composition without special-casing. Each transformation produces a new Jaxpr by applying interpretation rules, enabling nested transformations like grad(jit(vmap(f))). The system is fundamentally different from eager execution frameworks, where transformations are applied at runtime with less opportunity for optimization.

vs others: Enables arbitrary transformation composition with compiler optimization, whereas PyTorch's autograd and TensorFlow's eager execution apply transformations at runtime with less optimization opportunity.