Keras vs vLLM

Keras 3 compiles a single model definition into executable code for JAX, TensorFlow, PyTorch, or OpenVINO by deferring all numerical operations to a pluggable backend abstraction layer. The active backend is selected at import time via KERAS_BACKEND environment variable or ~/.keras/keras.json and cannot be changed post-import. During model construction, symbolic execution via compute_output_spec() infers shapes and dtypes without computation; during training/inference, calls dispatch to backend-specific implementations in keras/src/backend/{jax,torch,tensorflow,openvino}/. This architecture enables write-once-run-anywhere model code without backend-specific rewrites.

Unique: Keras 3's multi-backend architecture uses a two-path execution model: symbolic dispatch during model construction (compute_output_spec for shape/dtype inference) and eager dispatch during execution (forwarding to backend-specific implementations in keras/src/backend/). This differs from PyTorch (eager-first) and TensorFlow (graph-first) by supporting both paradigms transparently. The keras/src/ source-of-truth with auto-generated keras/api/ public surface ensures consistency across backends without manual duplication.

vs alternatives: Unlike PyTorch (PyTorch-only), TensorFlow (TensorFlow-only), or JAX (functional-only), Keras 3 enables identical model code to run on all four major frameworks with a single import-time configuration, eliminating framework lock-in without sacrificing backend-specific performance tuning.

Keras provides two high-level APIs for composing neural networks: Sequential (linear stack of layers) and Functional (arbitrary directed acyclic graphs with multiple inputs/outputs). Both APIs accept layer instances (Dense, Conv2D, LSTM, etc.) and automatically handle tensor shape inference, weight initialization, and forward pass construction. The Functional API supports layer sharing, multi-branch architectures, and residual connections by explicitly passing tensors between layer calls. Under the hood, layers inherit from keras.layers.Layer, which implements __call__ to dispatch to backend-specific compute_output_spec (symbolic) and call (eager) methods, enabling shape validation before execution.

Unique: Keras's Functional API enables arbitrary DAG construction by explicitly passing tensors between layer calls, unlike PyTorch's imperative nn.Module (which requires forward() implementation) or TensorFlow's eager execution (which mixes definition and execution). The symbolic compute_output_spec() method infers output shapes and dtypes during model construction without allocating memory or running computation, enabling early validation of architecture errors.

vs alternatives: Keras's declarative APIs require 50-70% less boilerplate than PyTorch's nn.Module for standard architectures and provide automatic shape inference that TensorFlow's Keras layer API also offers, but Keras 3 adds multi-backend portability that neither PyTorch nor TensorFlow alone provides.

Keras provides model.save() and keras.saving.load_model() for serializing and deserializing models. Models can be saved in three formats: Keras format (HDF5 or ZIP with architecture + weights), SavedModel (TensorFlow format with concrete functions), or ONNX. The Keras format stores model architecture as JSON and weights as HDF5 or NumPy files. Deserialization reconstructs the model from saved architecture and weights, and custom layers/losses/metrics can be registered via custom_objects parameter. Model checkpointing during training is handled by keras.callbacks.ModelCheckpoint, which saves the best model based on validation metrics. Weights can be saved/loaded independently via model.save_weights() and model.load_weights().

Unique: Keras 3's serialization system supports multiple formats (Keras, SavedModel, ONNX) and works across backends by storing architecture as backend-agnostic JSON and weights as NumPy arrays. Custom layers/losses/metrics are serialized via get_config() and can be reconstructed via from_config(), enabling full model reproducibility.

vs alternatives: Unlike PyTorch (torch.save for weights only, requires manual architecture saving) or TensorFlow (SavedModel-centric), Keras provides unified serialization to multiple formats with automatic architecture and weight saving, and unlike ONNX converters, Keras serialization is built-in and ensures consistency.

Keras provides keras.optimizers.schedules for learning rate scheduling (ExponentialDecay, CosineDecay, PolynomialDecay, etc.) and keras.callbacks for hyperparameter tuning (LearningRateScheduler, ReduceLROnPlateau). Learning rate schedules decay the learning rate over training steps or epochs to improve convergence. Callbacks enable dynamic hyperparameter adjustment during training (e.g., reducing learning rate when validation loss plateaus). Keras also integrates with external hyperparameter optimization frameworks (Keras Tuner, Optuna, Ray Tune) via callbacks. The fit() method accepts learning rate schedules and callbacks, enabling end-to-end hyperparameter optimization without custom training loops.

Unique: Keras's learning rate schedules (keras.optimizers.schedules) are decoupled from optimizers and can be composed with callbacks (LearningRateScheduler, ReduceLROnPlateau) for dynamic hyperparameter adjustment during training. This differs from PyTorch (torch.optim.lr_scheduler) and TensorFlow (tf.keras.optimizers.schedules) by providing a unified callback-based interface.

vs alternatives: Unlike PyTorch (torch.optim.lr_scheduler, which requires manual step() calls) or TensorFlow (tf.keras.optimizers.schedules, which is TensorFlow-only), Keras 3's learning rate schedules integrate seamlessly with fit() and callbacks, enabling automatic hyperparameter adjustment without custom training loops.

Keras enables custom layer implementation by subclassing keras.layers.Layer and implementing build() (weight initialization), call() (forward pass), and compute_output_spec() (shape inference). Custom loss functions can be implemented by subclassing keras.losses.Loss or as callables. Custom layers and losses automatically support automatic differentiation through the active backend (JAX, PyTorch, TensorFlow) without requiring manual gradient implementation. Custom operations can use keras.ops for backend-agnostic computation or backend-specific ops for optimization. The framework handles gradient computation, mixed-precision scaling, and distributed training for custom layers/losses without user code changes.

Unique: Keras's custom layer interface (subclassing keras.layers.Layer) requires implementing build(), call(), and compute_output_spec(), enabling both eager and symbolic execution. Custom layers automatically support automatic differentiation, mixed-precision training, and distributed training through the backend abstraction, without requiring manual gradient implementation.

vs alternatives: Unlike PyTorch (torch.nn.Module, which requires manual forward() and no shape inference) or TensorFlow (tf.keras.layers.Layer, which is TensorFlow-only), Keras 3's custom layer interface supports both eager and symbolic execution and works across backends, enabling custom layers to be written once and run anywhere.

Keras provides model.summary() to print a human-readable summary of model architecture (layer names, output shapes, parameter counts, connectivity). The summary includes total trainable and non-trainable parameters, enabling quick model size estimation. Keras also supports model graph visualization via keras.utils.plot_model(), which generates a visual diagram of the model architecture (useful for Functional API models with complex connectivity). Model introspection methods (model.get_config(), model.get_weights()) enable programmatic access to architecture and weights. These tools are backend-agnostic and work identically across JAX, PyTorch, and TensorFlow.

Unique: Keras's model.summary() and keras.utils.plot_model() are backend-agnostic and work identically across JAX, PyTorch, and TensorFlow. The summary includes parameter counts and connectivity information, enabling quick model size estimation and architecture validation.

vs alternatives: Unlike PyTorch (torchsummary or torchinfo for summary, no built-in visualization) or TensorFlow (tf.keras.utils.plot_model, TensorFlow-only), Keras 3 provides unified model introspection and visualization across backends with minimal dependencies.

Keras provides built-in regularization through layer parameters and dedicated layers: kernel_regularizer/bias_regularizer (L1/L2 weight regularization), activity_regularizer (activation regularization), Dropout layer (random unit dropping), and BatchNormalization layer (feature normalization with learnable scale/shift). Regularization is applied during training via the loss function (for weight regularization) or forward pass (for dropout, batch norm). Dropout randomly zeros activations during training and scales them during inference. BatchNormalization normalizes activations to zero mean and unit variance, reducing internal covariate shift and enabling higher learning rates. All regularization techniques are backend-agnostic and work identically across JAX, PyTorch, and TensorFlow.

Unique: Keras integrates regularization into layer parameters (kernel_regularizer, activity_regularizer) and dedicated layers (Dropout, BatchNormalization), enabling regularization to be specified declaratively without custom code. Regularization is applied automatically during training and inference, and all techniques are backend-agnostic.

vs alternatives: Unlike PyTorch (torch.nn.Dropout, torch.nn.BatchNorm, manual weight regularization in optimizer) or TensorFlow (tf.keras.regularizers, TensorFlow-only), Keras 3 provides unified regularization across backends with declarative layer parameters, reducing boilerplate by 50-70%.

Keras delegates automatic differentiation to the active backend (JAX's jax.grad, PyTorch's autograd, TensorFlow's tf.GradientTape) through a unified keras.ops interface that wraps backend-specific gradient functions. During training, the fit() method constructs a loss function, computes gradients via backend-native autodiff, and applies optimizer updates. Custom training loops can use keras.ops.grad() to compute gradients of arbitrary functions. The backend abstraction ensures that gradient computation, mixed-precision scaling, and gradient clipping work identically across JAX, PyTorch, and TensorFlow without user code changes.

Unique: Keras 3 abstracts automatic differentiation through keras.ops.grad(), which dispatches to backend-specific implementations (jax.grad, torch.autograd, tf.GradientTape) while maintaining a unified API. This enables custom training loops to work identically across backends without conditional logic. Gradient checkpointing (remat) is implemented as a backend-agnostic decorator that can be applied to layers to reduce memory usage during backpropagation.

vs alternatives: Unlike PyTorch (torch.autograd-specific) or TensorFlow (tf.GradientTape-specific), Keras 3's unified gradient API allows the same training code to run on any backend, and unlike JAX (which requires functional programming), Keras supports imperative gradient computation through fit() and custom training loops.

Implements virtual memory-style paging for KV cache tensors, allocating fixed-size blocks (pages) that can be reused across requests without contiguous memory constraints. Uses a block manager that tracks physical-to-logical page mappings, enabling efficient memory fragmentation reduction and dynamic batching of requests with varying sequence lengths. Reduces memory overhead by 20-40% compared to contiguous allocation while maintaining full sequence context.

Unique: Introduces block-level virtual memory paging for KV caches (inspired by OS page tables) rather than request-level allocation, enabling fine-grained reuse and prefix sharing across requests without memory fragmentation

vs alternatives: Achieves 10-24x higher throughput than HuggingFace Transformers' contiguous KV allocation by eliminating memory waste from padding and enabling aggressive request batching

Implements a scheduler (Scheduler class) that dynamically groups incoming requests into batches at token-generation granularity rather than request granularity, allowing new requests to join mid-batch and completed requests to exit without stalling the pipeline. Uses a priority queue and state machine to track request lifecycle (waiting → running → finished), with configurable scheduling policies (FCFS, priority-based) and preemption strategies for SLA enforcement.

Unique: Decouples batch formation from request boundaries by scheduling at token-generation granularity, allowing requests to join/exit mid-batch and enabling prefix caching across requests with shared prompt prefixes

vs alternatives: Reduces TTFT by 50-70% vs static batching (HuggingFace) by allowing new requests to start generation immediately rather than waiting for batch completion

Tracks request state through a finite state machine (waiting → running → finished) with detailed metrics at each stage. Maintains request metadata (prompt, sampling params, priority) in InputBatch objects, handles request preemption and resumption for SLA enforcement, and provides hooks for custom request processing. Integrates with scheduler to coordinate request transitions and resource allocation.