Inference kernels & attention libraries
Inference kernels & attention libraries covers low-level GPU kernels and DSLs that engines call for attention, GEMM, MoE.
In brief
Inference kernels & attention libraries, in brief.
What it isInference kernels & attention libraries covers low-level GPU kernels and DSLs that engines call for attention, GEMM, MoE.
Where it sitsOne of 13 categories in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
What we track28 products, including AITER, BitBLAS, Composable Kernel, cuBLAS / cuDNN and CUTLASS / CuTe DSL.
The products
28 products in Inference kernels & attention libraries.
Each name opens a search for that product in our index.
ProductMakerIts own site
From our census of 2026-11-08. A product listed here is one that belongs in this category; the search answer says whether we have checked it.
Questions
Inference kernels & attention libraries, answered.
What is Inference kernels & attention libraries in AI infrastructure?Inference kernels & attention libraries covers low-level GPU kernels and DSLs that engines call for attention, GEMM, MoE.
Which layer of AI infrastructure is Inference kernels & attention libraries in?Inference kernels & attention libraries is in the Inference and serving layer. Inference and serving is everything that turns a trained model into a running service: hosted inference providers, gateways that route between them, serving engines, and the tools that make models smaller and faster.
Which products are in Inference kernels & attention libraries?We track 28 products in Inference kernels & attention libraries, including AITER, BitBLAS, Composable Kernel, cuBLAS / cuDNN, CUTLASS / CuTe DSL, DeepEP, DeepGEMM and FBGEMM / FBGEMM_GPU. The full list is on this page.
Inference and serving
The other categories in Inference and serving.
AI gateways & model routersBatch inferenceGenerative media inference platformsGenerative media workflow toolsInference benchmarking & load testingInference providersKV-cache & prompt-caching infrastructureLoRA serving & adapter hubsModel compilers & graph optimizersOn-device / edge runtimesQuantization / optimizationServing engines & model servers