Grammar Constrained Text Generation With Token Healing

via “grammar-constrained text generation with token healing”

Microsoft's language for efficient LLM control flow.

Unique: Implements token healing at the text level (not token level) with an immutable GrammarNode AST architecture, allowing constraints to be composed and reused across programs while maintaining correct behavior at token boundaries. The TokenParser/ByteParser dual-engine design handles both token-level and byte-level constraints without requiring external validation passes.

vs others: More efficient than post-generation validation (no retry loops) and more flexible than simple prompt engineering because constraints are enforced during generation, not after, reducing wasted tokens and guaranteeing format compliance on first attempt.

via “grammar-constrained text generation with token-aware parsing”

A guidance language for controlling large language models.

Unique: Implements token healing at the text level rather than token level, allowing precise constraint enforcement across token boundaries without requiring model retraining. Uses immutable GrammarNode AST with TokenParser/ByteParser engines that integrate directly with model tokenizers via llguidance, enabling sub-token-level constraint enforcement.

vs others: Faster and more reliable than post-processing validation because constraints are enforced during generation rather than after, and more flexible than LORA-based approaches because it works with any model backend without fine-tuning.

via “grammar-constrained generation with ebnf rules”

Python bindings for the llama.cpp library

Unique: Integrates llama.cpp's grammar engine for token-level constraint enforcement, guaranteeing syntactic correctness without post-processing, while maintaining semantic quality from the model's learned patterns

vs others: More reliable than prompt-based JSON generation (no hallucinated fields), and faster than post-processing validation because constraints are enforced during generation rather than after