Logical Deduction Task Evaluation

Zero-shot LLM evaluation for reasoning tasks.

Unique: Provides unified evaluation framework for both symbolic logic and natural language reasoning puzzles in zero-shot setting, with answer verification that can handle both formal symbolic validation and semantic similarity-based matching for natural language conclusions

vs others: More specialized than general reasoning benchmarks; focuses specifically on logical deduction without few-shot examples, enabling cleaner measurement of foundational logical capability vs. pattern-matching from examples

via “logical deduction and inference evaluation”

23 hardest BIG-Bench tasks where models initially failed.

Unique: Isolates formal logical reasoning as a distinct capability by presenting logic problems in natural language with few-shot examples, testing whether models can apply logical rules consistently without explicit training. This approach measures logical inference generalization.

vs others: More focused on formal logical reasoning than general reasoning benchmarks; more accessible than formal logic verification because it uses natural language rather than symbolic logic notation.

via “logical-reasoning-and-deduction”

Mercury 2 is an extremely fast reasoning LLM, and the first reasoning diffusion LLM (dLLM). Instead of generating tokens sequentially, Mercury 2 produces and refines multiple tokens in parallel, achieving...

Unique: Applies diffusion-based parallel reasoning to logical deduction and constraint satisfaction, enabling fast multi-step logical reasoning without sequential token overhead

vs others: Faster logical reasoning than sequential reasoning models because parallel token refinement computes multiple logical steps simultaneously while maintaining logical coherence

via “logical reasoning and deduction”

via “reasoning and logical inference”