agent-evaluation · agent-evaluation verified and signed

Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation w

langfuse · langfuse verified and signed

Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.

arize-evaluator · arize-evaluator verified and signed

Handles LLM-as-judge and code evaluator workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, code evaluator, h

phoenix-evals · phoenix-evals verified and signed

Build, run, and validate evaluators for AI/LLM applications with Phoenix, in Python or TypeScript (including Vercel AI SDK apps) — code-based checks and LLM-as-a-judge, run over exported traces or dataframes or as Phoenix experiments, gate them in CI, and validate judges against human-labeled data.

agentic-eval · agentic-eval verified and signed

Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-j

evalgate_compare_models · evalgate found in code, not yet checked

Run an A/B benchmark shootout comparing two LLMs on the same evaluation suite.

· agentclash website read, tools not yet checked

Build, replay, score, and regression-test AI agents with open-source evals, traces, benchmarks, and CI gates. A LangSmith alternative for agent workflow testing and prompt regression testing.

create-task · create-task verified and signed

Create a new Harbor task for evaluating agents. Use when the user wants to scaffold, build, or design a new task, benchmark problem, or eval. Guides through instruction writing, environment setup, verifier design (pytest vs Reward Kit vs custom), and solution scripting.

evaluate_with_llm_judge · iris-eval/mcp-server found in code, not yet checked

deepeval · deepeval verified and signed

DeepEval evaluation workflow for AI agents and LLM applications. TRIGGER when the user wants to evaluate or improve an AI agent, tool-using workflow, multi-turn chatbot, RAG pipeline, or LLM app; add evals; generate datasets or goldens; use deepeval generate; use deepeval test run; send results to C