arize-evaluator · arize-evaluator verified and signed

Handles LLM-as-judge evaluation workflows on Arize including creating/updating evaluators, running evaluations on spans or experiments, managing tasks, trigger-run operations, column mapping, and continuous monitoring. Use when the user mentions create evaluator, LLM judge, hallucination, faithfulne

llm-as-a-judge · langfuse verified and signed

evaluate_with_llm_judge · iris-eval/mcp-server found in code, not yet checked

agent-evaluation · agent-evaluation verified and signed

Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation w

adk-eval-guide · adk-eval-guide verified and signed

MUST READ before running any ADK evaluation. ADK evaluation methodology — eval metrics, evalset schema, LLM-as-judge, tool trajectory scoring, and common failure causes. Use when evaluating agent quality, running adk eval, or debugging eval results. Do NOT use for API code patterns (use adk-cheatshe

· CompletionKit website read, tools not yet checked

Prompt evaluation for AI apps. Run prompts against real inputs, score outputs 1-5 with an LLM judge, ship only what improved. Free on Cloud, or self-host.

· Ollama website read, tools not yet checked

Ollama is the easiest way to run and build with open AI models locally or in the cloud on macOS, Windows, and Linux.

add · mcp-evals verified and signed

mlflow · mlflow-skinny verified and signed

get_evaluation_details · promptfoo found in code, not yet checked