10 actions in 8 pieces of software · ranked by words, meaning and measured trust · 636 ms

1. 1llm-evaluationverified and signedpassport

   skillwshobson/agentsrecord

   • llm-evaluation

     Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.

2. 2llm-evaluationverified and signedpassport

   skillyonatangross/orchestkitrecord

   • llm-evaluation

     LLM output evaluation and quality assessment. Use when implementing LLM-as-judge patterns, quality gates for AI outputs, or automated evaluation pipelines.

3. 3agent-evaluationverified and signedpassport

   skillmlflow/skillsrecord

   • agent-evaluation

     Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents systematically using MLflow evaluation w

4. 4promptfoo-evaluationverified and signedpassport

   skilldaymade/claude-code-skillsrecord

   • promptfoo-evaluation

     Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promp

5. 5evaluationverified and signedpassport

   skillmuratcankoylan/Agent-Skills-for-Context-Engineeringrecord

   • evaluation

     This skill should be used when the user asks to "evaluate agent performance", "build test framework", "measure agent quality", "create evaluation rubrics", or mentions LLM-as-judge, multi-dimensional evaluation, agent testing, or quality gates for agent pipelines.

6. 6agent-evaluationverified and signedpassport

   skilldavila7/claude-code-templatesrecord

   • agent-evaluation

     Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

7. 7LLM Evaluation Harness MCP Serverfound in code, not yet checkedno passport yet
   • score_sample

     Score a single RAG output on faithfulness, answer relevancy, and context precision using the LLM-as-a-Judge backend.

   • run_eval

     Run the full RAG evaluation on a JSONL dataset and return a report with per-metric aggregates and a pass/fail gate decision.

8. 8ia-qa mcp llm and RAG testing - Dev/QA toolboxverified and signedpassport

   MCP serverrecord

   • generate_eval_yaml

     Generate a complete .ia-eval.yaml evaluation contract from a plain-language description of what your LLM should do. Uses Groq llama-3.3-70b (server-side, no API key needed). Returns ready-to-run YAML for the LLM Test Runner (run_eval_contract). Picks appropriate evaluators (cosine_similarity, contai

   • run_eval_contract

     Parse a .ia-eval.yaml LLM test suite, call the specified LLM model for each scenario, run all configured scorers, and return a structured JSON report with per-scenario Pass/Fail verdicts and a Markdown summary. Use list_local_tests to discover available test files.