48 evaluation tools for Claude Code, Cursor, and any MCP-compatible client. Manage agents, ground truth datasets, eval runs, deployments, and drift monitoring — without leaving your editor.
1. Install
2. Set your API key
3. Add to Claude Code
Or add to project config
11 categories · 52 tools
Inspect and manage your instrumented agents and their version history.
list_agentsList all agents in the project with their versions, environments, and status.
get_agentGet details for a specific agent by name or ID.
list_agent_versionsList all versions for an agent.
get_agent_versionGet a specific version of an agent including prompts.
diff_agent_versionsShow the diff between two agent versions (what changed in prompts/config).
create_agent_versionCreate a new agent version with updated prompts.
link_gt_to_versionLink a ground truth dataset to an agent version for evaluation.
Create, manage, and evolve ground truth datasets that encode domain expertise.
list_gt_datasetsList ground truth datasets with optional filters.
get_gt_datasetGet details of a ground truth dataset.
create_gt_datasetCreate a new ground truth dataset.
delete_gt_datasetDelete a ground truth dataset and all its entries.
list_gt_entriesList entries in a ground truth dataset.
add_gt_entriesAdd entries to a ground truth dataset.
update_gt_entryUpdate a ground truth entry.
delete_gt_entryDelete a single ground truth entry.
diff_gt_datasetsCompare two versions of a ground truth dataset.
get_gt_evolutionShow the version history of a GT dataset including expert enrichments.
preview_correctionsPreview expert corrections from an eval run's reviews before enriching GT.
enrich_gtCreate a new GT version enriched with expert corrections from an eval run.
Run evaluations comparing agent versions against ground truth benchmarks.
run_evalRun an evaluation comparing an agent version against a ground truth dataset.
list_eval_runsList recent eval runs with optional filters.
get_eval_runGet detailed results for an eval run.
list_eval_entriesList entries from an eval run (pass/fail per GT entry).
analyze_eval_runTrigger AI Analyst to analyze an eval run's failures and find patterns.
generate_review_schemaGenerate targeted review questions from eval failure patterns.
send_to_reviewSend failed eval entries to the expert review queue.
AI-generated context improvements synthesized from expert feedback.
list_suggestionsList AI-generated improvement suggestions for an eval run.
update_suggestionAccept, reject, or edit a suggestion. Accepting creates a new agent version.
synthesize_suggestionsTrigger AI Editor to synthesize expert feedback into prompt improvement suggestions.
Promote agent versions across environments with full rollback support.
promote_agentPromote an agent version to staging or production.
rollback_agentRollback an agent to its previous environment.
get_promotion_historyGet promotion/rollback history for an agent.
Track agent quality over time and detect regressions across versions.
get_agent_performanceGet eval score timeline across all versions of an agent.
get_regressionsDetect regressions across all versions — entries that were right then wrong.
analyze_regressionsAI root cause analysis on regressions — explains WHY entries broke.
Confusion matrices, precision/recall tracking, and classification transitions.
get_confusion_matrixGet confusion matrix and metrics (precision, recall, F1, accuracy) for an eval run.
get_metrics_historyGet precision/recall/F1/accuracy history across eval runs for an agent.
get_classification_transitionsGet classification transitions (TP to FN, TN to FP, etc.) from the previous eval run.
Configure and check production drift detection against ground truth.
check_driftCheck production drift against ground truth.
list_drift_reportsList recent drift reports for an agent.
get_monitoring_configGet drift monitoring configuration for an agent.
update_monitoring_configConfigure drift monitoring for an agent.
Full and canary deployments with validation and rollback.
create_deploymentCreate a new deployment (full or canary).
validate_deploymentRun validation on a deployment (compares against GT baseline).
start_canaryStart canary traffic for a deployment.
promote_deploymentPromote a canary deployment to full rollout.
rollback_deploymentRollback a deployment.
Search traces, feedback, and aggregate agent statistics.
query_spansQuery agent execution traces with optional filters.
query_feedbackQuery expert feedback/reviews with optional filters.
get_agent_statsGet aggregate statistics for an agent (spans, reviews, versions).
Run N automated improvement rounds: analyst → editor → new version → re-eval.
run_automated_loopRun N automated improvement rounds (analyst → editor → new version → re-eval).
get_loop_statusGet status of an automated improvement loop.