Tagged evals

Plan agent-run engineering with eval-first checks and cost-aware model routing

Run autonomous agent loops that self-check, gate on evals and recover from failures

Build task-local agent harnesses with eval gates and reusable skill extraction

Define capability and regression evals before coding and track pass@k reliability

Block unsafe healthcare deployments with automated patient safety test suites

Run production ML work with data contracts, reproducible training, evals and rollback

Benchmark language models on MMLU, GSM8K and other tasks with lm-eval

Create, edit, eval and benchmark your own agent skills

Preregister a paid model eval with bars, arms and stop rules before any spend

Create LLM eval suites, run benchmarks and compare model scorecards from the CLI

Remove eval and other unsafe dynamic code execution from scripts

Create, edit and benchmark agent skills, including evals and description tuning

Design agent evals with pass@k metrics, verifiers and LLM-as-a-Judge rubrics

Design agent eval tasks, verifiers and datasets that resist leakage

Run evals on a skill against a baseline and a rubric, then report results

Run and debug RAG evaluations on long file attachments using test fixtures

Build the eval harness, golden sets and graders that gate every fine-tuning run

Build automated and human evals with benchmarks to measure LLM app quality

Turn eval traces and production logs into SFT examples and preference pairs

Improve agent outputs with self-critique loops, rubrics and evaluator passes

Create, append to and export Arize datasets and eval examples with the ax CLI

Improve an AI app through evals: define criteria, build golden datasets and act on results

Build and run Phoenix evaluators that score AI and LLM application outputs

Set up continuous online evaluations that score live AI generations and traces

Monitor LLM traces for cost, latency, error and eval regressions

Design and run evals for LLM, agent and retrieval systems with clear grading

Evaluate LLM apps with automated metrics, human review and A/B tests

Write skill evals from real usage history and user corrections instead of specs

Turn a raw feature into a tested, resolvable agent skill with an eval contract

Build regression evals that prove an agent change did not make it worse

Challenge AI feature plans on evals, model choice, cost and risk before shipping

Have three AI models cross-review a board memo before it reaches the boardroom

Score and rank parallel agent results by metric or LLM judge

Grade a managed agent against a rubric and iterate until a version passes evals

Interview a founder to scope a managed agent and draft its build sheet and eval plan

Test marketing prompts with A/B evals, version history and ready-made templates

Block risky code edits like injection, XSS and unsafe eval before writing

Score AI work on two axes and catch inflated self-grades before you trust them

Optimize prompts and score LLM outputs and RAG retrieval against eval sets

Set up an autoresearch experiment with target file, metric and eval command

Measure and improve model and agent quality with eval datasets and failure analysis

Write DeepEval evals and datasets to test AI agents, RAG pipelines and chatbots

Build LLM-as-judge evals with pairwise comparison, rubrics and bias checks

Design autonomous agent harnesses with eval scaffolds, rollback and approval gates

Write, refactor and evaluate LLM prompts with output schemas and test suites

Design RAG pipelines with chunking, embeddings, hybrid search, reranking and evals

Trace and evaluate LLM apps with the Phoenix observability platform

Run model evaluations locally with inspect-ai and lighteval on your own GPUs

Audit an AI phase's evaluation coverage and write a remediation plan