Agent Evaluation & Benchmarking

You've designed AI workflows. But how do you know if they actually work?

"It seems good" isn't good enough. This lab teaches you to systematically evaluate AI agents with the same rigor used by leading AI labs—but adapted for business applications.

What you'll create:

• A golden test set with input-output pairs
• Evaluation criteria and scoring rubrics
• Consistency measurements across multiple runs
• Failure mode documentation with improvement recommendations

"If you can't measure it, you can't improve it."

AI evaluation differs from traditional software testing:

Without systematic evaluation, you're flying blind.

Before testing, understand what you're measuring:

Step 1: Choose Your Evaluation Target

Select an AI workflow to evaluate. This could be:

• An existing AI assistant or chatbot
• A workflow you designed in Lab 3
• A prompt chain you've built
• A customer support automation

Document your target:

EVALUATION TARGET
─────────────────
System name: ________________________________
Purpose: ____________________________________
Key tasks it performs:
1. _________________________________________
2. _________________________________________
3. _________________________________________

Step 2: Define Success Criteria

For each key task, define what "success" looks like:

Example for a customer support agent:

Your turn—define 5 success criteria:

SUCCESS CRITERIA
────────────────
Task 1: ____________________
  Success means: ____________________
  Measured by: ____________________

Task 2: ____________________
  Success means: ____________________
  Measured by: ____________________

Task 3: ____________________
  Success means: ____________________
  Measured by: ____________________

Task 4: ____________________
  Success means: ____________________
  Measured by: ____________________

Task 5: ____________________
  Success means: ____________________
  Measured by: ____________________

Step 3: Create a Scoring Rubric

Convert subjective criteria to scores:

Example rubric for "response quality":

Create rubrics for your top 3 criteria:

Golden Examples (10 test cases)

Golden examples are input-output pairs where you know what "good" looks like.

Structure:

TEST CASE #___
──────────────
Input: [What the user says/submits]
Expected output: [What a good response includes]
Success criteria: [Which criteria this tests]
Evaluation notes: [What to look for when scoring]

Create 10 golden examples:

Test Cases 1-4: Happy Path Standard requests that should work perfectly.

TEST CASE #1 (Happy Path)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

TEST CASE #2 (Happy Path)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

TEST CASE #3 (Happy Path)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

TEST CASE #4 (Happy Path)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

Test Cases 5-7: Edge Cases Ambiguous, unusual, or boundary-testing inputs.

TEST CASE #5 (Edge Case - Ambiguous)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

TEST CASE #6 (Edge Case - Unusual)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

TEST CASE #7 (Edge Case - Boundary)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

Test Cases 8-9: Adversarial Attempts to confuse, manipulate, or break the system.

TEST CASE #8 (Adversarial - Confusion)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

TEST CASE #9 (Adversarial - Manipulation)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

Test Case 10: Out of Scope Request the agent should refuse or escalate.

TEST CASE #10 (Out of Scope)
Input: ________________________________________________
Expected: _____________________________________________
Tests: ________________________________________________

Execute Your Test Suite

Run each test case through your AI system. For each response:

1. Record the actual output
2. Score against your rubric
3. Note any unexpected behaviors

Evaluation Log Template:

TEST CASE #___
──────────────
Input: [copy from test set]

Actual Output:
______________________________________________
______________________________________________
______________________________________________

Scores:
  Criterion 1 (________): ___ / 5
  Criterion 2 (________): ___ / 5
  Criterion 3 (________): ___ / 5

Overall: ___ / 5

Notes:
______________________________________________

Measure Consistency

Run 3 of your test cases three times each. Record variation:

CONSISTENCY CHECK
─────────────────
Test Case #___:
  Run 1 score: ___
  Run 2 score: ___
  Run 3 score: ___
  Variance: ___
  Notes on differences: _______________________

Test Case #___:
  Run 1 score: ___
  Run 2 score: ___
  Run 3 score: ___
  Variance: ___
  Notes on differences: _______________________

Test Case #___:
  Run 1 score: ___
  Run 2 score: ___
  Run 3 score: ___
  Variance: ___
  Notes on differences: _______________________

Failure Categorization

Review your results and categorize any failures:

Pattern Analysis

Look for patterns in your failures:

FAILURE PATTERNS
────────────────
Most common failure type: _______________________
Triggers for failures: __________________________
Percentage of tests with issues: _____%

Root cause hypotheses:
1. ____________________________________________
2. ____________________________________________
3. ____________________________________________

Improvement Recommendations

Based on your analysis, what changes would improve performance?

RECOMMENDED IMPROVEMENTS
────────────────────────
Priority 1 (Critical):
  Problem: _____________________________________
  Solution: ____________________________________
  Expected impact: _____________________________

Priority 2 (Important):
  Problem: _____________________________________
  Solution: ____________________________________
  Expected impact: _____________________________

Priority 3 (Nice to have):
  Problem: _____________________________________
  Solution: ____________________________________
  Expected impact: _____________________________

Compile your work into an Agent Evaluation Report:

AGENT EVALUATION REPORT
═══════════════════════

System Evaluated: _______________________________
Date: __________________________________________
Evaluator: _____________________________________

EXECUTIVE SUMMARY
─────────────────
Overall score: ___ / 5
Tests passed: ___ / 10
Consistency score: ___ (low variance = good)
Production ready: Yes / No / With changes

KEY METRICS
───────────
Task success rate: ____%
Average quality score: ___ / 5
Failure rate: ____%
Top failure type: _______________

DETAILED FINDINGS
─────────────────
Strengths:
• ___________________________________________
• ___________________________________________

Weaknesses:
• ___________________________________________
• ___________________________________________

RECOMMENDATIONS
───────────────
1. ___________________________________________
2. ___________________________________________
3. ___________________________________________

APPENDIX
────────
[Attach your test cases and evaluation logs]

For context, here's how leading benchmarks evaluate agents:

Your business evaluation doesn't need to match these formats—but understanding them helps you think systematically.

1. What surprised you most about your evaluation results?
2. How would you explain these results to a non-technical stakeholder?
3. What's the minimum acceptable score for production deployment?
4. How often should you re-run this evaluation?
5. What test cases would you add for the next evaluation cycle?

After completing this lab:

• Lab 4: Design quality gates based on your evaluation criteria
• Module 3.4: Learn deployment strategies that incorporate evaluation
• Lab 5: Build an assistant with evaluation built-in