QA MentorUniversityApply now

Turn Your QA Experience Into AI Evaluation Skills.

Practical AI Testing and Evaluation for QA Professionals

Put your QA skills to work on AI. Get 50 hours of training to evaluate chatbots, RAG applications, and AI agents. Complete a chatbot capstone, build a report that demonstrates your testing judgment, and practice presenting your expertise through mock interviews.

Apply for the January ProgramSee your learning journey

Starts January 9, 2027 · Saturdays at 10:00 AM EST · Live online

Your capstone deliverables16-hour project

Leave with evidence of how you test AI.

  • Evaluation test plan
  • Metrics and test datasets
  • Quality and robustness findings
  • Safety and red-teaming results
  • Final evaluation report

A practical chatbot evaluation you can explain and showcase in interviews.

50 hours
Total training and coursework
9 Saturdays
Live online sessions
1 capstone
Complete chatbot evaluation
EvaliQA
Learn the platform we built
Your QA experience. Your next opportunity.

Built for Your Next Move in AI

You want to bring your testing skills into AI evaluation and build practical evidence of what you can do.

You're a manual QA professional ready to put your testing judgment to work on AI.

Turn your eye for risks, edge cases, and unexpected behavior into a practical approach to evaluating chatbots, RAG applications, and AI agents.

You're an automation engineer or SDET ready to evaluate AI behavior.

Expand your toolkit with evaluation datasets, custom metrics, LLM judges, trace analysis, and robustness testing.

You lead QA and want a clear approach to assessing AI quality.

Learn to define evaluation scope, select meaningful metrics, calibrate judgments, and explain findings that help your team assess product risks.

You're new to AI evaluation and want guided practice with a tool you can apply.

Build your foundation step by step and learn EvaliQA, the AI testing and evaluation platform we developed, through practical coursework.

You're pursuing an AI QA role and want work you can show hiring teams.

Complete a chatbot evaluation capstone, produce a report you can showcase, and practice presenting your skills through mock interviews with personalized feedback.

You're a QA professional who was laid off and you're ready to get back in the game.

Build on your experience with practical AI evaluation skills, a capstone you can showcase, and interview preparation to help you pursue your next opportunity.

Your journey into AI evaluation

Build skills. Prove your ability.
Take your next career step.

A path from your existing QA experience to practical AI evaluation, career preparation, and an optional project practicum.

  1. 01

    Build practical AI testing and evaluation skills

    Learn to evaluate chatbots, RAG, and AI agents. Choose meaningful metrics, build datasets, analyze results, and assess robustness, bias, and toxicity.

  2. 02

    Demonstrate your skills through a capstone project

    Complete a chatbot evaluation from test planning through reporting. Produce a report that demonstrates your approach, evidence, and testing judgment.

  3. 03

    Strengthen your job search and interview confidence

    Connect your QA background with your AI skills in your résumé and LinkedIn profile. Practice through mock interviews with personalized feedback.

  4. 04Optional add-on · $999

    Build experience through the Eight-Week AI Evaluation Project Practicum

    Extend your learning through practical AI evaluation projects. Apply course methods, deepen your practice, and build examples to discuss with employers.

05Your career goal

Get hired in AI testing and evaluation

Bring your QA experience, evaluation skills, and capstone report into your applications and interviews. Show hiring teams how you identify AI failures, assess quality, and communicate findings. Add practicum project experience if you choose the optional program.

Built by QA professionals. Built for your next step.

The QA Mentor University Advantage

Build the judgment to evaluate AI and the evidence to back it up. Learn practical methods and our own evaluation platform, then put those skills to work on your AI projects.

EvaliQA
Built by QA Mentor

Learn EvaliQA from the team that built it.

We developed EvaliQA, our own AI testing and evaluation platform. Learn to build datasets, configure metrics, run evaluations, and analyze results. Take that practical workflow straight into your own AI projects.

A tool you learn here. Skills you can apply immediately.

Know when to trust an evaluation score.

Select metrics, create custom criteria, and calibrate automated judgments against human assessments. Explain why a result deserves confidence.

Follow the evidence inside the AI workflow.

Read traces, interpret evaluation results, and examine non-functional behavior. Connect a failed response with evidence for investigation.

Find the failures easy test cases miss.

Build golden datasets with happy paths, edge cases, negative tests, and robustness scenarios. Make evaluations repeatable and track quality over time.

Adapt your approach to different AI products.

Explore evaluation for chatbots, RAG, agents, harnesses, voice, and images. Understand how each architecture changes your evaluation strategy.

Show employers how you evaluate AI.

Complete a 16-hour chatbot capstone covering planning, datasets, quality, robustness, and red teaming. Present your findings in a final report.

Turn new skills into a stronger interview.

Strengthen your résumé and LinkedIn profile. Explain your approach through mock interviews with personalized feedback and a focused job search strategy.

The January 2027 learning roadmap

Your AI Evaluation Skills,
Week by Week

Build your foundation, practice the evaluation process, and prepare to present your skills to hiring teams.

  • 9 live sessions
  • 50 total hours
  • 16-hour capstone
Week 1You know what reliable AI evaluation requires.

January 9, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

Introduction to AI Evaluation

  • Understand AI classifications and why AI systems need evaluation.
  • Recognize how probabilistic outputs change traditional testing approaches.
  • Identify evaluation challenges and criteria for reliable results.
Week 2You understand the AI system you are testing.

January 16, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

LLMs, Agents, and AI Architectures

  • Learn how generative models and LLMs are trained and how they work.
  • Understand RAG, single-turn and multi-turn agents, and voice agents.
  • Explore AI harnesses and their role in an AI system.
Week 3You choose metrics and calibrate AI judgments.

January 23, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

Metrics for AI Evaluation

  • Understand deterministic metrics, AI detectors, and LLM-as-a-judge approaches.
  • Choose metrics for RAG, agents, and chatbots, and define custom criteria.
  • Calibrate metrics and align automated judgments with human assessments.
  • Video · 1 hour: Getting started with EvaliQA and DeepEval.
  • Assignment · 1 hour: Prepare metrics for different AI solutions.
Week 4You build a test plan and evaluation datasets.

January 30, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

Test Planning and Dataset Management

  • Define evaluation scope and document your test plan before execution.
  • Build golden datasets, happy paths, edge cases, negative tests, and robustness scenarios.
  • Understand how evaluation-driven development guides iteration.
  • Video · 1 hour: Building a dataset for AI evaluation.
  • Assignment · 2 hours: Prepare happy-path, edge-case, and robustness datasets.
Week 5You run evaluations and investigate failures.

February 6, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

Running Evaluations and Analyzing Results

  • Prepare evaluation inputs and interpret results.
  • Read traces, understand OpenTelemetry, and examine non-functional behavior.
  • Understand evaluation pipelines and prepare an evidence-based report.
  • Videos · 2 hours: EvaliQA and DeepEval execution, trace analysis, and custom metric calibration.
  • Assignment · 3 hours: Run your evaluation, analyze results, and prepare a report.
Week 6You test AI for bias, toxicity, and safety.

February 13, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

Safety, Toxicity, and Bias

  • Understand bias and toxicity and how to evaluate them.
  • Recognize common leaks, vulnerabilities, and attack types.
  • Define a red-teaming strategy and run controlled attacks in an authorized test environment.
  • Videos · 2 hours: Bias and toxicity evaluation with EvaliQA and a practical red-teaming guide.
  • Assignment · 2 hours: Evaluate bias and toxicity and execute your red-teaming strategy.
Week 7You compare LLMs using meaningful evidence.

February 20, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

LLM Evaluation

  • Identify what can be evaluated at the model level and when it is useful.
  • Explore common benchmarks and perform A/B evaluations.
  • Learn to use Hugging Face and evaluation cards.
Week 8You adapt evaluation to agents, voice, and images.

February 27, 2027 · 10:00 AM to 12:00 PM EST · Live online · 2 hours

Evaluating AI Harnesses, Voice Agents, and Images

  • Evaluate AI teammates and personal agents.
  • Build golden datasets for harnesses and track quality over time.
  • Explore voice-agent considerations and image-evaluation approaches and metrics.
Week 9You present your AI skills to hiring teams.

March 6, 2027 · 10:00 AM to 1:00 PM EST · Live online · 3 hours

Interview Preparation and Job Search

  • Build a résumé that connects QA experience with AI evaluation skills.
  • Prepare for common interview questions and practice through mock interviews with personalized feedback.
  • Improve LinkedIn visibility, develop your job search strategy, and review optional practicum enrollment.
Capstone project · 16-hour assignment

Bring the evaluation process together.

Evaluate a chatbot using the provided documentation. Define your test plan and datasets, assess quality and robustness, run red-teaming scenarios, and present your evidence and findings in a final report.

Optional program add-on · Eight weeks

Eight-Week AI Evaluation Project Practicum

Continue building experience after the course. Apply course methods through practical AI evaluation projects, deepen your skills, and build examples of your work to discuss in interviews.

$999Optional program add-onAvailable separately from the 50-hour course
Your next step starts in January

Our Next Program

January 2027 Program

Practical AI Testing and Evaluation for QA Professionals

  • Start dateJanuary 9, 2027
  • End dateMarch 6, 2027
  • Course commitment9 live Saturday sessions · 50 total hours
  • FormatLive online instruction, videos, and assignments
  • Capstone16-hour chatbot evaluation project
Early registration tuition · USD
$2,999$3,999
Save $1,000

Pay by January 1, 2027 to receive the discount.

Optional add-onEight-Week AI Evaluation Project Practicum
$999
Apply for the January Program

Course Schedule

January to March 2027 · Saturdays

All times EST

19 hours of live instruction. Videos and assignments complete the 50-hour course.

Reserve your seat

Apply for the January 2027 Program

Leave your details and our team will contact you to talk through the program, tuition, and your goals. Early registration tuition is $2,999 when you pay by January 1, 2027.

  • Starts January 9, 2027, Saturdays at 10:00 AM EST
  • 9 live sessions, 50 total hours
  • 16-hour chatbot capstone and interview preparation

We use these details only to contact you about the program.

By applying you agree to our Privacy Policy.

Your questions, answered

Frequently Asked Questions

Get a clear picture of the training, practical work, schedule, and tuition before choosing your next step.

What makes this program different?

You learn AI evaluation through a QA lens, from test planning and dataset design to metric calibration, trace analysis, safety testing, and reporting.

You also learn EvaliQA, the AI testing and evaluation platform developed by QA Mentor, and complete a chatbot evaluation capstone you can showcase to hiring teams.

Who is this program designed for?

The program is designed for manual testers, automation engineers, SDETs, QA leaders, and QA professionals moving into AI evaluation. It also supports experienced testers returning to the job market after a layoff.

Do I need previous AI evaluation experience?

The course starts with AI evaluation foundations and how LLMs, RAG applications, and agents work. You build your understanding progressively through live sessions, supplementary videos, and practical assignments.

No previous AI evaluation experience is required. Familiarity with QA concepts will help you connect the material to your existing experience.

How long is the program, and when are classes?

The January program runs from January 9 to March 6, 2027, with nine live online sessions on Saturdays starting at 10:00 AM EST.

Live sessions add up to 19 hours. Videos and assignments complete the 50-hour course.

What is the optional eight-week project practicum?

The Eight-Week AI Evaluation Project Practicum is an optional $999 add-on for continued practical learning after the course.

Apply course methods through AI evaluation projects, deepen your skills, and build examples of your work to discuss in interviews.

Will the program help me prepare for interviews?

Yes. The final three-hour live session covers résumé preparation, LinkedIn visibility, job search strategies, common AI evaluation interview questions, and mock interviews with personalized feedback.

Your capstone report also gives you concrete work to explain during interviews.

Is a job guaranteed after completion?

Employment is not guaranteed. The program develops your practical AI evaluation skills and provides career preparation to support your job search.

You can present your QA experience, capstone findings, and evaluation approach to potential employers.

What do I need to participate?

You will need a computer, reliable internet access, and access to the tools used for the practical exercises.

Confirm detailed setup instructions and tool-access arrangements with QA Mentor before the program begins.