Master LLM evaluation and testing: LLM-as-a-judge, golden datasets, RAG and agent evals, CI/CD regression testing and production monitoring. 2026 course.
The course’s skills. The same ones appear on your Certificate of Completion, so you can show what you covered.
A premium, complete and advanced course on evaluating and testing LLM-powered systems, updated for 2026. You will learn why evals are the hardest and most important discipline in AI engineering, the full evaluation taxonomy (offline vs online, reference-based vs reference-free), why classic metrics such as BLEU, ROUGE and exact match fail on open-ended text, and how embedding-based metrics help and mislead. You will master LLM-as-a-judge (rubric design, position/verbosity/self-preference bias, calibration against humans), building golden datasets with edge cases and slices, and the real 2026 toolchain: promptfoo, LangSmith, Langfuse, Braintrust, DeepEval, Ragas and pytest. The course goes deep on evaluating RAG (retrieval and generation), evaluating agents (task completion and trajectory), safety and guardrail evals, red-teaming, regression testing prompts in CI/CD, online A/B testing, quality-drift monitoring, human annotation and the real cost of evaluation. Includes runnable code, production patterns, a strong focus on data privacy and GDPR for eval data, and a comprehensive final assessment. Educational content, not legal advice.
For developers, engineers and technical professionals who build or integrate AI solutions. It belongs to AI Engineering, on the IT track.
Advanced level: assumes hands-on experience with AI in the course’s field.
10 modules · 30 lessons · about 25 hours · 370 quiz questions
Modules and lessons in order, plus the link to the preview of lesson 1 — so you can decide at your own pace.
You read the lesson, ask the AI Professor right on the page and check what you remember.
Why LLM Evaluation Is Hard — and Non-Negotiable
Most engineering teams can ship a web service they can reason about: the same input yields the same output, a failing assertion is unambiguous, and a green test suite means the behavior is locked in. LLM-powered systems break every one of those assumptions.
Written lessons, with examples and exercises. Your progress is saved automatically, so you can pick up where you left off.
Select a passage and it explains it, gives you an example or tells you why it matters. You can type or talk to it, in English.
Every lesson has its own quiz, with explanations for the answers: 370 questions across the course. The professor also summarizes the lesson and prepares review flashcards.
Once you complete the lessons and reach the minimum quiz score, you issue it from your account, with a unique number, publicly verifiable.
The AI Professor is included in your subscription, per-course access included, with a monthly fair-use limit. It is an AI system, not a human teacher, and its answers may contain errors; details in the Terms. See how the AI Professor works.
The Certificate of Completion for “LLM Evaluation and Testing: Shipping Reliable AI”: a private attestation with a unique number and a QR code. A recruiter confirms it is authentic on the public verification page, with no account and no cost.
Public verification page, QR code and cryptographic fingerprint, with no account for the person checking.
Add it to your CV and LinkedIn profile; a recruiter confirms authenticity with one click.
The course, the tier you earned (Pass, Merit or Distinction), the serial number and the issue date, plus the percentage of lessons completed.
A private attestation of course completion. It is NOT a diploma or a state-recognised qualification under Romanian Laws 198/2023, 199/2023 or Government Ordinance 129/2000; employers may take it into account at their own discretion. Certification policy.
A subscription by number of courses: 1, 3, 5, 10, 15 or 25 courses a month, or the full catalog. You pick them and can change them at every renewal; your progress and certificates stay. Billed monthly or yearly (12 months for the price of 10), in EUR. You can stop the renewal anytime from your account, and access starts right after payment. Prices are totals, with VAT included at the rate of your country.
This course goes first on your list; you pick the others in your account and can change them at every renewal.
€99 a course
€99a month €990a year, 12 months of access
under €67 a course
€199a month €1,990a year, 12 months of access
under €52 a course
€259a month €2,590a year, 12 months of access
under €32 a course
€319a month €3,190a year, 12 months of access
under €24 a course
€359a month €3,590a year, 12 months of access
under €16 a course
€399a month €3,990a year, 12 months of access
under €10 a course, across all 50
€499a month €4,990a year, 12 months of access
Total prices, VAT included. Move up anytime and pay only the difference for the days left; move down at renewal. Renews automatically, you stop it anytime from your account; your progress and certificates stay. Digital content with immediate access: see the withdrawal terms.
About the course, the price, the AI Professor and the certificate.
A premium, complete and advanced course on evaluating and testing LLM-powered systems, updated for 2026. You will learn why evals are the hardest and most important discipline in AI engineering, the full evaluation taxonomy (offline vs online, reference-based vs reference-free), why classic metrics such as BLEU, ROUGE and exact match fail on open-ended text, and how embedding-based metrics help and mislead. You will master LLM-as-a-judge (rubric design, position/verbosity/self-preference bias, calibration against humans), building golden datasets with edge cases and slices, and the real 2026 toolchain: promptfoo, LangSmith, Langfuse, Braintrust, DeepEval, Ragas and pytest. The course goes deep on evaluating RAG (retrieval and generation), evaluating agents (task completion and trajectory), safety and guardrail evals, red-teaming, regression testing prompts in CI/CD, online A/B testing, quality-drift monitoring, human annotation and the real cost of evaluation. Includes runnable code, production patterns, a strong focus on data privacy and GDPR for eval data, and a comprehensive final assessment. Educational content, not legal advice.
The course has 10 modules and 30 lessons, with 370 quiz questions. The full curriculum, module by module, is shown on the course page.
Yes. The course is Advanced level and assumes hands-on experience with AI in its field.
The estimated duration is about 25 hours, across 30 lessons. You learn at your own pace, and your progress is saved automatically at every lesson, so you can pick up where you left off.
This course alone: the 1 course plan, €99 per month. The course fits into every plan of the subscription, next to other courses you choose: for example 3 courses for €199, 10 courses for €319 (recommended) or the full catalog for €499 per month.
You pick your courses after paying and can change them at every renewal; starting from here, this one is already picked. Plans are billed in EUR, monthly or yearly (pay for 10 months, get 12). Every price is a total, with VAT included at the rate of your country. See all plans.
Yes. You can read a free preview of lesson 1, “Why LLM Evaluation Is Hard — and Non-Negotiable”, without an account. The preview shows you how the lesson is written; the rest of the lesson and the other lessons open after payment.
The AI Professor is built into every lesson and starts from the lesson you have open: it explains the passages you select, gives you examples, summarizes the lesson, tests you with a quiz and prepares review flashcards. You can type or talk to it, in English.
It is included in every plan of your subscription, with a fair-use limit. It is an AI system, not a human teacher, and its answers may contain errors; details in the Terms.
When you meet the completion thresholds shown on the platform (lessons completed and the minimum quiz score), you can issue your Certificate of Completion for “LLM Evaluation and Testing: Shipping Reliable AI” from your account, with a unique number, publicly verifiable on the site.
It is a private attestation of course completion, issued by GALIAN SOFTWARE DEVELOPMENT SRL. It is NOT a diploma or a state-recognised qualification under Romanian Laws 198/2023, 199/2023 or Government Ordinance 129/2000; employers may take it into account at their own discretion. Details in the Certification policy.
Yes. The subscription renews automatically at the end of each monthly billing period. You can see the next renewal date in your account at any time.
You can stop the renewal at any time, from your account or by email, with no other formality. Access stays active until the end of the period already paid for, and no pro-rata refund is made for a period already started, except in the cases provided by law. Your progress stays saved in your account. Details in the Terms & Conditions, section 5.5.
Yes. With the team library, the company admin picks 3, 5, 10, 15 or 25 courses a month, or the full catalog, for the whole team, from €49 per member per month, excl. VAT, for 5 members or more. A dashboard shows each member’s progress and manages seats centrally. Price it for your team.
You can pick them together with this one in a 3, 5 or 10-course subscription (3 courses for €199 a month), and the full catalog includes them all. A single course is €99 a month.
IT: AI Engineering
Build MCP servers in Python and TypeScript: protocol architecture, OAuth 2.1, security, Docker and Kubernetes, CI/CD to production. Hands-on course for 2026.
IT: AI Engineering
Engineer context and memory for AI agents: context anatomy, memory layers, retrieval, compaction, multi-agent systems and cost optimization. For 2026.
IT: AI Engineering
Master RAG end to end: embeddings, chunking, vector databases, hybrid search, re-ranking, Agentic RAG and production scaling. Hands-on course for 2026.
€99/monthVAT included
Start the course RAG: Retrieval-Augmented Generation in Practice, 1-course subscription“LLM Evaluation and Testing: Shipping Reliable AI”: 30 lessons and about 25 hours of content, with your progress saved at every lesson.
1-course subscription
€99/month
VAT included · stop the renewal anytime from your account
Start the course · €99/month Or 3 courses for €199/month, this one includedChatGPT, OpenAI, Claude, Anthropic, Gemini, Google, Microsoft 365 Copilot, Cursor, Midjourney and the other product and company names mentioned are trademarks or trade names of their respective owners. Cursuri-AI.ro is an independent training provider and is not affiliated with, sponsored by or endorsed by any of them.