Kurowski Consulting - clarity over complexity

ISTQB® Certified Tester AI Testing (CT-AI) v2.0

ISTQB® Certified Tester AI Testing (CT-AI)

Learn how to test AI-powered systems with confidence, applying practical techniques to evaluate machine learning models, data quality, Generative AI and real-world AI risks.

Course Overview

The ISTQB® Certified Tester AI Testing (CT-AI) certification provides software testing professionals with the knowledge and practical skills required to test Artificial Intelligence (AI) and Machine Learning (ML) based systems effectively. The course introduces the foundations of AI, machine learning, neural networks, Generative AI, and specialised testing techniques designed to address the unique challenges and risks associated with AI-powered solutions. [ISTQB _CTA…lease_ALL_ | PDF]

Delegates will learn how AI systems differ from traditional software, how to evaluate the quality of AI-based systems, test machine learning models and datasets, identify bias and ethical risks, and apply specialised testing techniques such as adversarial testing, metamorphic testing, drift testing and red teaming. The course includes practical exercises to reinforce learning and prepare delegates for the internationally recognised ISTQB® CT-AI certification. [ISTQB _CTA…lease_ALL_ | PDF]

Who Should Attend?

  • Software Testers
  • Test Analysts
  • QA Engineers
  • Test Automation Engineers
  • Test Managers
  • Test Consultants
  • Data Analysts
  • Data Scientists
  • Software Developers
  • User Acceptance Testers
  • Business Analysts
  • Quality Managers
  • Project Managers
  • IT Directors
  • Anyone involved in the development, testing or deployment of AI-based systems
  • Professionals seeking an internationally recognised AI testing certification

What You Will Learn

Upon completion of this course, delegates will be able to:

  • Understand key AI, Machine Learning and Generative AI concepts
  • Differentiate between conventional and AI-based systems
  • Understand supervised, unsupervised and reinforcement learning approaches
  • Apply quality characteristics specific to AI-based systems
  • Understand AI safety, transparency, robustness and ethical considerations
  • Create and evaluate machine learning models
  • Understand neural network architecture and behaviour
  • Test AI-based systems using specialised AI testing techniques
  • Identify and mitigate risks associated with training data and machine learning models
  • Detect data bias, drift, overfitting and underfitting
  • Apply adversarial testing, metamorphic testing and red teaming approaches
  • Evaluate AI systems using machine learning performance metrics
  • Understand testing strategies for Generative AI and Large Language Models (LLMs)
  • Contribute to effective AI testing strategies and governance frameworks

Course Modules

Module 1: Introduction to Artificial Intelligence

Build a solid understanding of AI technologies and their role in modern software systems.

Topics include:

  • AI-based versus conventional software systems
  • Narrow AI, General AI and Super AI
  • Machine Learning fundamentals
  • Deep Learning concepts
  • Generative AI and Large Language Models
  • AI technologies and architectures
  • AI hardware considerations
  • Development and hosting options for AI systems
  • Machine Learning development frameworks
  • AI regulations and standards
  • Responsible AI principles

Module 2: Quality Characteristics for AI-Based Systems

Learn how software quality principles are adapted for AI-driven systems.

Topics include:

  • AI-specific quality characteristics
  • AI functional correctness
  • Functional adaptability
  • User controllability
  • Transparency and explainability
  • AI robustness
  • Intervenability
  • Ethical and societal risk mitigation
  • AI safety considerations
  • Acceptance criteria for AI systems
  • Regulatory requirements and quality standards

Module 3: Machine Learning Fundamentals

Understand how machine learning systems are designed, trained and evaluated.

Topics include:

  • Supervised learning
  • Unsupervised learning
  • Reinforcement learning
  • Machine Learning workflows
  • Data preparation activities
  • Training, validation and test datasets
  • Pretrained models
  • Fine-tuning techniques
  • Retrieval-Augmented Generation (RAG)
  • Neural networks and deep learning
  • Machine Learning performance metrics
  • Accuracy, Precision, Recall and F1-Score
  • Neural network coverage measures
  • Hands-on machine learning exercises

Module 4: Testing AI-Based Systems

Discover the techniques and challenges associated with testing AI-powered applications.

Topics include:

  • Testability of AI systems
  • Locked versus adaptive AI systems
  • Statistical approaches to testing AI
  • AI test oracle challenges
  • Testing Generative AI
  • Testing Large Language Models
  • Exploratory testing of AI systems
  • Red teaming techniques
  • Risk-based testing for AI
  • AI-specific test levels
  • Machine learning test strategies

Module 5: Input Data Testing for Machine Learning Systems

Learn how to validate and assess the quality of datasets used by machine learning systems.

Topics include:

  • Input data risks and mitigation strategies
  • Testing for bias
  • Data provenance testing
  • Data pipeline testing
  • Data representativeness testing
  • Dataset constraint testing
  • Label correctness testing
  • Multiple annotation approaches
  • Statistical analysis of datasets
  • Exploratory data analysis
  • Data quality validation
  • Hands-on data testing exercises

Module 6: Model Testing for Machine Learning Systems

Gain practical knowledge of testing machine learning models and mitigating AI-specific risks.

Topics include:

  • Machine learning model risks
  • Adversarial testing
  • Metamorphic testing
  • Drift testing
  • Data drift and concept drift
  • Overfitting and underfitting
  • A/B testing
  • Back-to-back testing
  • ML functional performance testing
  • Model documentation review
  • AI robustness testing
  • Model evaluation and validation
  • Hands-on testing exercises

Module 7: Machine Learning Development Testing

Understand quality assurance activities that support AI system deployment and operation.

Topics include:

  • Machine learning development risks
  • Framework suitability reviews
  • API testing
  • Security testing
  • Deployment testing
  • Installability testing
  • Rollback testing
  • Canary testing
  • Shadow testing
  • Model conversion testing
  • Cross-device testing
  • Production monitoring strategies

Examination

The course prepares delegates for the ISTQB® Certified Tester AI Testing (CT-AI) v2.0 examination.

  • Multiple-choice examination
  • Internationally recognised certification
  • Specialist-level ISTQB® qualification
  • Based on the official CT-AI v2.0 syllabus
  • Covers AI, Machine Learning, Generative AI and AI testing techniques
  • Includes both theoretical and practical AI testing concepts
  • Demonstrates specialist-level knowledge of testing AI-based systems
  • Recognised internationally by employers and organisations worldwide

Prerequisite: Delegates must hold the ISTQB® Certified Tester Foundation Level (CTFL) certification before taking the CT-AI examination.

Course Duration

3 Days Instructor-Led Training

Will be available as:

  • Classroom
  • Virtual Classroom
  • On-site Corporate Delivery
Register Interest