Testing AI is not testing software

An AI system can produce an answer that is reasonable without it being uniquely correct. However, how do you test something when you cannot always say exactly what the right answer is?

Conventional software testing remains essential, but software testing alone may not be sufficient. As AI behaviour is also determined by data and models, and correctness can become a spectrum rather than a simple pass or fail.

So, how can testers understand what constitutes sufficient evidence to trust what an AI system might do next?

Join Bill Aitken for our webinar as he explores how AI changes the tester’s problem, from testing the model’s behavioural boundaries to establishing trustworthiness under uncertainty.

The webinar will also examine why the lifecycle cannot end neatly at release, with monitoring, reassessment and retesting becoming important when reality changes.

What you’ll learn

• Why conventional software testing remains essential for AI systems, but is not enough on its own
• How testing changes when there is no single expected output to compare against
• Why data and models need to be considered alongside code when assessing AI behaviour
• How to think beyond simple pass or fail when performance can vary by context, conditions and consequence
• Why AI testing needs to explore the boundaries of what a model has learned, including rare, under-represented and novel situations
• Why monitoring, reassessment and retesting matter when real-world conditions change after deployment
• How the tester’s role expands to include questions around performance, failure, risk and when human intervention may be needed

Who is this webinar for?

This webinar is for software testing professionals who want to understand why familiar software-testing principles remain necessary, but are no longer sufficient for AI systems.

If you are familiar with established software-testing principles, this session will help you explore how those principles apply when AI behaviour cannot always be judged against a predetermined correct answer.

We look forward to seeing you there!