The ITx 2016 Programme may change without notice

Machine learning and software testing

Tuesday 4:00pm - 4:30pm, TPN Testing Day (TPN Testing Room)

Are not machine learning and software testing diametrically opposed?

On one hand, machine learning involves creating programs can change through new experiences.
On the other hand, test cases should be idempotent: running the same test multiple times should not affect the outcome.

If machine learning is used in automated tests perhaps test steps which should fail will actually pass because the test case manoeuvres around what is in fact a defect. Or perhaps this kind of belief is why a field which has enjoyed such incredible growth over the past few years, machine learning, has not touched software testing in a significant way.
This does not need to be the case, indeed cannot be the case going if automated testing is going to deliver on its promise to enable true DevOps.

To give timely and accurate feedback to the organisation a test suite needs to minimise both Type I and Type II errors: both false positives and false negatives. At present, far more frequent in the execution of an automated suite are Type II errors, false negatives: test cases failing because of an unexpected pop-up, because a control has moved or changed its ID, because the system is running slowly, or because caps lock was left on… in short, because a means-to-an-end step failed, not a verification step. The analysis and fixing of these steps are time-consuming and costly.

Let us explore the possibilities of machine learning being applied to certain aspects of test automation, from fuzzy matching of controls and recovery steps, to perhaps test data and test case design.

Speaker