Combination Testing

Owned by Mike Deskis
Last updated: May 03, 2024 by Ioana Kocurova
2 min read

This lesson is focussed on testing different combinations and how complex it can become.

Learning Objectives

The video discusses the impossibility of complete testing in software development due to the sheer number of variables and combinations that need to be tested, as well as the potential for errors that occur over time. Combination testing and testing for consequences are important techniques to optimize sampling and uncover potential bugs. The complexity and number of paths in a program make it impossible to test every scenario, highlighting the need for skilled testers to prioritize and strategize their testing efforts.

 

Core Topics and Takeaways

  • Configuration testing

  • Data testing

  • Path and sub-path testing

  • All pairs technique

Video Highlights

Topic

Key Concepts

Video Location

Topic

Key Concepts

Video Location

Even when testing one variable at a time, it is impossible to test everything that can be thrown at that variable, and testing variables together can result in unexpected errors.

  • Testing variables together is known as combination testing or configuration testing.

  • The number of tests required for combination testing increases exponentially as more variables are added.

  • Combination errors can be surprising and may seem unreasonable.

00:01

Combinations of data can lead to overflow or rounding errors, which can cause problems in the field.

  • Overflow example given with a complex travel plan on an airline ticket.

  • Combination testing techniques can optimize sampling to find combination-related errors.

  • All pairs sampling technique would have yielded 80 tests out of 1600 for printers with video cards with memory configurations.

  • Errors that occur in time and vocabulary related to program paths explained.

  • Flowcharts and flow graphs used to represent program paths.

02:28

The lecturer discusses the importance of testing for consequences and provides examples of testing sequences.

  • Testing for consequences is important when the program does something more complicated with a variable.

  • Testing for consequences is one of the things that distinguishes skilled testers from juniors.

04:59

The  impossibility of testing all possible paths in a program.

  • There are 25 possible paths in which the program exits the second time it hits X.

  • The number of paths increases exponentially as the loop is executed multiple times.

  • Testing all possible paths is not feasible, especially for complex programs.

  • Testing a limited number of paths may not uncover certain bugs, such as a memory leak.

07:27

Learning Highlights

When testing individual variables, complete testing is often impossible due to various combinations and interactions in a program. The example highlights the oversight of not testing certain combinations, leading to system crashes. Combination testing, such as configuration testing, addresses this by testing variables together. The lecture introduces the concept of set-use pairs and emphasizes testing for consequences, considering how a program uses variable values. The complexity of testing sequences is illustrated with a program involving loops, demonstrating the impracticality of testing all possible paths.