This section discusses the impossibility of complete testing and introduces test estimation.

• Complete structural coverage doesn't mean complete testing.

• To achieve complete testing, all distinct tests would have to be run.

• Testing all individual variables, combinations of values, and hardware/software configurations would require a lot of tests.

• The usual approach is to test variables with a few values, including normal operation and handling of values that are too big, too small, or strange.

00:01

Doug Hoffman illustrated the incompleteness of testing by running all four billion inputs into a function on a super fast computer, and it only took six minutes.

• Experienced testers may use domain testing techniques, but it does not provide a complete set of tests.

• Most people would test the smallest and largest numbers, powers of 2, and some random or favorite numbers.

• Hoffman decided to run all four billion inputs into a function on a masspar computer, and it took only six minutes.

• Out of four billion tests, two tests failed.

02:09

Finding bugs in software cannot be done through exhaustive testing due to the enormous number of possible tests.

• Exhaustive testing requires an impractical amount of time and resources.

• Black box testing has limitations as it cannot consider all possible inputs and scenarios.

• Input problems, such as editing while typing or timing out, can lead to bugs.

• Result variables, like overflow in multiplication, also pose risks in testing.

04:19

Underflows, easter eggs, and invalid inputs can cause failures in programs, but testing for all possibilities is impossible.

• Underflows can cause failures when no data is entered in a data entry dialog.

• Easter eggs are hidden surprises in programs that can be triggered by typing a specific sequence of characters.

• Testing for easter eggs at the user interface is challenging due to the vast number of possible keystroke sequences.

• Clearly invalid inputs, such as entering a zero into a data entry field, can have unexpected consequences.

• It is impossible to test for all the unexpected inputs that users might provide.

06:31