Versions Compared

Old Version 6

changes.mady.by.user Ioana Kocurova

Saved on

compared with

New Version Current

changes.mady.by.user Ioana Kocurova

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This lesson shows how hard it can be to find some complex bugs. It also mentions different activities which should be done by testers and why it is not possible to do everything because our time is always limited.

Learning Objectives

Understanding testing limitations to uncover all bugs, as shown by a real-world example of a phone system bug that went unnoticed during testing. How to spot certain long sequence bugs which can only be uncovered through extensive testing. What trade-offs between various testing tasks should be made as the infinite number of possible tests cannot all be conducted.

Core Topics and Takeaways

  • Irreproducible bugs

  • Activities testers should do

  • Tradeoffs between activities we need to do

Widget Connector
overlayyoutube
_templatecom/atlassian/confluence/extra/widgetconnector/templates/youtube.vm
width400px
urlhttps://www.youtube.com/watch?v=cYxySProFHA
height300px

Video Highlights

Topic

Key Concepts

Video Location

Only a small percentage of tests can expose bugs, and extreme cases are often dismissed as unrealistic.

  • Testing reveals the limitations of our imagination in finding bugs.

00:01

The video discusses a bug in the code that caused call records to remain on the stack even after the caller hung up.

  • In the early 1980s, customer service centers had a high rate of callers being taken off hold within two minutes.

  • The code didn't handle the case when the caller hung up, resulting in call records remaining on the stack.

  • Originally, the code checked the stack every time something was added or removed, but it was removed later due to system time constraints.

  • To prevent stack problems, a stack reset command was added in places where the stack was supposed to be empty.

02:23

Whole forwarding is a process where held calls are transferred to other phones, causing a rotating outage in the system.

  • Whole forwarding transfers held calls to other phones.

  • In a rotating outage, error handling triggers failures in subsystems one after another.

  • The outage clears when call pace slows down, but during busy times it can last for hours.

  • The bug causing the rotating outage could have prevented potential customers from getting help under life-critical circumstances.

04:46

Long sequence bugs are difficult to find and are not easily detected with coverage or error-oriented testing.

  • Long sequence bugs only manifest after a long series of events.

  • Testers are expected to perform various tasks in addition to testing, but the time required for testing is infinite, leaving little time for other activities.

  • The decision to fully document tests is complex because it takes time away from other important tasks.

  • Statements that testers should always do certain tasks are unreasonable because every task is subject to trade-offs.

  • Hiring more testers does not solve the problem of an infinitely large set of testing tasks.

07:10

Learning Highlights

In long sequence testing, even a small percentage of the vast number of possible tests can expose critical bugs. An example from a telephone system illustrates how a seemingly extreme case, repeating an action ten times without passing through a specific point, led to a real-world bug. Despite the belief that no one would conduct such a test, the bug occurred, causing a rotating outage. The example set in this section emphasizes the difficulty of anticipating all potential scenarios in testing and the challenges of making trade-offs among various testing tasks when facing an infinite set of testing possibilities.