Versions Compared

Old Version 1

changes.mady.by.user Mike Deskis

Saved on

compared with

New Version Current

changes.mady.by.user Ioana Kocurova

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this video bug counting is discussed

...

as well as its consequences. It also shows some ideas how important are also other activities on the project and what can happen if people know about which numbers are measured.

Lecture6bFoundations2010.mp4

Discussed terminology:

...

Surrogate measurement

...

Why do we count bugs and where it can lead

...

Predictable problems - like testers are focused just on activities connected with measurement

Learning Objective

This section aims to help you understand why the Weibull model for bug curves is often misapplied in testing due to its invalid assumptions; and what the pressure to meet milestones based on bug-find rates can lead testers to do.

Core Topics and Takeaways

  • Surrogate measurement

  • What bug counts can sometimes affect

  • Why bugs should not be always counted the same way - how serious bugs can be?

...

We are not testing just because of nice numbers of bugs

...

Widget Connector
overlayyoutube
_templatecom/atlassian/confluence/extra/widgetconnector/templates/youtube.vm
width400px
urlhttps://www.youtube.com/watch?v=HqWaiyT8VLc
height300px

Video Highlights

Topic

Key Concepts

Video Location

Surrogate measures are often used in software engineering to measure attributes that are difficult to directly measure, but they can lead to predictable problems.

  • Surrogate measures are used when there is a lack of understanding of how to measure an attribute.

  • Surrogate measures are often something that can be easily counted and is believed to be correlated with the attribute being measured.

  • Using surrogate measures can have negative impacts on collaboration within a group and the quality of work done.

  • The example of bug counting as a surrogate measure for tester productivity is given.

00:01

Measurement can lead to changes in behavior, and surrogate measures can be misleading.

  • Testers may change their testing methods to improve their statistics if tester value is measured with bug counts.

  • Surrogate measures, such as bug counts, can lead to incorrect assumptions about the status of a project.

  • Curves showing bug statistics only reflect what is being found by the test group, not the actual state of the product.

02:16

The Weibull model assumptions don't apply well to software testing.

  • The model assumes testing occurs in a similar way to how the software will be operated, which is not true.

  • The model assumes bugs are equally likely to be encountered, which is not true.

  • The model assumes defects are corrected instantaneously without introducing additional defects.

04:32

The pressure to meet bug curve expectations can lead testers to focus on finding cheap and easy bugs, rather than building tools or improving efficiency.

  • Executives put pressure on testers to generate bug curves that align with the model's expectations.

  • Testers may prioritize finding bugs early in testing to predict an early completion date.

  • Testers may spend less time on other important tasks like building tools or training new people.

  • Testers are pressured to keep the total count of open bugs below a certain threshold to meet milestones.

06:48

Learning Highlights

Bug counting, surrogate measurement often becoming a focal point in testing. The quest for numerical validation can lead testers to prioritize easily quantifiable aspects, potentially neglecting crucial facets of software evaluation. Predictable problems arise when testers fixate solely on measurable activities, redirecting attention from holistic testing approaches. Not all bugs should be tallied alike; seriousness varies. Relying solely on numbers can skew perception, emphasizing quantity over severity. Testing isn't just about accumulating bug statistics; it's about ensuring robust software. Bug counts can inadvertently influence resource allocation, overshadowing nuanced assessments.

Core Learning Concepts

The Weibull Model

The Weibull model is not well-suited for software testing because it is based on assumptions that do not align with the characteristics of software defects and testing processes. The Weibull model assumes that failures are independent and identically distributed (IID), which means that the probability of failure at any given time is constant and does not depend on previous failures. However, in software testing, the occurrence of defects is often influenced by factors such as code complexity, module interdependencies, and testing strategies, making the IID assumption invalid.

Additionally, the Weibull model is typically used in reliability engineering to model the lifetime of hardware components, where failures are caused by wear-out mechanisms. In contrast, software defects do not necessarily accumulate over time like hardware failures, as they can be introduced or removed at any stage of the software development lifecycle.

Therefore, applying the Weibull model to software testing can lead to inaccurate predictions and unreliable results, making it unsuitable for analyzing defect data and bug curves in software engineering contexts.