Measurement of Testing

Owned by Mike Deskis
Last updated: May 03, 2024 by Ioana Kocurova
2 min read

This lesson is about how to measure testing. You will find why just counting tests will not help us and what should we take into account when we need to say how much we tested and how much testing is left.

Learning Objectives

Understanding the attribute being measured and the purpose behind the measurement. Exploring concepts such as measurement error, precision, scale of measurement, and the validity of measurements.

 

Core Topics and Takeaways

  • Right measurement

  • Measured Attribute

  • Scale of measurement

  • What measurement should contain

  • Measurement validity

Video Highlights

Topic

Key Concepts

Video Location

Topic

Key Concepts

Video Location

Measurement in software testing.

  • Measurement is not about counting things but about understanding attributes.

  • A model is needed to turn numbers into answers for estimation.

  • Measurement error is a challenge in obtaining accurate measurements.

00:01

Understanding the purpose of a measurement is more important than precision and variation.

  • Precision and variation are technical issues in measurement.

  • Asking why you are taking a measurement helps you realize your underlying goal.

  • The sequence of discovering your underlying goal by asking "why" is called the five whys analysis.

  • The purpose of the measurement determines the level of precision needed.

02:20

The importance of scale in measurements and how it affects the interpretation of data.

  • Different scopes of interest include specific programs, teams of programmers, or companies.

  • The scale of measurement determines the meaning of a measurement, such as 40 inches on a tape measure.

  • Scale is also important in measuring sound levels, where decibels indicate the perceived loudness.

  • The interpretation of numbers may not always align with our expectations, as shown by the example of sound ratings.

04:40

Measurement can be represented in different scales: interval, ordinal, and nominal.

  • In an interval scale, the difference between two values is consistent.

  • In an ordinal scale, the order of values is known, but the magnitude of difference is unknown.

  • In a nominal scale, differences are based on different names, but no comparison of magnitude is possible.

  • The validity of a measurement depends on understanding the attribute being measured.

07:01

Learning Highlights

Counting elements, such as branches covered or bugs found, isn't sufficient for assessing the quality of testing. To convert numbers into meaningful answers, a model is needed. There are measurement challenges like error, precision, purpose clarification, and the scale of measurement. It is important to understand the attribute being measured to evaluate the validity of measurements, highlighting the need for a more sophisticated approach to measurement in software engineering.

Core Learning Concepts

Attribute

An attribute is a characteristic or quality that describes an object, entity, or phenomenon. In the context of software testing, an attribute could refer to any measurable aspect of a software system, such as performance, reliability, or usability.