Testing Algorithms, LLC.
  • Home
  • About Us
  • Solutions
  • Case Study
    • Job Portal
  • FAQ
  • Blog
  • Video
  • Tutorial
  • Contact Us

How the quality of test execution should be measured?

4/23/2016

0 Comments

 
Picture


As a test manager, I always wanted to assess at the end of a testing project whether the quality of testing was good enough to ensure quality of the product. Literature of software testing suggests hundreds of metrics that can be used by the managers. However, not all of them are intended to measure the performance of the testing team.

Various test metrics can be divided into three primary categories: Product metrics (intended for Application Managers), Project metrics (intended for Project Managers) and Process metrics (intended for Test Managers). Here, our primary interest is Process metrics, which specifically measures the quality of testing, or in other words, the performance of the testers. Let's see which serves this purpose.

Is it total number of defects found?

No. This is neither of Product, Project or Process metric. This is because it doesn't tell you a story. Let's take an example where the testers found 200 defects. We can't infer anything from this number. The reason is, we don't know whether the testers did a good job or a bad job because we don't know how many defects are still unidentified.

Is it Test Execution Productivity?

This Process metric do measure the tester's performance, but in terms of speed, not quality. So, if a tester executes test cases in lightning speed but with errors, then it doesn't serve the purpose of testing in the first place.

Then what are the metrics that assesses quality of test execution? Well, in order to assess this, we should answer two following questions:

(A) Did the testers identify all the valid defects?
(B) Did the testers spend too much time and effort on invalid defects?

And, these two questions can be answered very easily with the following two metrics:

(A) Defect Leakage (to the next upper environment) = Total number of defects identified in next upper environment / Total number of defects identified (in lower + next upper environments)
(B) Defect Rejection Ratio = Total number of defects rejected / Total number of defects (valid + rejected)
​

Note that both the above metrics are "lesser the better". This is how I was measuring the quality of test execution for a long time. Request the readers to share thoughts on this as well.

0 Comments



Leave a Reply.

    RSS Feed

    Author

    Abhimanyu Gupta is the co-founder & President of Testing Algorithms. His areas of interest are innovating new algorithms and processes to make software testing more effective & efficient.

    Archives

    April 2017
    March 2017
    January 2017
    December 2016
    November 2016
    October 2016
    August 2016
    July 2016
    June 2016
    May 2016
    April 2016
    March 2016

    Categories

    All
    Agile Testing
    Analytics In Software Testing
    Automated Test Case Design
    Business Model
    Defect Prediction
    Model Based Testing
    Outsourcing
    Quality Assurance
    Requirement Analysis
    Requirement Traceability
    Return Gift
    Status Report
    Test Approach
    Test Automation
    Test Coverage
    Test Efficiency
    Testing Algorithms
    Testing Survey
    Test Management
    Test Metrics
    Test Strategy
    Training
    Types Of Testing
    User Story

    View my profile on LinkedIn
© 2015 - 2018 Testing Algorithms, LLC.
​All rights reserved.
​
support@testingalgorithms.com
  • Home
  • About Us
  • Solutions
  • Case Study
    • Job Portal
  • FAQ
  • Blog
  • Video
  • Tutorial
  • Contact Us