Number of bugs as a metrics – useless?

The evidence against the number of bugs as a metric is massive. Not only does it not tell us anything about the remaining number of bugs in a system, or about the severity of the resolved bugs, but it also threatens the testers work.

If managers or the team are concerned about following the classical S-curve for bug finds during project development, testers will be concerned with fullfilling the model. The quest for an early peak might cause for superficial testing and a lack of incentive for finding the bugs that are more difficult to discover. As the project moves along, testers aren’t expected to find as many bugs anymore and it is now ok for them to do other activities or work with less effort.

The model holds because work is done in a way that  confirms it.

On the other hand, if the model is set aside, a tester who sees the number of bugs decreasing could take it as a sign that it is time to change methods. By applying other tools, other approaches, bringing in new people, focusing on different areas, and so on, the number of bugs found can go up again. A new decrease leads to another period of search for other methods.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s