How do you know if you have a critical bug?
There is no tester in the world who hasn’t bothered with this question ever once in a while. Is the bug I found critical? Most of the bug reporting systems qualify priority or severity of the bugs. Some even depict priority in terms of severity (minor, major, critical, blocker), some leave out one in favor of the other. So is there a difference?
In the old days we could qualify bugs in different scales – severity, impact, visibility, class and priority. This practice seems to be abandoned in favor of easier project management, so there is frequently only one field. However, making only one field available for depicting bug importance hasn’t helped much in qualifying bugs critical or making decision on how soon the fix is needed.
Even if “priority” is predominant field in bug trackers (eg, “severity” is removed to make things simpler), tester nor test lead cannot determine priority of a bug. The only person who can prioritize things is the project manager. Project manager actually often strongly disagrees with testers about the importance of specific bugs. This is acceptable, since the tester does not make decisions about the project or the product, and is only providing information to stakeholders. So we should get used to it – severe bug can get lower priority and trivial bugs can get higher priority. But why is that?
Testers role is not to prioritize bug solving, only to determine the severity. Bug severity has an impact on the perceived quality of a product.Â So we can have minor, major, critical… bugs. Also, besides impact of the bug to perceived quality of a product, we also try to determine how it is likely that average user will encounter the bug.
Let’s say we are testing music player and we find a bug which makes the buttons on our application a bit blurry. The music plays fine, however typical user can see the blurry button. Maybe he will not notice it, because they want to play music and not look at the buttons. So we decide to mark this bug as minor in severity. We’ve also found a bug which makes application crash if the user wants to edit the ID3 tag. This feature is not very popular, but app crashes so we make it major.
Let’s say our player is about to be reviewed by the major music label. They probably wouldn’t like blurry buttons, even if the music plays fine. Project manager is the one who must anticipate this. Our minor bug became priority #1 for fixing. Project manager also knows that the next version of the player will drop the tag editor, so our major bug gets low priority. Were we wrong in our judgement?
The one thing we need to know is that even if our bug tracker has onlyÂ “priority” field, tester is not the one who should prioritize. We provide information about the quality of the product. We asses the impact which bug has to product quality and how it is likely the user will experience it. So, how do we know if we’ve found a critical bug? The answer is not in the box. Bear in mind that critical severity does not necessarily means bug is critical for fixing.