Categorizing Defects by Severity

Introduction

What is Defect Severity?

Defect Severity or Bug Severity?

Some tips on finding the severity

Defect Classification by severity

Introduction

Is Testing an Art or Science? We have different answers for the above question, but certainly Testing is an Art while

determining the severity of a defect found in a system. The classification of impact of Defect is important for following

reasons:

It helps to determine the efficiency of Test Process.

It helps to decide the priority of the defect, hence improves overall development process by fixing higher priority

defects first.

The bug tracking process can be made more effective if the severity of the defect is clearly defined.

The focus of this paper is providing some ideas on Defect Severity and its classifications.

What is Defect Severity?

A defect is a product anomaly or flaw, which is variance from desired product specification. The classification of defect

based on its impact on operation of product is called Defect Severity.

Defect Severity or Bug Severity?

A bug is matured term of defect. A defect usually refers to as bug only if it affects operation of system and negatively

impacts the user of the system, while defect itself may not have any impact on operation of system. In other terms, all bugs

are defects but not all defects are bugs. Since severity classification also includes those anomalies, which doesn’t have any

impact on operation of system (like cosmetic errors etc.), it is appropriate to mention as Defect Severity rather than Bug

Severity.

Answer yourself the following before determining the severity

Following questions allows you to decide yourself the measure of severity.

• Does the system allow me to work even after defect occurs?


• Does the system recover from the defect by any means?


• If the defect is recoverable, does the system can do this on its own or any external effort is needed to recover from the
  
  defect?


• Did I check whether the same defect is reflected in all other related sections (or entire system)?


• Can I be able to repeat the defect in some other system having same configuration (O/S, Browsers etc.) as that of the
  
  system
• where I found the defect?


• Can I be able to repeat the defect in other configurations also?


• Does the defect affect only particular category of users or all?


• How frequently the defect occurs?


• Which inputs make the defect?

The severity level increases if the answer for some of the above question is ‘Yes’ and for some others ‘No’. For example,

if the answer for question 1 is ‘Yes’, then further testing of the system is not possible and hence severity is high. Also

the defect should be generalized as far as possible. i.e. after you find the defect, try to find out that the defect is

repeated in all cross-browsers, cross-O/S etc.

Some tips on finding the severity

Decide the impact

Some defects are obvious to decide its severity. For example, “HTTP error occurs when navigating to particular screen”.

Sometimes, a minor defect repeats in all sections or the frequency of such defect is more. In such cases, impact of the

defect is more in users perspective even though it is minor defect. Hence such defects get higher severity.

Isolate the defect

Isolating the defect helps to find out its depth of impact.

Analyze the defect with what class of inputs does the defect supports.

Make sure that the defect occurs only with particular sequence of operation or list out other sequences, which cause the

defect.

Defect Classification by severity

The impact of Defect Severity can be classified into four categories:

• Fatal
  
• Major
  
• Minor
  
• Cosmetic
  

Fatal Defects are the defects, which results in the failure of the complete software system, of a subsystem, or of a

software unit so that no work or testing can be carried out after the occurrence of the defect.

Major Defects are one, which also causes failure of entire or part of system, but there are some processing alternatives,

which allows further operation of the system.

Minor Defects does not result in failure but causes the system to produce incorrect, incomplete, or inconsistent results,

or the defect impairs the system usability.

Cosmetic Defects are small errors that do not prevent or hinder functionality.

Following are examples of type of Defects, which falls under each category.

Fatal Defects

1. Functionality does not permit for further testing.
   
2. Runtime Errors like JavaScript errors etc.
   
3. Functionality Missed out / Incorrect Implementation (Major Deviation from Requirements).
   
4. Performance Issues (If specified by Client).
   
5. Browser incompatibility and Operating systems incompatibility issues depending on the impact of error.
   
6. Dead Links.
   
7. Recursive Loop.
   

Major Defects

1. Functionality incorrectly implemented (Minor Deviation from Requirements).
   
2. Performance Issues (If not specified by Client).
   
3. Mandatory Validations for Mandatory Fields.
   
4. Images, Graphics missing which hinders functionality.
   
5. Front End / Home Page Alignment issues.
   

Minor Defects

1. Screen Layout Issues
   
2. Spelling Mistakes / Grammatical Mistakes.
   
3. Documentation Errors
   
4. Page Titles Missing.
   
5. Alt Text for Images.
   
6. Background Color for the Pages other than Home page.
   
7. Default Value missing for the fields required.
   
8. Cursor Set Focus and Tab Flow on the Page.
   
9. Images, Graphics missing, which does not, hinders functionality.
   

Cosmetic Defects

1. Suggestions
   
2. GUI image color etc.