Defect Metrics:
Active Defects:
1. Active defects by severity and priority
2. Active defects by assignee
3. Active defects by group
3. Active defects by group, subgroup
All Defects:
1. Number of active, resolved and closed defects
2. Defects by severity and priority
3. Defects by assignee
4. Defects by group
5. Defects by group, subgroup
Test Case Metrics:
1. Number of test cases passed vs. failed vs. remaining to run
2. Number of test cases run vs. left to run
3. % of test cases passed vs. failed vs. remaining to run
4. % of test cases run vs. left to run
Bug Triage Meeting – Severity & Priority
"Triage" is a medical term. It refers to dividing wounded or sick people into three
categories: those who will die no matter what you do, those who will recover even if
unaided, and those who will recover only if aided. In a situation where there's too
much to do, you must concentrate on the third group.
Bug Triage Meetings (sometimes called Bug Councils) are project meetings in which
open bugs are divided into categories. The most important distinction is between
bugs that will not be fixed in this release and those that will be
There are three categories for the medical usage, software also three categories -
bugs to fix now, bugs to fix later, and bugs we'll never fix
Triaging a bug involves:
Making sure the bug has enough information for the developers and makes sense
Making sure the bug is filed in the correct place
Making sure the bug has sensible "Severity" and "Priority" fields
Let us see what Priority and Severity means
Priority is Business;
Severity is Technical
In Triages, team will give the Priority of the fix based on the business
perspective. They will check “How important is it to the business that we fix the
bug?” In most of the times high Severity bug is becomes high Priority bug, but it
is not always. There are some cases where high Severity bugs will be low Priority
and low Severity bugs will be high Priority.
In most of the projects I worked, if schedule drawn closer to the release, even if
the bug severity is more based on technical perspective, the Priority is given as low
because the functionality mentioned in the bug is not critical to business.
Priority and Severity gives the excellent metrics to identify overall health of the
Project. Severity is customer-focused while priority is business-focused.
Assigning Severity for a bug is straightforward. Using some general guidelines
about the project, testers will assign Severity but while assigning a priority is
much more juggling act. Severity of the bug is one of the factors for assigning
priority for a bug. Other considerations are might be how much time left for
schedule, possibly ‘who is available for fix’, how important is it to the business to
fix the bug, what is the impact of the bug, what are the probability of occurrence
and degree of side effects are to be considered.
Many organizations mandate that bugs of certain severity should be at least
certain priority. Example: Crashes must be P1; Data loss must be P1, etc. A severe
bug that crashes the system only once and not always reproducible will not be P1,
where as an error condition that results re-entry a portion of input for every user
will be P1
Microsoft uses a four-point scale to describe severity of bugs and three-point
scale for Priority of the bug. They are as follows
Severity
---------------
1. Bug causes system crash or data loss.
2. Bug causes major functionality or other severe problems; product crashes in
obscure cases.
3. Bug causes minor functionality problems, may affect "fit anf finish".
4. Bug contains typos, unclear wording or error messages in low visibility fields.
Priority
---------------
1. Must fix as soon as possible. Bug is blocking further progress in this area.
2. Should fix soon, before product release.
3. Fix if time; somewhat trivial. May be postponed
All about Bug or Issues
All about Bug, Error and Defects
Error: Mistake in code is error. Error is an undesirable deviation from
requirement
Bug: if defect accepted by the development team it is called bug. Bug is an
error found before the application goes into production
Defect: Defect is a variance from desired product attributes or normal flow.
When defect reaches to end customer is termed as failure. Defect is an error
found after the application goes into production.
Defects can be from product specs
Defects can be from customer or user expectation.
Attributes of Defect
1. Defect Naming – requirement defect, design defect, documentation
defect
2. Severity of Defect
· Critical (high) –show stopper
· Major (high) incorrect output derived
· Minor (Low) spelling mistake
3. Type of Defect
· Wrong- requirements implemented incorrectly
· Missing – requirement given by end customer but it were not
implemented at all
· Extra – requirement implemented correctly but were not
requested by end customer
About Severity and Priority
Severity: Seriousness of defect with respect to functionality
Priority: importance of defects with respect to customer
· High: without resolving defect we are unable to continue testing
· Medium: compulsory to solve but able to continue
· Low: may or may not be resolved
Examples:
· User interface bug- low severity
· Calculation bug- high severity and high priority bug
· Improper alignment – low priority and low severity bug
· Spell mistake – high priority and low severity bug
· Final output wrong – low priority
· Dependent output wrong – high priority
· Annual report print functionality not working – High severity and low
priority issue
Components of Bug
Title of bug
Severity of bug
Priority of bug
Bug type
Module name
Assigned to
Project manager
Steps to reproduce
Actual output
Expected output
Attachment of log and screen shot
Comments
Test environment – browser, OS information etc
Build date and platform
Guidelines to write bug
Be precise
Be clear – explain it so other can reproduce the bug
Mention exact steps to reproduce with details
Provide sufficient test environment details
Mention build date and number with details
When you close your bug mention on which build and date you
verify it, so in future if require to reopen it we can have track on which
build it works last
Report the problem immediately
Reproduce the bug at least one more time before post it
Test the same bug occurrence on the other similar module of the
application
Read bug report before submit it or send it
Never ever criticize any developer or any individual
Bug Impacts
Low impact
This is for Minor problems, such as failures at extreme boundary conditions that are unlikely to
occur in normal use, or minor errors in layout/formatting. These problems do not impact use of
the product in any substantive way.
Medium impact
This is a problem that a) Effects a more isolated piece of functionality. b) Occurs only at certain
boundary conditions. c) Has a workaround (where "don't do that" might be an acceptable answer
to the user). d) Occurs only at one or two customers. or e) Is very intermittent
High impact
This should be used for only serious problems, effecting many sites, with no workaround.
Frequent or reproducible crashes/core dumps/GPFs would fall in this category, as would major
functionality not working.
Urgent impact
This should be reserved for only the most catastrophic of problems. Data corruption, complete
inability to use the product at almost any site, etc. For released products, an urgent bug would
imply that shipping of the product should stop immediately, until the problem is resolved.
