Bug Reporting

We all have seen many bug reports. Bug reports are written in a vast variety of styles. A bug report is used to communicate important problem data between testing, development, support and management groups. It is possible that the bug report created by the reader is read and acted upon in a different country and/ or in a different organization. All the currently open bug reports collectively contribute to indicate the present quality of the system.

When we uncover a defect, we need to inform the developers about it. Bug report is a medium of such communication. The primary aim of a bug report is to let the developers see the failure with their own eyes. If you can't be with them to make it fail in front of them, give them detailed instructions so that they can make it fail for themselves. The bug report is a document that explains the gap between the expected result and the actual result and detailing on how to reproduce the scenario.

Next, let us look at the various data elements that can exist in a bug report. You can use the fields available in the defect tracking system to convey the bug to the intended audience clearly.

1. Id:
Each bug should have a unique identifier. If a defect tracking system is being used, this Id is usually generated automatically. If not, you can assign a unique Id to the bug report. Since managers use bug Ids to track the bugs and the bug Ids are present in past reports, you should never alter the bug Ids.

2. Summary:
The summary is a very important part of the bug report. Ideally, the summary should capture the core of the problem in a few words. A summary can contain other supporting information e.g. project/ product version, build, component, platform/ environment, team name, name of the bug reporter etc. You should consider the norms of writing bug reports in your project/ organization and summarize appropriately. Make sure your summary is reflecting what the problem is and where it is.

3. Status:
When you are logging the bug in any bug tracking system then by default the bug status is "New".
Later on bug goes through various stages like Fixed, Verified, Reopen, Won’t Fix etc.

4. Project/ Product:
The bug reporter usually files the bug report against a single project or product in the system under test.

5. Component/ Module:
If the bug belongs to a particular part of the system, you should name that part of the system. On the other hand, if it were not possible to attribute the bug to a particular part of the system, you may not name the Component/ Module or may term the bug as a “system-wide” bug.

6. URL:
While testing web applications, it is common to state the URL that is required to retrace the steps and observe the problem.

7. Build:
The Build number is the build (major number and/ or minor number) in which you discovered the problem.

8. Type of bug:
The Type of bug can be a functional problem, a data problem, a performance problem, a problem with the build/ installer or a problem with the user interface.

9. Priority:
Priority based on how urgently a bug should to be fixed.It is based on functionality of the application.

10. Severity:
Severity decided by checking that how much Bug is impacting the functionality of the system. e.g. it can be showstopper or blocker or a high severity bug or a medium severity bug or a low severity bug.

10. OS/ Platform/ Environment:
Mention the hardware platform where you found this bug. The various platforms like PC, MAC, HP, Sun etc. If testing was done in different environments e.g. different operating systems (Windows XP, Vista, Linux etc.) or different browsers (IE, Firefox, Safari etc.), it is useful to specify the particular environment, in case the bug is observed in that environment only. For bugs that exist in all environments, you may mention the extent of the problem in the Description field.

Description: This is perhaps the most important part of the bug report. It contains the basic steps required to reproduce the problem. Sometimes, it is useful to mention both the expected results as well as the actual results encountered on execution of the steps. The actual results are the problem statement. In order to obtain support to get the bug fixed, it may be useful to state the impact of the actual results on the users. You may mention any data element that does not fit in the fields available in the defect tracking system in the Description part of the bug report under appropriate heading(s).

A detailed description of bug. Use following fields for description field:

- Reproduce steps: Clearly mention the steps to reproduce the bug.
- Expected result: How application should behave on above mentioned steps.
- Actual result: What is the actual result on running above steps i.e. the bug behavior.

12. Attachments:
A screen capture or a short video (consider the size of the attachment) showing the problem is usually quite effective in explaining the problem to the recipients of the bug report. This file should complement the Description of the bug report. If the file shows a number of items, encircling or otherwise highlighting the problem area in the file would save the recipients time to understand the problem. In addition to screen-shots/ videos, you may attach other items e.g. error log, a report, other documents to build support for bug fixing.

13. Reporter:
In many defect-tracking systems, this field is automatically populated. In case it is not, you should mention your name and contact information here.

14. Assigned To:
You could keep any incomplete (or “in progress”) bug report assigned to yourself. A completed bug report should be assigned to the correct person in development (a development manager/ lead or a developer). A fixed bug report should be assigned to the bug reporter so that it may be re-tested and marked as closed. In case the assignee requires information or a decision from another stakeholder e.g. management or marketing, the assignee may assign the bug report to the concerned person in the relevant group. Note that at any particular time, the bug report should stay assigned to exactly one person.

15. Cc:
Often, some persons are interested in the bug reports logged against the system under test. However, they may not play any direct role in fixing the bug. Such persons can include leads and management. These persons should stay informed about any changes to the bug report.

16. Dates:
A single bug report can include a number of dates. Common dates included in the bug report include Date Created, Date Assigned, Date Resolved, Date Reopened, and Date Closed. In case the date fields are not automatically maintained by the defect tracking system, care should be taken to log the correct dates. A number of testing metrics e.g. bugs opened/ closed, average age of an open bug and so on may depend on these dates.

17. Blocks/ Depends On:
It is possible that a particular bug has a dependency on another bug. If the reported bug blocks another bug from being tested, this can be mentioned in the bug report. Similarly, if the closure of another bug is a pre-condition to observe the reported problem, you can mention the Id of that bug.

Some Bonus tips to write a good bug report:

1. Report the problem immediately: If you found any bug while testing, do not wait to write detail bug report later. Instead write the bug report immediately. This will ensure a good and reproducible bug report. If you decide to write the bug report later on then chances are high to miss the important steps in your report.

2. Reproduce the bug three times before writing bug report: Your bug should be reproducible. Make sure your steps are robust enough to reproduce the bug without any ambiguity. If your bug is not reproducible every time you can still file a bug mentioning the periodic nature of the bug.

3. Test the same bug occurrence on other similar module: Sometimes developer use same code for different similar modules. So chances are high that bug in one module can occur in other similar modules as well. Even you can try to find more severe version of the bug you found.

4. Write a good bug summary: Bug summary will help developers to quickly analyze the bug nature. Poor quality report will unnecessarily increase the development and testing time. Communicate well through your bug report summary. Keep in mind bug summary is used as a reference to search the bug in bug inventory.

5. Read bug report before hitting Submit button: Read all sentences, wording, steps used in bug report. See if any sentence is creating ambiguity that can lead to misinterpretation. Misleading words or sentences should be avoided in order to have a clear bug report.

6. Do not use Abusive language: It’s nice that you did a good work and found a bug but do not use this credit for criticizing developer or to attack any individual.

Conclusion: No doubt that your bug report should be a high quality document. Focus on writing good bug reports, spend some time on this task because this is main communication point between tester, developer and manager. Mangers should make aware to their team that writing a good bug report is primary responsibility of any tester. Your efforts towards writing good bug report will not only save company resources but also create a good relationship between you and developers.

The bug reporter should carefully review the completed bug report. Usually, it is possible to improve the content of the bug report. You may use the following guidelines to review your bug report:

1. Do not be in a hurry to submit the bug report. You can come up with several ideas to improve the bug report if you give yourself a little time to think about the bug report.

2. Ensure that the bug report is within the scope of the current test. Another thing to ensure is that the bug report is against a completed feature/ function/ module of the system under test.

3. Ensure that the bug report is as compact as feasible.

4. The bug report should not directly or indirectly point to any person or any role in the project or organization. The bug report should only provide factual data about the observed problem.

5. Sometimes a few bugs are discovered in a single area of the system under test. It may be feasible to group some such bugs in a single bug report. This is another benefit of taking some between writing the bug reports and submitting them.

6. Ensure that the bug report is not a duplicate of any other bug report. It makes sense to search the defect tracking system using different searches to find any existing duplicates.

The self-review and revision is an iterative process. You should iterate this process until you are very satisfied with the bug report. Now, it is time to request a colleague or a senior to review the bug report and point out problems or improvement areas. You should then address their review comments in your bug report. Note that sometimes it makes sense to bunch a group of bug reports and ask for their collective review.

Nine Tips For an Effective Bug Reporting

It is important that a software tester reports everything that he feels. A software tester role is that of a catalyst in any team. He makes up the team on one hand and breaks up the application on the other. It is important to clearly understand all the issues in the application may be big or small with due course of understanding in terms of business and the application. Therefore a strong Bug Report serves as a definite proof in Software Development Life Cycle, provided the status of bugs is updated at all stages. Sole objective of your reporting a bug is that your intention is to get the bug fixed.

1. Clarity of Bug Description - Bug description refers to a small statement, which briefly points towards the exact problem. May be the problem requires a couple of steps to reproduce it, however this small statement of bug description must be able to communicate the exactly nature of problem. For instance in case of a problem concerning an error from the server, the bug description could clearly elaborate the meaning by stating that the server error takes place while performing such an such operation.

2. Don't Pass your Verdict - even if you are brimming with confidence about authenticity of the bug detected by you, avoid writing a bug report which would reflect as if you are trying to pass your verdict on the genuinity of the bug. In every probability this could initiate a controversy which would reflect your superiority complex as a tester. Your main aim should be to keep your bug report conclusive supporting your bug plus the sole motive must be to get the bug closed ultimately. Try to use diplomacy in bug report, instead of using authoritative statements in favor of your bug thereby making your bug report unpleasant, best way is to be suggestive. Such an approach shall always be taken in good spirit.

3. Steps to Reproduce - How to reach the precise point of bug, with due explanation of the set of conditions so as to reproduce it must be clearly defined in the bug report. For instance, for graphical software, tester needs to communicate to the developers, as to what he had done before getting the bug. The details must be elaborated like which buttons were pressed and in what order. For a program executed by keying in a command across the prompt, the details of the command typed before getting the bug needs to be indicated precisely.

4. Use of Simple Language - People do not like to read long paragraphs containing complex jargons & tongue twisters. A good bug report contains small bulleted like containing small but clear sentences. It should describe the observations relevant to the concerned bug only. Do not make the bug report unnecessarily complex and lengthy by writing too many facts. Avoid narrating superfluous details, which may not be of any help in reproducing the bug. Do not write things, which are commonly known, to everyone.

5. Quote-Relevant Examples - In majority of situations, we find that for reproducing a particular bug, some particular group of inputs is essential. Thus instead of making vague statements like feed invalid name of a person in the contact list, & save; it can be said that feed an invalid input like 035bbb@$% in the name field & click save. In order to get the bug fixed quickly tester must try to help the programmers by providing all the relevant information / to-the-point information.

6. Provide Back References - In case a particular bug happens to contradict the specification document or any other document related to the project; the bug report must provide adequate reference to the particular chapter or clause number of the concerned document being contradicted.

7. Allocate Bug Priority & Severity - Bug reporting is not complete without earmarking the level of Bug Severity & Bug Priority.

Bug Severity: refers to the quantum of danger as to how badly the bug can harm the system. It describes as to how bad the bug is. Severity is a feature of constant nature associated with the bug.

Bug Priority: refers to the need as to how urgently bug is required to be fixed. It describes the importance of the bug. Bug priority may change according to the schedule of testing.

Three levels of Bug Severity:

Severity level - Critical: is the most dangerous level, which does not permit continuance of the testing effort beyond a particular point. Critical situation can arise due to popping up of some error message or crashing of the system leading to forced full closure or semi closure of the application. Criticality of the situation can be judged by the fact that any type of workaround is not feasible. A bug can fall into "Critical" category in case of some menu option being absent or needing special security permissions to gain access to the desired function being tested.

Severity level - High: is a level of major defect under which the product fails to behave according to the desired expectations or it can lead to malfunctioning of some other functions thereby causing failure to meet the customer requirements. Bugs under this category can be tackled through some sort of workaround. Examples of bugs of this type can be mistake in formulas for calculations or incorrect format of fields in the database causing failure in updating of records. Likewise there can be many instances.

Severity level - Medium: defects falling under this category of medium or average severity do not have performance effect on the application. But these defects are certainly not acceptable due to non-conformance to the standards or companies wide conventions. Medium level bugs are comparatively easier to tackle since simple workarounds are possible to achieve desired objectives for performance. Examples of bugs of this type can be mismatch between some visible link compared with its corresponding text link.

Severity level - Low: defects falling under low priority or minor defect category are the ones, which do not have effect on the functionality of the product. Low severity failures generally do not happen during normal usage of the application and have very less effect on the business. Such types of bugs are generally related to looks & feel of the user interface & are mainly cosmetic in nature.

Three levels of Bug Priority:

High Priority: Such bugs if not fixed immediately, are going to affect the normal functioning at customer end. Hence such bugs are given immediate or topmost priority for fixing.

Medium Priority: is allocated to the bugs with major defects having great effect on the functioning of the customer. Such bugs are allocated great priority so those related problems are resolved prior to release of the present software version. If due to some time constraint it is not possible to resolve this issue, some sort of patch or service pack must be released

Low Priority: is generally allocated to bugs, which do not have significant effect on the performance of the software at the customer end. Effort is made to resolve such bugs prior to release of the present version, if this is not feasible due to constraint of time such fixation can wait till the release of next version.

8. Explain by Screen-shots - As per an old saying "A picture is worth more than 100 words". When we encounter an error, it is best to capture the screen-shot of the particular moment. If an error message is seen, its screen-shot will help the developer in having a precise understanding of the problem. This is the stage the developer does not try to fix the problem, rather focuses his attention to firstly understand the problem clearly.
Such screen-shot should be an appendix of the bug report as evidence. This way tester is able to communicate & explain his bug in a better way & with more clarity to the developer.

9. Standby your Genuine Bugs - Most interesting portion of bug reporting is when the software tester needs to be standby the bugs found by him & needs to defend that bugs covered under his report are genuine bugs needing fixation since it is going to affect the performance of the application. During this the course, the software-testing engineer must be ready to face situations with the programmers like the few ones mentioned below.

Case 1: Developers usually hit back saying that particular bug can't be reproduced. The best way of reporting the bug is practically showing it to the developer. This can be done by asking them to have a look at the scenario on your computer system, load the application & provide live demonstration of the problematic event. This would provide them actual look & feel of the situation as to how you had fired the application, how you had been interacting with the application & how the software reacts to the inputs provided. As a best practice avoid reporting of non-reproducible bugs in an enthusiasm to report maximum number of bugs in shortest possible time.

Case 2: Sometimes the software tester comes across funny circumstances with some application having inconsistent pattern of failure. Such situations can arise when the tester encounters pressure of deadlines & the application under test fails, but he faces embarrassing moments while demonstrating the same to the developers, since the application behaves normally at that moment. Thus a good tester needs to be patient & always build a defense mechanism in the form of preserving test data & screen-shots etc. to justify his statements.

Case 3: If the tester provides the developers a big list of various actions & inputs etc. but the program fails to show anything wrong when executed on the system of the developer. This means that the tester has not provided sufficient information. It is quite possible the systems of the developer & the tester differ in some configuration, thereby causing the fault not to appear on computer system of the developer. It is quite possible that the tester had misunderstood the expectations of the program, while the tester & the developer are witnessing the same display, but with a difference in viewpoint. It might be possible that what appears, as an error to the tester might be correct from the point of view of the developer. Thus to come out of such situations, it is preferred to say as to what you had expected & what you had seen exactly & what had happened.

Thanks & Regards ,

Prashant Vadher | QA Engineer


Post a Comment

Design by Prashant Vadher