Blog

go back to the blog

The Essence of a Good Bug Report

  • 26/10/2011
  • 6906 Views
  • no comments
  • Posted by EuroSTAR
-->

Writing an incident description

The issue’s title is also a summary, so we aim to be as concise as possible, trying to make sure that the language used would help someone have a good understanding of the issue or find the issue easily when searching. For the incident description, we try to define a summary of what occurred, the environment it occurred on, and the build of the software (if there is not a separate field for this), numbered reproduction steps that make it easy for someone to repeat the issue and then list the expected and actual results. It is important to define the expected result as sometimes it is possible that the expected result is wrong for some reason. We also try to identify workarounds for the issue as this can affect the speed at which it is addressed. It is sometimes difficult to know how to write the reproduction steps, but ideally you want anyone to be able to follow them, for example a tester in another team, or your product or project manager. This can seem laborious at times, but pays off.As we run our projects using the agile methodology we often cannot point people at test procedures, or test case specifications to provide more information, so we need to document all the necessary information with the issue itself.

JIRA issues have severity, priority and incidence levels which allow us to identify the actual or potential damage an issue might have. This relates to the impact of an issue as described in IEEE 829. We have in-house procedural documents to provide guidance on what levels to set for these fields. Many teams leave the priority field blank and allow this to be assigned by the project lead, who has a better appreciation of the project schedule.

Example: Issues in practice

The following example illustrates how we might use JIRA in practice. In the example, we have a Windows desktop application called TestBuddy that allows us to save various testing documents. When we try to save a test case specification in TestBuddy, it crashes.

Rather than raising a bug immediately, we would first investigate further to try and help narrow down and isolate the issue. We would look at the exception and what information it provides, check the application’s logging system (if there is one), and check the event viewer. Then we would want to ascertain whether there were any environmental factors, such as the operating system or the processor being 32-bit or 64-bit. This should help us clearly define the environment on which the issue occurs and elaborate on our findings. Then we would look at how the issue manifests itself, so you would want to know whether all documents are saved in the same format, whether the crash happens for all document types, whether all documents get saved to the same location, and what happens when you open a previously saved document (of that type/format).

Writing an incident report

JIRA provides a rough template of information that a user should provide when raising an issue and it can also generate a unique Test Incident Report Identifier for us.

In our example, we have determined that TestBuddy’s test case specifications are stored in XML, and the XML schema used to save the file are not being found, causing the application to crash. In this case, we would submit the following incident report:

Title

TestBuddy crashes when serialising/deserialising xml documents

Severity

High (assuming critical, high, medium and low)

Priority

Must fix (assuming Must fix, high, medium and low)

Incidence

Unavoidable (assuming unavoidable, regular, occasional and rare)

Description

Seen on operating system, processor, software build

When saving or opening a test case specification (XML document) the application crashes irrespective of operating system version or processor.

Steps to reproduce:

1) Install software

2) Open software

3) Create new document

4) Select Test case specification type

5) Click Save

 

Actual result: The application crashes.

Expected result: The document should be successfully saved.

Workaround: There is no workaround for this issue.

Conclusion

Writing a good incident report with all the information clearly laid out and easy to follow is really important, especially when it is done in conjunction with some initial investigation into the issue itself. The more information available, the more likely it is that developers will spot further implications the issue might have, or simply speed up their ability to fix the issue.

Blog post by

go back to the blog

eurostar

Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery