Intro Models Documentation Design Scripting Tools Execution Defects Reporting  

Test and defect reporting and metrics

The Test Strategy specifies the frequency and components of test reporting. Test script progress and defect counts are the most common reports distributed across the project. If test reporting is effective the development organisation will be able to see which areas of the system have a good degree of quality and which areas are of lower quality or perhaps largely untested.

Test reporting

Test reports should be broken down into the packages of features or areas that were developed. The reports should cover the number of tests for each feature, the number scripted, the number executed, total passes and total failures. If these are recorded over a period of time you can view trends and predict when the scripting or test execution might complete.

A typical test report might look like this:

Feature Total Scripts Scripted Run Pass Fail
Order 25 25 17 11 6
Invoicing 18 18 18 18 0
Billing 22 22 0 0 0
Reporting 17 10 0 0 0
Total 82 75 35 29 6
% 100 91 43 35 7

You can see that in this example testing is only part way through and that although scripting is almost complete not even a half of the total tests have been executed. If a history of test reports is kept you can generate test progress charts as below:

Graph plotting testing progress against time

Test progress graph

Pie chart showing test status

Test status chart

Defect reporting

Defect reporting is well catered for in the testing tools mentioned earlier. Each provde a way of exporting data which means you can create your own reports if you wish. The rate of defect incidence is a good measure as that combined with test progress indicates when the sofware is close to achieveing the test completion criteria specified in the Test Strategy.

The graph and chart below are examples of these reports. The first shows defect progress against time, you can see that the total number of open defects has declined slightly, as has the number of defects assigned. The second chart shows how defect status is distributed, half of the current defects are in an assigned state.

Graph plotting  defect incidence rate against time

Defect incidence graph

Pie chart showing defect status

Defect status chart


Home      |      Testing      |      Telecoms      |      Resources      |      Clients      |      Contact     |     Care Database
 Copyright © 2016 Chic Computer Consultants. All rights reserved