ISTQB ATM Advanced Test Manager – Defect Management

1. Introduction

And the chapter we are going to discuss is about defect management. And the discussion points would be to gain insight of status of project and present it in correct way. Two critical factors are defect Management process tool used for management of defects. Taste manager must be familiar with what critical data to capture how to properly use process and selected defect management tool.

2. Defect Lifecycle and Software Development Lifecycle

So, before getting into how to manage defects, let’s try to understand the defect lifecycle and the software development lifecycle. Defects are introduced when a person makes a mistake while creating a work product. During software development or maintenance process, the work product can be requirement specification, a user story, a technical document, a test case or the program code. Some Facts about defects Each phase of software development lifecycle should include activities related to defects. Defects Finding because defect can be introduced at any point in life cycle, the earlier each defect is detected and removed, the lower the overall cost of quality for the system. Cost of quality for a given level of defects is minimized when each defect is removed in the same phase in which it was introduced. Static testing finds defects directly rather than finding failures, and thus the cost of removing the defect is lower because debugging activities are not required to isolate the defect during dynamic testing. Activities such as unit testing, integration testing and system testing. The presence of a defect is revealed when it causes a failure, which results in a discrepancy between the actual result and the expected results of a test.

In test driven development, automated unit tests are used as a form of executable design specifications. As code is developed, it is immediately exercised using this test. Until the development of the unit is complete, some or all of the tests will fail. Therefore, the failure of such a test does not constitute a defect and is typically not tracked. Defects workflow and different states of Defect every defect passes through a defect life cycle which has multiple states for a defect. Three states of defects where testing team needs to take action on defects are the initial state. In the initial state of defect, information to reproduce the defect is gathered by testers for the day team to work on it. This may also be referred to as the open or new state. Then comes the return state. In this state, the defect has been rejected or additional information has been asked. It indicates shortfall in the initial information gathering or testing process itself. The test manager should monitor excessive rates of return.

Tester must provide additional information or confirm the rejection. This state may also be referred as rejected or clarification state and the third state is confirmation state. In this state, defect fixed verification happens by testing. If the defect has been fixed, state changes to closed. If the defect has not been fixed, state changes to reopen and should be reassigned to the previous owner. It may also refer as result or verification state how to manage duplicate and invalid defect Report sometimes, due to the problem with test environment test data, other element of testware or testers own misunderstanding, a false positive result occurs.

Such results are closed or canceled as invalid defect Report when two or more defect reports are subsequently found indicating same root cause, one of the defect report is retained and all others are closed with duplicate status. The test manager should understand that duplicate and false positive reports cannot be completely removed and that is perfectly fine. Cross functional defect management in addition to test manager, following stakeholders are part of defect rich committee and have an interest in software under development. These people are development manager, project manager, product manager and other stakeholders. The defect committee should meet and analyze every defect whether they are valid or not, whether they should be fixed or deferred.

The defect management committee should consider following factors to decide whether a defect should be fixed or not, and those factors are benefits, risks and costs associated with fixing the defects. Found out test manager and test team should provide relative information about in which priority defects should be resolved. For effective and efficient defect management, following factors are important it’s communication, adequate tool support, a well defined defect lifecycle, and engaged defect management committee. So that’s about what is the defect and the defect life cycle. Let’s move to the next subtopic.

3. Defect Report Information

Let’s talk about defect report information. Purpose of Defect report information is management of the report. So the defect life cycle assessment of project status, especially in terms of product quality and taste progress and assessment of process capability. Benefits of collecting Defect Data collection of defect data is useful in test progress monitoring, control and exit criteria evaluation. For example, defect information should support defect density analysis trained analysis of defects detected and resolved average time from defect detection to resolution and failure intensity. Information expected from defect data. In common words, the defect data collected are being termed as bug report.

So what? All are the information to be expected from a typical defect data report or bug report? The name of the person who discovered the defect the role of the person the type of testing being performed a summary of the problem a detailed description of the problem steps to reproduce the failure along with the actual and expected results, including screenshots, database dumps and locks, wherever applicable.

The life cycle phase of introduction, detection and removal for the defect, including the test level, if applicable the work product in which the defect was introduced the severity of the impact on the system and the product stakeholders the priority to fix the problem the subsystem or component in which the defect lies the project activity occurring when the problem was detected the identification method which revolved the problem the type of defect the quality characteristics affected by the defect the test environment in which the defect was observed the project and product in which the problem exists the current owner the person currently assigned to work on the problem, assuming the report is not in a final state.

The current state of the report the specific work products in which the problem was observed along with the specific work products in which the problem was ultimately resolved the impact on project and product stakeholders interests. Conclusions, recommendations and approvals for the action taken or not taken to resolve the problem. Risks, costs, opportunities and benefits associated with fixing or not fixing the defect. The dates on which various defect lifecycle transitions occurred. The owners of the report based on each transition and the actions taken by project team members to isolate, repair and verify the defect a description of how the defect was ultimately resolved and recommendation for testing the fix. Other references, such as the test that reveled the defect and the risk requirement or other test basis element related to the defect. Test Manager can refer different standards to determine what all information should be included as part of defect reporting. Testers should enter all the information that is complete, concise, accurate, objective, relevant, and timely.

4. Assessing Process Capabilities

And the last topic in this chapter about defect management is assessing process capability with defect report information. Test managers should be aware of importance of defect report to assess the capability of testing and development process defect information should support process improvement activities like using the phase of introduction, detection and removal information on a phase by phase basis to assess phase containment and suggest ways to improve defect detection effectiveness in each phase.

Using the phase of introduction information for pareto analysis of the phases in which the largest number of defects are introduced to enable targeted improvements to reduce the total number of defects using the defect root cause information to determine the underlying reasons for defect introduction to enable process improvements that reduce the total number of defects using the phase of introduction detection and remove information to perform cost of quality analysis to minimize the cost associated with defects using defect component information to perform defect cluster analysis to better understand technical risks and to enable reengineering of troublesome components.