|
Testing is one of the biggest problems faced by both the hardware and software industries. The cost of testing is usually between 40 – 80 percent of the development process (Microsoft spends 70 percent, for example) as compared to less than 20 percent for the coding itself [4]. The practice of letting users find the bugs and then having them fixed in the next release is becoming dangerous and costly for three main reasons: damage to reputation of the company and brand name, cost of replacing software for a large install-base, and risk of litigation if the software error harmed the user. It is important to verify that testing resources are used efficiently and that testing is thorough.
The main technique for demonstrating that testing is thorough is called "test coverage analysis" or simply "coverage" in context [10]. The technique creates, in a systematic fashion, a large and comprehensive list of tasks, and checks that each task is covered (that is exercised) by testing The person who plans the tests chooses tasks over a range of granularity, for example from the task of adding two numbers to the task of backing up a large database. Coverage helps monitor the quality of testing, assists in creating tests for areas that have not been tested before, and helps form small, yet comprehensive regression suites [5].
There are two general types of coverage: code and functional. Functional coverage focuses on the capabilities of the program. It measures the extent to which testing covers program capabilities. Therefore, functional coverage focuses on program semantics (what the program does). Functional coverage is implemented either manually or with the aid of custom-made tools. Code coverage measures syntactic properties of execution (for example, which statements or branchs are executed). The domain independence of program syntax makes practical and available many code coverage tools for popular programming languages. For example C [9], C++ [13], and Java [14].
IBM makes a number of coverage tools available:
- Focus: Implements functional coverage methodology. Available for free at www.alphaworks.ibm.com. Focus can also measure code coverage for Java. Meteor: Focus' big brother. It is also available from Haifa, for a fee.
- xSuds: C C++ code coverage tool from the former Bell labs. Available for free when used within IBM.
- PureCoverage: C, C++, Fortran and assembler coverage tool from Rational (not free).
| 
