|
Classification by Information Source
- WhiteBox: model based on internal implementation details.
- BlackBox: model based on interface details.
- Broken Box: model that combines internal and external information
Additional Classifications of Coverage Models
- Snapshot: Coverage models that look at the situation at a single point in time. The point in time might be abstract. For example, if two routines are called sequentially, a model containing the values of variables from both routines will be considered a Snapshot model, even though the variables were not "alive" at the same time.
- Temporal: Coverage models that check scenarios. They usually follow this pattern: first event 1 happened, then event 2, and then event 3. Coverage is measured on the events (event 1 could have several legal types), on the time between the events, and on the way that the events interact.
- Stress: Coverage models that check how many things happen within a short interval of time, or at the same time. This also includes models that check if resources have been used to their capacity.
- Statistically oriented: Coverage models that are used to measure statistics. Usually coverage is a given, but it is interesting to know how many times each task has been covered. The main motivation is usually performance (e.g., do we need a stack that large?), but it can also be improvement of the biasing of the test generators.
Legal and Illegal Tasks
A coverage model is usually composed of the cross-product of the values for the events. For example, a coverage model can be composed of all instructions and all interrupts. The cross product checks that all combinations of instructions, taking all interrupts into account, have been seen. In this case, not all combinations can happen. For example, a branch instruction with a floating point divide interrupt will never be observed. It is very important to define the illegal tasks during model definition, if possible. There are a number of reasons for this. The first is that every illegal task encountered may signify a problem. The second is that without defining the illegal tasks, the coverage statistics are meaningless. The third, and possibly the most important, is that in defining the illegal tasks the designers learn a lot about the design.
Task Coverage List - TCL
A Task Coverage List is the list of all legal coverage tasks in the model. Typically, in a functional coverage tool, the list also contains records for each task, with information such as the number of times it was covered, the first time is was covered, and the name of the test that covered it.
Domains
A single application may have many coverage models associated with it. The information needed for these coverage models often overlaps. When models are built hierarchically, some models are subsets of others. Even if the models are not contained, they tend to share information. In many cases, it is therefore more efficient to query the union of the information needed for the models than to create the information for each model separately. A domain is the name we gave for all the information combined. In the domain paradigm, the application prints all the information together to the same trace and each model collects the information it needs from that trace.
Data-mining
Coverage can yield overwhelming amounts of data. It is very useful to be able to see the "big picture." Any tool that can report on large uncovered areas, or on the common features of many uncovered tasks, can be very useful. Data-mining can be used for this purpose.
Regression Sets
Regression sets are compact sets of tests run after each code modification or at certain intervals (usually short). It is very important for these tests to be as comprehensive as possible to increase the likelihood of finding new bugs. One way to create such a set of tests is to sort the tests by degree of coverage achieved. Given a fixed amount of time for running the tests, the subset with the highest degree of coverage is selected. There are programs that select such subsets (we have one in Haifa).
| |
|
