Cross-product Functional Coverage

This example is based on a system in which transactions are created. Each transaction has the following attributes:

• Transaction Name
• Transaction Sender
• Transaction Receiver
• Starting Time
• Ending Time

• Single Transaction Models
• Two Transaction Models
• Many Transaction Models

All of these models use the same data. When many models use the same data, we recommend a tool such as Meteor that is based on the domain paradigm.

This example shows the importance of hierarchy and the differences between temporal models and snapshot models. It also demonstrates models for stress testing.

Formats for Collecting Information in Traces

Transaction Name, Sender, Receiver, Starting Time, Ending Time.

This format is usually easy to process, but sometimes requires additional calculations from the user.

Time, Transaction Name, Transaction ID, End-or-Start, Sender or Receiver.

In this format, each transaction is associated with two lines in the trace. For example, look at the following trace segment:

• 23, Do-good, 1, Start, MyName
• 40, Do-bad, 2, Start, Criminal
• 45, Do-good, 2, End, Police
• .....

This transaction includes the following attributes:

• Transaction Name = Do-good
• Transaction Sender = MyName
• Transaction Receiver = Police
• Starting Time = 23
• Ending Time = 45

1. Story
2. Tupple
3. Values for each Attribute
4. Restrictions

All-Transactions Model

1. We would like to know that we have seen all the transactions
2. The tupple is <Transaction Name>
3. Attribute Values
1. Transaction Name has all the possible transactions as values
4. Restrictions
1. No Restrictions

• Find areas in which coverage is missing or low and improve testing in these areas.
• Measure statistics on how the application is used.

Transactions-Sender Model

1. We would like to know that we have seen all the transactions sent by all possible senders
2. The tupple is <Transaction Name, Transaction Sender>
3. Attribute Values
1. Transaction Name has all the possible transactions, as values.
2. Sender Name has all the entities that can create transactions, as values.
4. Restrictions
1. Not every Sender can send every possible transaction. Remove illegal combinations.

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Transactions-Receiver Model

Transaction-Time Model

1. We would like to know that we have seen all the transactions finished in reasonable time periods.
2. The tupple is <Transaction Name, Transaction Time>
3. Attribute Values
1. Transaction Name has all the possible transactions, as values.
2. Transaction Time = End Time - Start Time and can have the following values (e.g., real ranges are application independent)
1. 0
2. 1
3. 2
4. 3 ... 10
5. > 10
4. Restrictions
1. Not every Transaction can have every possible duration.

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Transaction-Sender-Receiver Model

1. We would like to know that we have seen all the transactions sent by all senders to all receivers.
2. The tupple is <Transaction Name, Transaction Sender, Transaction Receiver>
3. Attribute Values
1. Transaction Name has all the possible transactions, as values.
2. Sender Name has all the entities that can create transactions, as values.
3. Receiver Name has all the entities that can receive transaction, as values.
4. Restrictions
1. All the restrictions from the Transactions-Sender Model
2. All the restrictions from the Transactions -Reciever Model
3. A matrix on which receiver can get transactions from which sender

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Two-Concurrent-TransactionModel

1. We would like to see which transactions can occur at the same time. Two transactions occur at the same time if there exist a time T such that   S1 =< T =< E1 and  S2 =< T =< E2
2. The tupple is <T1 Name, T2 Name>
3. Attribute Values
1. T1 and T2 have the values of Transaction Name.
4. Restrictions
1. The restrictions are implementation-specific and depend on which transactions can co-exist.

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Two-Concurrent-Transaction-Sent Model

1. We would like to see which transactions can be sent at the same time. We would like to see two concurrent transactions (as per previous model) that are sent by the same sender.
2. The tupple is <Two-Concurrent-Transaction, Transaction Sender>
3. Attribute Values
1. Two-Concurrent-Transaction as per the Two-Concurrent-Transaction model
2. Transaction Sender has the values of Transaction-Sender
4. Restrictions
1. The restrictions of the Two-Concurrent-Transaction model
2. The restrictions of the Transactions-Sender model
3. Additional restrictions from the application on what transaction a sender can send, as long as he has a transaction outstanding.

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Two-Concurrent-Transaction-Received Model

Two-Transaction-Interplay Model

1. We would like to see how transactions interact with each other in the time domain.
2. The tupple is <T1 Name, T2 Name, Interplay>
3. Attribute Values
1. T1 Name T2 Name have the values of Transaction Name.
2. Interplay has the following values (some of them do not make sense in specific implementations).
1. Instance => S1=S2=E1=E2
2. Equal   => S1 = S2 and E1 = E2
3. Contained => S1 < S2 and E2 < E1
4. Consecutive => S2 = E1
5. Consecutive + 1 => S2 = E1=1
6. After => S2 > E1+1 and there is no third transaction between them
7. Staggered => S1 < S2 < E1 < E2
4. Restrictions
1. The restrictions are implementation-specific and depend on how transactions interact.

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Two-Transaction-Interplay-Send / Two-Transaction-Interplay-Receive Models

Three-Concurrent-Transaction Model

1. We would like to see how many transactions can occur concurrently.
2. The tupple is <Number-Of-Transactions>
3. Attribute Values
1. Number-Of-Transactions = 0,1,2,3,4, Many
4. Restrictions
1. No restrictions are expected

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used. This is the key reason for writing this model.

Number-of-Concurrent-Transactions * Transaction Model

1. We would like to see how many transactions can occur concurrently with a specific transaction.
2. The tupple is <Number-Of-Other-Transactions, Transaction Name>
3. Attribute Values
1. Number-Of-Transactions = 0,1,2,3,4, Many
2. Transaction Name has the possible transaction names as values
4. Restrictions
1. Some transactions can not have many simultaneous transactions. This should be reflected.

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Number-of-Concurrent-Sender / Number-of-Concurrent-Receiver Models

Number-of-Transaction-in-Unit-of-Time Model

1. We would like to see how many transactions can occur during a specific time interval. This is usually done to measure stress
2. The tupple is <Number-Of-Transactions>
3. Attribute Values
1. Number-Of-Transactions = 0,1,2,3..5,6..10, more (depends on the application)
4. Restrictions
1. None expected

• Find areas in which coverage is missing or low and improve testing in these areas.
• Find illegal combinations.
• Measure statistics on how the application is used.

Number-of-Transaction-in-Unit-of-Time-by-Sender, Number-of-Transaction-in-Unit-of-Time-by-Reciever Models