[Demo] [Overview of Examples] [User's Guide]
Examples\Predefined\Coverage

Test Coverage Predefined Probe: coverage.dll

Coverage Testing

A common need for certain projects is to ensure that each statement in a particular function has been executed at some point during the development cycle. For obvious reasons this is best done at the "unit" level (e.g. on a subset of the code, usually wrapped within drivers). Obviously the power of Aprobe is such that you can operate on your actual executable if you wish although you should understand that:

Coverage is very intensive since we are patching individual instructions. You will not want to call your functions you are getting coverage of very often! You should arrange things so that loops are executed as little as possible.

It can be difficult to arrange that all of your paths can be executed unless you build ways into your application to do this (not usually a good idea!).

The upshot of this is that it is seldom, if ever, a good idea to blindly attempt to get coverage of everything in your application at once. It's a development exercise, not an integration one.

A Simple Example

The example we will look at is pretty trivial; there is a more complex example using Coverage which uses custom probes to modify the behavior of the functions in Aprobe\Examples\Advanced\Test_Coverage.

The test application is very simple. It prints out a mini-menu for the user and waits for a selection. Depending on the selection it does one of two actions. You can see this in the DoProcessing() function in coverage_example.c. We want to exercise every line in this function (we refer to this as getting 100% coverage) - note that unlike some other applications of Aprobe, coverage requires that you are very familiar with the source code in order to get useful information!

While you can use the Aprobe Workspace to run the predefined probes as described in the Demo, we'll use the Aprobe command line to build the example application and run the probes, since the example program prompts for user input.

So choose Command Prompt from the Run menu, make sure you're in the Examples\Predefined\Coverage directory under Aprobe (or your local copy of this directory), and enter the following: 

nmake
aprobe -u Coverage -p -g coverage_example
The nmake compiles coverage_example.c into coverage_example.exe. Running aprobe as shown brings up the coverage predefined GUI.

Right now there's nothing selected so click on the Pick... button under the Coverage Functions:" list and select the coverage_example module. Select "DoProcessing()" and "main()" in the right hand list (note that you can use the Ctrl key to select multiple items), click the Add button to move them into the selected list and click OK. Click the Run & Save button to set our application going.

To do this without the GUI, you would edit the file "coverage_example.coverage.cfg" and add two lines: 

COVERAGE "DoProcessing()"
COVERAGE "main()"
Then you'd save this file and run the aprobe command without the "-p -g": 
aprobe -u Coverage coverage_example
Either way, once aprobe starts the application, it will prompt for input. Enter option 1 to print out the first string and the application will exit normally.

The data is stored in .apd files so we run the command apformat to view it: 

apformat coverage_example
This provides some output showing the coverage we've achieved: 
                                              Lines   Lines
                              Total   Total   Not     Not      Coverage
Subprogram Name               Calls   Lines   Probed  Executed Percent 
-----------------------------------------------------------------------
extern:"main()"                   1       3       0       0    100
extern:"DoProcessing()"           1      13       0       2     85
-----------------------------------------------------------------------
Total                             2      16       0       2     88
followed by specific details:
For 1 call to subprogram: extern:"DoProcessing()"
  Lines in subprogram: 13 total (incl. executed 11 of 13 probed lines)
  Percent of lines executed:  85
  The following probed lines were not executed:
    Lines: 28 .. 29 (coverage_example.c)

Using atcmerge

Obviously the lines not executed are those which would occur if we had selected option 2; we can easily execute those lines if we run it again but then we wouldn't be able to cover the option 1 lines! We could keep both sets of results and manually check that we got all the lines but Aprobe provides a better utility: atcmerge (aprobe test coverage merge). If you look in the local directory, you'll see a coverage_example.tc file. This was created at format time by the coverage predefined probe and is a binary representation of the above data.

We can use atcmerge to merge the results from two .tc files and display the combined data. Firstly we need to save the current .tc file away. Copying the file would do but we'll use atcmerge to demonstrate it a little: 

atcmerge -b combined.tc coverage_example.tc
This merges the data in coverage_example.tc with that in combined.tc (which, in this case, means that combined.tc ends up with the same data). You can verify that this .tc file has the same data by just running atcmerge on it: 
atcmerge combined.tc
Now run the application again using aprobe; this time we won't invoke the coverage GUI since we already picked the functions we wanted: 
aprobe -u Coverage coverage_example
Select option 2 and re-run apformat. This time we get a different set of lines executed (as expected). We use the same atcmerge command as before to combine the data in combined.tc (from the first run) with the data from this run (in coverage_example.tc): 
atcmerge -b combined.tc coverage_example.tc
Now we get the 100% coverage we wanted!

Conclusion

You should now have some idea about using the Coverage predefined probe. The reference for it can be found in "Test Coverage Probe: coverage.dll" in Appendix D and a more complex example of using coverage that includes writing your own probes to modify the execution flow during testing can be found in Aprobe\Examples\Advanced\Test_Coverage.
[Demo] [Overview of Examples] [User's Guide]