Coverage Predefined Probe

From OC Systems Wiki!
Jump to: navigation, search

Test Coverage Predefined Probe: coverage.ual

NOTE: As of Aprobe version 4.4.9 the coverage.ual probe includes branch coverage features from the brcov.ual prototype. For older versions of Aprobe coverage.ual the branch coverage features described here are not available.

The coverage.ual predefined probe supports test coverage analysis on user applications. Test coverage tells you exactly which lines and branches within a function were actually called during program execution. Furthermore, you can tag each function, line, and branch execution with the name of the test which executes it to provide traceability. This allows you to develop unit test cases which can be used to confirm that all the lines and branches of code you have written are executed. The reports produced by coverage.ual support function, line, branch, and decision coverage.

While this probe is a unique and powerful tool for collecting coverage information, it is only a tool. Getting complete execution coverage on a function can be a difficult task, and requires the user to compose test cases which will cause all paths to be executed, including error conditions and exception propagation.

But it should be noted that it is in testing these tricky cases that Aprobe can especially useful, since you as a user can write a probe that modifies the data on_entry to a function to force a specific path through the function, without worrying about how to get the normal caller of that function to pass that erroneous data.

You can find an example that works through some of the issues that arise from using Aprobe to do this in the $APROBE/examples/learn/test_coverage directory. Additionally you are encouraged to contact OC Systems to discuss using Aprobe on your specific application.

Usage

This probe is applied at run time using aprobe as described under Test Coverage UAL Parameters below. The only functions for which data will be collected are functions selected in the configuration file (see Test Coverage Configuration File and Test Coverage Configuration GUI). Selecting a function means that all lines and branches in the function will be instrumented. Conversely, lines and branches in functions that are not selected will not be instrumented, and no record will be kept of invocations of these functions, lines, or branches.

The instrumentation of lines requires that there be accurate source line debugging information recorded in the executable. This generally means that the application or library containing the covered functions:

  • must have been compiled with the appropriate debug flag (-g);
  • should have been compiled without optimization; and
  • must have the debug information still available.

A snapshot of the current state of collected coverage data may be written to the APD file while the program is running (see Configuration of Snapshots). Taking such periodic snapshots at known events, like the entry to or exit from a particular function, may help correlate state data with the execution path recorded by the coverage probe. A final snapshot occurs automatically at program termination.

After the application program terminates, the apformat command must be run to produce an ASCII report and a test coverage .tc and a binary coverage .bc file from the collected data. The report can be viewed with any text editor. Data from multiple runs may be merged into a single .tc or .bc file and a single report using Atcmerge (see atcmerge below).

The apformat command will also produce a branch coverage .brc file from the collected data. Reports, both text and HTML, are generated from the branch coverage data using abrmerge. The branch coverage data from multiple runs can be merged into a single brc file using Abrmerge (see abrmerge below).

The ASCII text report produced by a single run or by Atcmerge consists of two parts: the "Subprogram Summary" and the "Subprogram Details" (A subprogram is another name for a function, procedure, subroutine -- any callable entity.) The Summary lists the total lines in the subprogram, how any lines were not instrumented nor executed, and the percentage of instrumented lines that were executed for the subprogram. The Details section reports whether the subprogram was called, and if it was, it will report how many of the instrumented lines were executed. In cases where test coverage is less than 100% for a particular subprogram, the Details section will also list the line numbers of the instrumented lines which were not executed. The formatted output coverage data will look similar to that in Test Coverage Summary Report.

The report produced by Abrmerge consists of several parts: the "Summary Section", the "Details Section", and the "Annotated Source Section". Not all section are generated by default. The Summary lists the total lines and branches in the subprogram, how any lines and branches were not instrumented nor executed, and the percentage of instrumented lines and branches that were executed for the subprogram. The Details section reports whether the subprogram was called, and if it was, it will report which lines and branches were instrumented and executed. The formatted output coverage data will look similar to that in Branch Coverage Summary Report.

Test Coverage UAL Parameters

coverage.ual is specified on the aprobe command line or in an APO file, and with apformat. The specific options are:

 
 aprobe   -u coverage[.ual]    [-p " [-c config_filename] [-f] [-h] [-v] [-g] [-o tc_filename]"   your_program 

For example:

 aprobe -u coverage -p "-c foo1.coverge.cfg" foo.exe
 aprobe -u coverage -p -g

where:

-c config_filename

specifies that the name of the probe configuration options file will follow immediately after -c. The default file name is your_program.coverage.cfg. For example, if your executable program is called wilbur.exe, then the default file name would be wilbur.exe.coverage.cfg.
-f length
truncates function names from the left to the specified length for easier-to-read output. By default, function names are not truncated (applies only with apformat).
-g
starts the Java configuration GUI at probe startup, before running your program (applies only with aprobe).
-h
produces brief help text.
-o tc_filename

specifies the name of the output test coverage data file that will be created when formatting the data. By default, its name will be your_apd_file.tc.
-v
verbose mode, which produces additional progress messages.

Note that if you use no UAL parameters, you need not specify UAL names at all. For example: apformat progname will format progname.apd with all the UALs with which it was created.

Test Coverage Configuration File

The Test Coverage configuration file is used to specify what subprograms are to be analyzed, when snapshots are to be taken, and other options, as described in Configuration File. The example below shows one possible Test Coverage configuration file.

 
 PROBE CONFIGURATION FILE FOR COVERAGE VERSION 2.0.0
 
 CoverageEnabledInitially TRUE
 StartWithGUI FALSE
 ReportAllOffsetsSeparately FALSE
 CommaOutputFile coverage.dat
 
 // Here we select which subprograms we want to cover:
 COVERAGE "main"
 COVERAGE extern:"YET_ANOTHER_LOCAL_PROC"
 COVERAGE "STILL_ANOTHER_LOCAL_PROC"
 
 // Here we define which subprogram invocations cause snapshots
 // to be saved.
 SNAPSHOT "printf" IN "libc.a(shr.o)" ON ENTRY IS "printf called"

Example D-2. coverage.cfg File

Note that if you do not provide a configuration file, the probe will create a default configuration file for you. This is a good way to get started.

Configuration Variables

The following are the only valid keywords that identify lines to set configuration variables. Each such line must begin with one of these keywords, and the keyword must be followed by a value. Nothing else is allowed on the same line.

CoverageEnabledInitially

This must be followed by the value TRUE or FALSE. The default is TRUE which indicates that data logging will begin as soon as the application program starts running. If set to FALSE, data logging will begin only after a call is made to the probe's function ap_EnableCoverage(), rather than as soon as the application program starts running.

StartWithGUI

This must be followed by TRUE or FALSE. The default is FALSE. A TRUE value indicates that the pre-runtime GUI should be started before running the target program, even if -g wasn't specified on the command-line. A FALSE value is overridden by the -g command-line option.

CommaOutputFile

This must be followed by a filename. The presence of this option indicates that the named file is to be created containing the summary portion of the coverage report represented as lines comma-separated fields that may be easily imported into a spreadsheet application such as Excel.

ReportAllOffsetsSeparately

This must be followed by TRUE or FALSE. The default is FALSE, which indicates that a line consisting of multiple physical code offsets will be treated as one logical line, so that if any of the offsets is executed, the line is marked as having been executed. Conversely, TRUE indicates that each offset should be treated as a separate line. This option only has effect at format-time, not run-time.

ShowAllSnapshots

This must be followed by TRUE or FALSE. The default is FALSE. When FALSE, the data from all intermediate snapshots are rolled into the one final report. When TRUE the data for every snapshot will be reported in separate summaries.

ResetSnapshotCounts

This must be followed by TRUE or FALSE. The default is FALSE. When FALSE, the data from all intermediate snapshots accumulate into the one final report. When TRUE the counts are reset for each separate snapshot.

AlternateOutputFormat

This must be followed by TRUE or FALSE. The default is FALSE. When FALSE, the symbol names are reported in the left-most column. When TRUE, the symbol names are reported in the right-most column.

MaxSymbolNameLength

This must be followed by an integer value greater than or equal to 0. The default is 0. This controls the length of the symbol names in the report. Longer symbol names will be truncated. Use 0 for unlimited length.

ExpandDottedLines

This must be followed by TRUE or FALSE. The default is TRUE. When FALSE, dotted lines are compressed "into" the top-most line. When TRUE, dotted lines are reported individually.

EnableBranchCoverage

This must be followed by TRUE or FALSE. The default is TRUE. When TRUE, function, line, and branch coverage information is produced in a .brc file for use with abrmerge.

Configuration of Coverage Functions

Each function for which test coverage data is to be collected must be specified explicitly using the COVERAGE, COVERFILE, or COVERUNIT keywords.

The COVERAGE keyword is followed by the name of the function, as described in Configuration of Selected Functions. This directive selects specific functions by name (with possible wildcards).

The COVERFILE keyword is followed by the name of a file and an option module name. This keyword selects the functions originating from the designated source file (with possible wildcards).

The COVERUNIT keyword is followed by the name of a PowerAda unit. This keyword selects the functions originating from the designated PowerAda unit (with possible wildcards).

By default, if no COVERAGE/COVERFILE/COVERUNIT lines exist, then coverage data will only be collected for the "main()" function of the application module.

The REMOVE keyword allows you to specify subprograms that should not be instrumented for logging coverage data. This is useful when used in conjunction with a wildcard ("*"), to gather data about everything except certain routines.

Note: COVERAGE "*" is not a particuarly useful concept since it is important for you to identify and isolate your functions carefully in order to use test coverage for its intended purpose.

Configuration of Snapshots

The coverage probe configuration file allows you to specify the name of some subprograms for which snapshots of the coverage data are to be automatically taken. This is done with lines beginning with the keyword SNAPSHOT.

Each SNAPSHOT line must specify a particular subprogram in the usual manner, just as is done for a COVERAGE line.

The remainder of the SNAPSHOT line contains pairs, where each pair has a special identifier keyword followed by its own associated value. These pairs give supplementary information about the snapshot.

  • ON - This optional special identifier must be followed by the value ENTRY or EXIT, to denote that the snapshot is to be taken on entry to or exit from the subprogram respectively.
  • IS - This optional special identifier must be followed by an (arbitrarily long) string enclosed within quotation marks (""). It specifies a textual description or title that is to be logged along with the snapshot.

Configuration of TestNames

The coverage probe configuration file allows you to specify the name of some subprograms for which test names can be set. This is done with lines beginning with the keyword TESTNAME.

Each TESTNAME line must specify a particular subprogram in the usual manner, just as is done for a COVERAGE line.

The remainder of the TESTNAME line contains pairs, where each pair has a special identifier keyword followed by its own associated value. These pairs give supplementary information about the test name change.

  • ON - This optional special identifier must be followed by the value ENTRY or EXIT, to denote that the test name is to be set on entry to or exit from the subprogram respectively.
  • IS - This optional special identifier must be followed by an (arbitrarily long) string enclosed within quotation marks (""). It specifies a test name that is to be logged along with the snapshot.

Here is an example of a TESTNAME directive:

TESTNAME "my_function" ON ENTRY IS "test 1"

Configuration of TestName Hook Function

The coverage probe configuration file allows you to specify the name of some subprograms which are called when test name changes. The designated function should have a single (in) parameter which is a pointer to a nil-terminated string which is the test name. A value of NULL indicates no test name.

Each TESTNAMEHOOK line must specify a particular subprogram in the usual manner, just as is done for a COVERAGE line.

Here is an example of a TESTNAMEHOOK directive:

TESTNAMEHOOK "my_function"

Test Coverage Configuration GUI

The configuration GUI (Graphical User Interface) for coverage.ual is the "Test Coverage Configuration Dialog." This dialog allows you to set the options that control the coverage probe and is generally as described in Configuration GUI. The Help button provides the following description:

This dialog allows you to set the options that control the Coverage probe, and to override any options from either the configuration file or the command-line. The default settings for the options will come from the configuration file, if one exists.

There are two lists of function names in this dialog:'Coverage Functions', and 'Snapshot Functions'.

Each of these two lists can be changed by clicking on the 'Pick' button below the list, and then selecting which functions will cause the test coverage actions to occur. Each module inside the target program may contain many function symbols that you can select.

'Coverage Functions' lists the functions whose line coverage data you want to collect. The coverage data will be collected for all instrumentable lines in all these functions.

NOTE: If the 'Coverage Functions' list is empty, then coverage data will, by default, be collected only for the "main()" function of the application module

'Snapshot Functions' lists the functions whose invocation will cause the contents of the collected data in memory to be logged to an APD file.

'Coverage enabled initially' indicates whether the logging of coverage data will begin immediately upon target program startup or deferred until after function ap_EnableCoverage is called from some other probe.

'Start configuration GUI' causes this graphical user interface to appear every time this UAL is about to begin probing the program, whether or not the -g option was specified on the UAL command line.

'Treat offsets as lines' will treat a line consisting of multiple physical code offsets as one logical line. That is, if any of the offsets is executed, the line is marked as having been executed.

'Comma-delimited output file' indicates the name of a file (if any) to which spreadsheet or database readable tables should be written.

Buttons:The 'Save As' button saves the current options to a configuration file.

The 'Run (no save)' button runs the target program using the selected options.

'Save & Run' saves the options, then runs the target program using them.

The 'Abort' button will abort the GUI and application program immediately.

Test Coverage API

Users can control the behavior of the coverage probe by calls from within their own probes. The API for the trace probe is defined by [../include/coverage.h $APROBE/include/coverage.h]. Some of the functions exported by coverage.ual are:

  • ap_Coverage_Enable - Enables logging of coverage data.
  • ap_Coverage_Disable - Disables logging of coverage data.
  • ap_Coverage_DoSnapshot - Dumps current coverage information.

See Snapshots for more information and an example.

Coverage Demand Actions

(Since 4.4.8).

You can control coverage.ual using demand.ual and apdemand.

Include demand.ual on the Aprobe command line:

 aprobe -u coverage -u demand myapp.exe
 

then use apdemand to send actions:

 apdemand coverage snapshot
 

statprof.ual responds to the following actions:

  • coverage snapshot all threads

Note that all actions containing the action string will be triggered.

Test Coverage Performance Issues

See Performance Issues for a general discussion of factors that affect performance.

The coverage probe is applied to the functions specified in the configuration file, so the run-time overhead is directly proportional to the number of calls to the functions selected and the number of lines executed in each function. The coverage probe simply increments an integer for each line executed, so the amount of data is a constant multiple of the number of lines being covered, though additional snapshots increase this.

As test coverage analysis is generally a unit-testing activity, it is mostly done on the small number of related functions under test, so performance should not be an issue. Application-wide test coverage analysis during integration testing is not recommended.

Atcmerge

The results of multiple runs of coverage.ual over the same executable may be merged into a single "combined summary" report using atcmerge.

A merged summary report is useful when running many tests, each with different input data or perhaps different probes, to force all logic paths to be executed.

A simple example

This example of the coverage predefined probe can be found in $APROBE/examples/predefined_probes/coverage.

The test application is also very simple. It just prints out a mini-menu and waits for you to select 1 or 2. Depending on your selection, it does one of two actions. You can see this in the DoProcessing() function in coverage_example.c. We want to exercise every line in this function -- we refer to this as getting 100% coverage.

Note that, unlike some other uses of Aprobe, coverage requires you to be familiar with (the lines in) your source code to make most effective use of the probe.

As before, we'll use make to build the example application and then we'll use the coverage GUI to pick the function:

cd $APROBE/examples/predefined_probes/coverage
make
aprobe -u coverage -p -g coverage_example

This brings up the coverage GUI. By default, only the function "main()" is selected, so click on the 'Pick...' button under the "Coverage Functions:" list. This presents a list of all modules linked into this executable, so select the module named "coverage_example" and then click OK to get a symbol selection dialog showing all functions in that module. Click on function "DoProcessing()" in the right hand list, then click the '<Add<' button to move it to the selected list on the left, and then click OK. Finally, click the Save & Run button to save your selections and run the application.

To do this manually (without the GUI), edit the coverage_example.coverage.cfg file and add two lines:

COVERAGE "DoProcessing()"
COVERAGE "main()"

Save the file and run:

aprobe -u coverage coverage_example

When the example application prompts you for a choice, enter the number 1. That will print out the first string and then the application will exit normally.

As before, aprobe stored the data in .apd files, so use apformat to view it:

apformat coverage_example

Here's a portion of the formatted output showing the coverage we've achieved:

                                              Lines   Lines
                              Total   Total   Not     Not      Coverage
Subprogram Name               Calls   Lines   Probed  Executed Percent
-----------------------------------------------------------------------
extern:"main()"                   1       3       0       0    100
extern:"DoProcessing()"           1      13       0       2     85

-----------------------------------------------------------------------
Total                             2      16       0       2     88

...

For 1 call to subprogram: extern:"DoProcessing()"
  Lines in subprogram: 13 total (incl. executed 11 of 13 probed lines)
  Percent of lines executed:  85
  The following probed lines were not executed:
    Lines: 28 .. 29 (coverage_example.c)

Using atcmerge

Sometimes we want to combine several coverage reports into a single report. But to avoid manually merging reports, especially when each covers different lines of code, Aprobe provides a better utility: atcmerge (aprobe test coverage merge).

The above 2 lines that were not executed are specific to choice number 2. We could easily execute the lines of choice number 2 if we ran it again, but then we wouldn't be able to cover the choice 1 lines. If you look in the local directory, you'll see a coverage_example.tc file. This was created at format time by the coverage predefined probe and is a binary representation of the above data. It contains the coverage data obtained during choice 1, so let's save it.

We typically use atcmerge to merge results from many .tc files and display the combined data, but it can also act on just one file. We need to save the current .tc file, and copying it would do, but we'll use atcmerge to demonstrate it a little:

atcmerge -b combined.tc coverage_example.tc

This merges the data in coverage_example.tc with that in combined.tc (which, in this case, means that combined.tc ends up with the same data). You can verify that this combined.tc file has the same data by just running atcmerge on it:

atcmerge combined.tc

Now run the application again using aprobe; but this time we won't invoke the coverage GUI since we already picked the functions we wanted:

aprobe -u coverage coverage_example

Select choice 2, then after the application ends re-run apformat:

apformat coverage_example

This time we get a different set of lines executed, as expected. Use the same atcmerge command to combine the data in combined.tc (from the first run) with the data from this run (in a new coverage_example.tc):

atcmerge -b combined.tc coverage_example.tc

Now we get the 100% coverage we wanted!

The on-line example in $APROBE/examples/learn/test_coverage extends this by illustrating how to use your own probes in combination with coverage.ual to ensure all lines are executed.

Test Coverage Report

The final output of the learn/test_coverage' example looks like this:

Test Coverage Summary Report

Test Run Summary:

Execution Start Time       End Time                   Original
------------------------  ------------------------ -------
Wed Apr 26 17:20:32 2000  Wed Apr 26 17:20:32 2000
Wed Apr 26 17:20:35 2000  Wed Apr 26 17:20:35 2000

End of Test Run Summary.

Subprogram Summary:

Module testoutfile:
------------------
                                     Lines   Lines
                      Total   Total   NOT     NOT    Coverage
Subprogram Name      Calls   Lines   Probed  Exec.  Percent
-----------------------------------------------------------
extern:"main()"        2      </nowiki>11       0       0     100
 
 -----------------------------------------------------------
 Total                  2      11       0       0     100
 
 End of Subprogram Summary.
 
 Subprogram Details:
 
 Module testoutfile:
 ------------------
 For subprogram: extern:"main()"
   Lines in subprogram: 11 (incl. executed 11 of 11 probed)
  Percent of lines executed:  100

End of Subprogram Details.

Example D-3. Test Coverage Summary Report

Branch Coverage Performance Issues

See Performance Issues for a general discussion of factors that affect performance.

The branch coverage probe is applied to the functions specified in the configuration file, so the run-time overhead is directly proportional to the number of calls to the functions selected and the number of branches executed in each function. The branch coverage probe simply increments an integer for each branch executed, so the amount of data is a constant multiple of the number of branches being covered, though additional snapshots increase this.

As branch coverage analysis is generally a unit-testing activity, it is mostly done on the small number of related functions under test, so performance should not be an issue. Application-wide branch coverage analysis during integration testing is not recommended. Branch coverage can result in a 30% overhead.

Abrmerge

The results of multiple runs of brcov.ual (and coverage.ual) over the same executable may be merged into a single "combined summary" report using abrmerge.

A merged summary report is useful when running many tests, each with different input data or perhaps different probes, to force all logic paths to be executed. abrmerge can also merge branch data files to accumulate data from multiple runs into a single file.

A simple example

The test application is very simple. It just prints out a mini-menu and waits for you to select 1 or 2. Depending on your selection, it does one of two actions. You can see this in the DoProcessing() function in coverage_example.c. We want to exercise every branch in this function -- we refer to this as getting 100% coverage.

Here is coverage_example.c which should be saved in the local directory:

/* This is a simple example to demonstrate using brcov.ual with coverage.ual */
#include <stdio.h>
#include <stdlib.h>

/* This is the function we are interested in. */
void DoProcessing (void)
{
   int  Option = 0;
   char Buffer [100];
   
   printf ("1. Print string #1\n");
   printf ("2. Print string #2\n");
   
   while (Option < 1 || Option > 2)
   {
      printf ("      Enter your choice (1 or 2): \n");
      fgets (Buffer, 99, stdin);
      Option = atoi (Buffer);
   }
      
   switch (Option)
   {
   case 1:
      printf ("This is string #1 - pretty exciting, huh?!\n");
      break;

   case 2:
      printf ("This is string #2 - it doesn't get any better than this!\n");
      break;
   }
}

int main (int argc, char **argv)
{
   DoProcessing ();
   return 0;
}

Note that, unlike some other uses of Aprobe, branch and line coverage requires you to be familiar with (the lines in) your source code to make most effective use of the probe.

We can build the example as follows:

cc -g -v -o coverage_example coverage_example.c

We can run the example as follows:

aprobe -u brcov -u coverage coverage_example

When the example application prompts you for a choice, enter the number 1. That will print out the first string and then the application will exit normally.

As before, aprobe stored the data in .apd files, so use apformat to format it:

apformat coverage_example

and abrmerge to merge and view it:

abrmerge *.brc *.c

Here's a portion of the report showing the line and branch coverage we've achieved:

pkg.adb:


+  - This line/branch/edge was not executed.
.  - This internal branch/edge was not executed but may not be significant.
*  - This line/branch/edge was not probed.

   Line    Edge/Branch   Line Cnt   Source
  ------   -----------   ---------  ------- - -
       1 :              :        : 
       2 :              :        : with Text_Io;
       3 :              :        : 
       4 :              :        : package body pkg is
       5 :              :        : 
       6 :              :        :   function Ident(i : integer) return integer is
       7 :              :        :   begin
       8 :              :        :      return i;
       9 :              :        :   end;
      10 :              :        : 
      11 :              :      2 :   procedure p1(I : in out integer) is
      12 :              :        :   begin
      13 :              :      2 :     if I = 0 then
          B       1x -> 16
      14 :              :      1 :       Text_Io.Put_Line("p1, 0!");
      15 :              :        :     else
      16 :              :      1 :       Text_Io.Put_Line("p1, not zero");
      17 :              :        :     end if;
      18 :              :      2 :   end p1;
      19 :              :        : 
      20 :              :      2 :   procedure p2(I : in out float) is
      21 :              :        :   begin
      22 :              :      2 :     if I < 0.000001 and then I > -0.000001 then
          B       1x -> 25
+         B       0x -> 25
      23 :              :      1 :       Text_Io.Put_Line("p2, 0!");
      24 :              :        :     else
      25 :              :      1 :       Text_Io.Put_Line("p2, not zero");
      26 :              :        :     end if;
      27 :              :      2 :   end p2;
      28 :              :        : 
      29 :              :        :   procedure td(i : in integer) is separate;
      30 :              :        : 
      31 :              :        : end pkg;
      32 :              :        : 

You can use the -q html option with abrmerge to produce HTML reports which use colors to highlight the coverage status of lines and branches as well as other information in hover test.

Using abrmerge

Sometimes we want to combine several test and branch coverage reports into a single report. Use abrmerge to accomplish this.

The above 2 lines that were not executed are specific to choice number 2. We could easily execute the lines of choice number 2 if we ran it again, but then we wouldn't be able to cover the choice 1 lines. If you look in the local directory, you'll see coverage_example.*.brc files. These were created at format time by the brcov and coverage predefined probes. These contain the branch and line coverage data obtained during choice 1, so let's save it.

We typically use abrmerge to merge results from many .brc files and display the combined data:

abrmerge -m combined.brc coverage_example.*.brc 

This "merges" the data in the input .brc files and outputs it in combined.brc.

Now run the application again using aprobe:

aprobe -u brcov -u coverage coverage_example

Select choice 2, then after the application ends re-run apformat:

apformat coverage_example

This time we get a different set of lines executed, as expected. Use the abrmerge command to combine the data in combined.brc (from the first run) with the data from this run (in a new coverage_example.*.brc):

abrmerge -m combined.brc coverage_example.*.brc combined.brc

Now we get the 100% coverage we wanted!

Branch Coverage Report

The branch coverage example report looks like this:

Aprobe Coverage Summary Report


Generated: 27 Nov 2017 09:19:05 -0500



--------------------------------------------------------------------------
Contents
--------------------------------------------------------------------------

TimeStamps
Overall Summary
Function TOC
File TOC
Function Summaries
Function Details
File Coverage



--------------------------------------------------------------------------
Timestmaps
--------------------------------------------------------------------------

TimeStamp:
Program   : /home/swn/aprobe/test/brcov/tnamec/main.exe
Host      : centos7
PID       : 13805
Start Time: 2017-11-27 09:19:06
End Time  : 2017-11-27 09:19:06



--------------------------------------------------------------------------
Overall Summary
--------------------------------------------------------------------------


                 Lines   Lines      Line                Branches  Branches            Branches  Branch
Total    Total   Not     Not        Coverage  Total     Not       Not        Branches Not       Coverage
Calls    Lines   Probed  Executed   Percent   Branches  Probed    Executed   Taken    Taken     Percent   Name
------- ------- -------- --------- ========= --------- --------- ---------- --------- --------- ======== --------- - -
     2       5        0         0       100         1         0          0         1         1      100  extern:"pkg.p1[1]"
     2       6        0         0       100         2         0          0         1         2      100  extern:"pkg.p2[1]"
    10      19        0         1        94         5         0          0         5         4      100  extern:"pkg.td[1]"

Note: Negative counts indicate the function/line was not instrumented.

          Funcs   Func                  Lines     Lines      Lines                 Branches   Branches              Branches   Branch
 Total    Not     Coverage    Total     Not       Not        Coverage   Total      Not        Not        Branches   Not        Coverage
 Funcs    Called  Percent     Lines     Probed    Executed   Percent    Branches   Probed     Executed   Taken      Taken      Percent 
-------- -------- ======== ---------- ---------- ---------- ========== ---------- ---------- ---------- ---------- ---------- ==========
      3        0      100         30          0          1         96          8          0          0          7          7        100



--------------------------------------------------------------------------
Function TOC
--------------------------------------------------------------------------

extern:"pkg.p1[1]" (/home/swn/aprobe/test/brcov/tnamec/pkg.adb)
extern:"pkg.p2[1]" (/home/swn/aprobe/test/brcov/tnamec/pkg.adb)
extern:"pkg.td[1]" (/home/swn/aprobe/test/brcov/tnamec/pkg-td.adb)


--------------------------------------------------------------------------
File TOC
--------------------------------------------------------------------------

pkg.ads
pkg.adb
pkg-td.adb
main.adb


--------------------------------------------------------------------------
Function Coverage Summaries
--------------------------------------------------------------------------


extern:"pkg.p1[1]"	(/home/swn/aprobe/test/brcov/tnamec/pkg.adb)


Calls:  2
-----

Test Names: [ test p1 ]
----------

Lines:
-----
          Lines   Probed  Not Probed   Hit 
         ------- -------- ---------- ----------
Count:        5         5         0         5
Percent:              100         0       100


Branches:
--------
          Branches    Probed      Hit       Taken     NotTaken
         ---------- ---------- ---------- ---------- ----------
Count:            1          1          1          1          1
Percent:                   100        100        100        100


extern:"pkg.p2[1]"	(/home/swn/aprobe/test/brcov/tnamec/pkg.adb)


Calls:  2
-----

Test Names: [ test p1, test p2 ]
----------

Lines:
-----
          Lines   Probed  Not Probed   Hit 
         ------- -------- ---------- ----------
Count:        6         6         0         6
Percent:              100         0       100


Branches:
--------
          Branches    Probed      Hit       Taken     NotTaken
         ---------- ---------- ---------- ---------- ----------
Count:            2          2          2          1          2
Percent:                   100        100         50        100


extern:"pkg.td[1]"	(/home/swn/aprobe/test/brcov/tnamec/pkg-td.adb)


Calls:  10
-----

Test Names: [ test p1, test p2 ]
----------

Lines:
-----
          Lines   Probed  Not Probed   Hit 
         ------- -------- ---------- ----------
Count:       19        19         0        18
Percent:              100         0        94

    Lines Not Executed:
    ------------------
    21 / pkg-td.adb

Branches:
--------
          Branches    Probed      Hit       Taken     NotTaken
         ---------- ---------- ---------- ---------- ----------
Count:            5          5          5          5          4
Percent:                   100        100        100         80



--------------------------------------------------------------------------
Function Coverage Details
--------------------------------------------------------------------------


extern:"pkg.p1[1]"	(/home/swn/aprobe/test/brcov/tnamec/pkg.adb)

Lines:
-----
      Line     I     Count      File  [ Test Names ]
    ---------- -  -----------  ---------------------
            11 Y :          2 : pkg.adb [ test p1 ]
            13 Y :          2 : pkg.adb [ test p1 ]
            14 Y :          1 : pkg.adb [ test p1 ]
            16 Y :          1 : pkg.adb [ test p1 ]
            18 Y :          2 : pkg.adb [ test p1 ]

Branches:
--------
     Offset  I      Hit       Taken     NotTaken    Line/File -> Line/File  [ Test Names ]
    -------- -   ---------- ---------- ----------   --------------------------------------
    0x00000b Y :          2          1          1 : 13 / pkg.adb -> 16 / pkg.adb [ test p1 ]

Edges:
------
       Count     Offset/Line/File -> Offset/Line/File [ Test Names ]
    ----------  ---------------------------------------------------

extern:"pkg.p2[1]"	(/home/swn/aprobe/test/brcov/tnamec/pkg.adb)

Lines:
-----
      Line     I     Count      File  [ Test Names ]
    ---------- -  -----------  ---------------------
            20 Y :          2 : pkg.adb [ test p2 ]
            22 Y :          2 : pkg.adb [ test p2 ]
            22 Y :          1 : pkg.adb [ test p2 ]
            23 Y :          1 : pkg.adb [ test p2 ]
            25 Y :          1 : pkg.adb [ test p2 ]
            27 Y :          2 : pkg.adb [ test p2 ]

Branches:
--------
     Offset  I      Hit       Taken     NotTaken    Line/File -> Line/File  [ Test Names ]
    -------- -   ---------- ---------- ----------   --------------------------------------
    0x000013 Y :          2          1          1 : 22 / pkg.adb -> 25 / pkg.adb [ test p2 ]
    0x000021 Y :          1          0          1 : 22 / pkg.adb -> 25 / pkg.adb [ test p2 ]

Edges:
------
       Count     Offset/Line/File -> Offset/Line/File [ Test Names ]
    ----------  ---------------------------------------------------

extern:"pkg.td[1]"	(/home/swn/aprobe/test/brcov/tnamec/pkg-td.adb)

Lines:
-----
      Line     I     Count      File  [ Test Names ]
    ---------- -  -----------  ---------------------
             4 Y :         10 : pkg-td.adb [ test p1, test p2 ]
             5 Y :         10 : pkg-td.adb [ test p1, test p2 ]
             6 Y :         10 : pkg-td.adb [ test p1, test p2 ]
             7 Y :         10 : pkg-td.adb [ test p1, test p2 ]
             8 Y :         10 : pkg-td.adb [ test p1, test p2 ]
             9 Y :         10 : pkg-td.adb [ test p1, test p2 ]
            11 Y :         10 : pkg-td.adb [ test p1, test p2 ]
            21 Y :          0 : pkg-td.adb [  ]
            11 Y :          8 : pkg-td.adb [ test p1, test p2 ]
            21 Y :          6 : pkg-td.adb [ test p2 ]
            13 Y :          1 : pkg-td.adb [  ]
            22 Y :          1 : pkg-td.adb [ test p1 ]
            15 Y :          1 : pkg-td.adb [ test p1 ]
            22 Y :          1 : pkg-td.adb [ test p1 ]
            17 Y :          1 : pkg-td.adb [ test p1 ]
            22 Y :          1 : pkg-td.adb [ test p2 ]
            19 Y :          1 : pkg-td.adb [ test p2 ]
            22 Y :          1 : pkg-td.adb [ test p2 ]
            23 Y :         10 : pkg-td.adb [ test p1, test p2 ]

Branches:
--------
     Offset  I      Hit       Taken     NotTaken    Line/File -> Line/File  [ Test Names ]
    -------- -   ---------- ---------- ----------   --------------------------------------
    0x000037 Y :         10          1          9 : 11 / pkg-td.adb -> 15 / pkg-td.adb [ test p2, test p1 ]
    0x00003c Y :          9          8          1 : 11 / pkg-td.adb -> 11 / pkg-td.adb [ test p2, test p1 ]
    0x000041 Y :          1          1          0 : 11 / pkg-td.adb -> 13 / pkg-td.adb [  ]
    0x000048 Y :          8          1          7 : 11 / pkg-td.adb -> 17 / pkg-td.adb [ test p2, test p1 ]
    0x00004d Y :          7          1          6 : 11 / pkg-td.adb -> 19 / pkg-td.adb [ test p2 ]

Edges:
------
       Count     Offset/Line/File -> Offset/Line/File [ Test Names ]
    ----------  ---------------------------------------------------


--------------------------------------------------------------------------
File Coverage
--------------------------------------------------------------------------

pkg.ads:


+  - This line/branch/edge was not executed.
.  - This internal branch/edge was not executed but may not be significant.
*  - This line/branch/edge was not probed.

   Line    Edge/Branch   Line Cnt   Source
  ------   -----------   ---------  ------- - -
       1 :              :        : 
       2 :              :        : package pkg is
       3 :              :        : 
       4 :              :        :   procedure p1(I : in out integer);
       5 :              :        :   procedure p2(I : in out float);
       6 :              :        : 
       7 :              :        :   procedure td(I : in integer);
       8 :              :        : 
       9 :              :        : end pkg;
      10 :              :        : 

pkg.adb:


+  - This line/branch/edge was not executed.
.  - This internal branch/edge was not executed but may not be significant.
*  - This line/branch/edge was not probed.

   Line    Edge/Branch   Line Cnt   Source
  ------   -----------   ---------  ------- - -
       1 :              :        : 
       2 :              :        : with Text_Io;
       3 :              :        : 
       4 :              :        : package body pkg is
       5 :              :        : 
       6 :              :        :   function Ident(i : integer) return integer is
       7 :              :        :   begin
       8 :              :        :      return i;
       9 :              :        :   end;
      10 :              :        : 
      11 :              :      2 :   procedure p1(I : in out integer) is
      12 :              :        :   begin
      13 :              :      2 :     if I = 0 then
          B       1x -> 16
      14 :              :      1 :       Text_Io.Put_Line("p1, 0!");
      15 :              :        :     else
      16 :              :      1 :       Text_Io.Put_Line("p1, not zero");
      17 :              :        :     end if;
      18 :              :      2 :   end p1;
      19 :              :        : 
      20 :              :      2 :   procedure p2(I : in out float) is
      21 :              :        :   begin
      22 :              :      2 :     if I < 0.000001 and then I > -0.000001 then
          B       1x -> 25
+         B       0x -> 25
      23 :              :      1 :       Text_Io.Put_Line("p2, 0!");
      24 :              :        :     else
      25 :              :      1 :       Text_Io.Put_Line("p2, not zero");
      26 :              :        :     end if;
      27 :              :      2 :   end p2;
      28 :              :        : 
      29 :              :        :   procedure td(i : in integer) is separate;
      30 :              :        : 
      31 :              :        : end pkg;
      32 :              :        : 

pkg-td.adb:


+  - This line/branch/edge was not executed.
.  - This internal branch/edge was not executed but may not be significant.
*  - This line/branch/edge was not probed.

   Line    Edge/Branch   Line Cnt   Source
  ------   -----------   ---------  ------- - -
       1 :              :        : 
       2 :              :        : separate(pkg)
       3 :              :        : 
       4 :              :     10 : procedure td(i : in integer) is
       5 :              :     10 :   i0 : integer := 0;
       6 :              :     10 :   i1 : integer := 1;
       7 :              :     10 :   f0 : float := 0.0;
       8 :              :     10 :   f1 : float := 1.0;
       9 :              :     10 :   l  : integer := 0;
      10 :              :        : begin
      11 :              :     10 :   case i is
          B       1x -> 15
          B       1x -> 13
          B       1x -> 17
          B       1x -> 19
      12 :              :        :   when 1 =>
      13 :              :      1 :     p1(i0);
      14 :              :        :   when 2 =>
      15 :              :      1 :     p1(i1);
      16 :              :        :   when 3 =>
      17 :              :      1 :     p2(f0);
      18 :              :        :   when 4 =>
      19 :              :      1 :     p2(f1);
      20 :              :        :   when others =>
      21 :              :      6 :     null;
      22 :              :      1 :   end case;
      23 :              :     10 : end td;
      24 :              :        :   

main.adb:


+  - This line/branch/edge was not executed.
.  - This internal branch/edge was not executed but may not be significant.
*  - This line/branch/edge was not probed.

   Line    Edge/Branch   Line Cnt   Source
  ------   -----------   ---------  ------- - -
       1 :              :        : 
       2 :              :        : with Text_Io;
       3 :              :        : with pkg;
       4 :              :        : 
       5 :              :        : procedure Main is
       6 :              :        : begin
       7 :              :        :    for i in 1..10 loop
       8 :              :        :       pkg.td(i);
       9 :              :        :    end loop;
      10 :              :        : end Main;
      11 :              :        : 

Example D-3. Branch Coverage Summary Report

For more information about the Branch Coverage Report see Branch_Coverage_Cheat_Sheet.

Here is and example of the HTML branch coverage report:

Brcov example HTML.jpeg