Difference between revisions of "Coverage data merging"
m (→Coverage Data Merging) |
|||
(One intermediate revision by one other user not shown) | |||
Line 21: | Line 21: | ||
aprobe -u coverage test1.exe | aprobe -u coverage test1.exe | ||
apformat test1.apd | apformat test1.apd | ||
− | mv test1.tc test1- | + | mv test1.tc test1-run2.tc |
mv test1.exe.bc test1.exe-run2.bc | mv test1.exe.bc test1.exe-run2.bc | ||
Latest revision as of 18:12, 8 May 2017
If you need to merge coverage data from several test runs with the same or different test applications/drivers you can use the atcmerge and/or apbc tools.
The coverage UAL produces two types of output:
- .tc - test coverage file
- .bc - detail coverage file
The files are produced by running apformat on the .apd file which is produced by running aprobe. For example:
aprobe -u coverage test1.exe apformat test1.apd
Note that if you run the same test application/driver multiple times, you need to save the .tc and .bc files between each apformat operation:
aprobe -u coverage test1.exe apformat test1.apd mv test1.tc test1-run1.tc mv test1.exe.bc test1.exe-run1.bc
aprobe -u coverage test1.exe apformat test1.apd mv test1.tc test1-run2.tc mv test1.exe.bc test1.exe-run2.bc
You can merge the .tc coverage data with the following command:
atcmerge -b all.tc test1.tc test2.tc test3.tc
This will merge all the .tc files into a new file all.tc. Atcmerge will also generate a coverage summary report of the merged coverage data to stdout.
You don't need to merge .bc data before displaying. You can use the following command to display merged .bc coverage data:
apbc test1.exe.bc test2.exe.bc test3.exe.bc file1.adb file2.adb file3.adb
This will merge the detailed coverage data from the three .bc files and display line counts associated with the source lines of the three Ada source files.
In Aprobe 4.4.6d you can produce a merged .bc file using atcmerge at the same time you merge the .tc files:
atcmerge -b all.tc -d all.bc test1.tc test2.tc test3.tc
This will merge all the .tc files into all.tc and produce a new .bc file all.bc wth the merged data.
Example
The following is an example of merging coverage data. In this example the code under test in the Ada package named pkg. Three test applications (test1.exe test2.exe, and test3.exe) were created to exercise this code, each one performing a set of tests. The coverage data from running all three test applications will then be merged to produce a single summary report and source report.
First, we make three test runs. (We don't have to worry about .tc or .bc files being overwritten because the test applications have unique names.)
aprobe -u coverage test1.exe apformat test1.apd aprobe -u coverage test2.exe apformat test2.apd aprobe -u coverage test3.exe apformat test3.apd
This will have produce six coverage data files:
- test1.tc and test1.exe.bc
- test2.tc and test2.exe.bc
- test3.tc and test3.exe.bc
We can merge the coverage data and produce a overall summary report with:
atcmerge -b all.tc test1.tc test2.tc test3.tc > all.rpt
We can merge the detail coverage data and produce a source report with:
apbc test1.exe.bc test2.exe.bc test3.exe.bc pkg.ads pkg.adb
Here are the reports:
Test Coverage Summary Report PID 790542 - Test Run Summary: Execution Start Time End Time Original Report File ------------------------ ------------------------ ------------------------ Wed Mar 29 15:52:41 2017 Wed Mar 29 15:52:41 2017 main1.tc Wed Mar 29 15:52:42 2017 Wed Mar 29 15:52:42 2017 main2.tc Wed Mar 29 15:52:42 2017 Wed Mar 29 15:52:42 2017 main3.tc End of Test Run Summary. List of application modules for this report: main1.exe main2.exe main3.exe Subprogram Summary: Merged Coverage Data Module <multiple application modules>: ------------------------------------- Lines Lines Total Total Not Not Coverage Subprogram Name Calls Lines Probed Executed Percent -------------------------------------------------------------------------------------- extern:"pkg.td.body_elab" in main3.exe 3 0 0 0 - extern:"pkg.td[1]" in main3.exe 4 11 0 1 91 extern:"pkg.body_elab" in main3.exe 3 5 0 0 100 extern:"pkg.s1[1b]" in main3.exe 2 1 0 0 100 extern:"pkg.s2[1b]" in main3.exe 2 1 0 0 100 extern:"pkg.put[1b]" in main3.exe 2 2 0 0 100 extern:"pkg.get[1b]" in main3.exe 2 2 0 0 100 extern:"pkg.p1[1]" in main3.exe 2 6 0 0 100 extern:"pkg.p2[1]" in main3.exe 2 6 0 0 100 extern:"pkg.spec_elab" in main3.exe 3 4 0 0 100 extern:"pkg.s1[1]" in main3.exe 2 1 0 0 100 extern:"pkg.s2[1]" in main3.exe 2 1 0 0 100 extern:"pkg.put[1]" in main3.exe 2 2 0 0 100 extern:"pkg.get[1]" in main3.exe 2 2 0 0 100 -------------------------------------------------------------------------------------- Total 33 44 0 1 98 End of Subprogram Summary. Subprogram Details: Merged Coverage Data Module <multiple application modules>: ------------------------------------- For 3 calls to subprogram: extern:"pkg.td.body_elab" in main3.exe Lines in subprogram: 0 For 4 calls to subprogram: extern:"pkg.td[1]" in main3.exe Lines in subprogram: 11 total (incl. executed 10 of 11 probed lines) Percent of lines executed: 91 The following probed lines were not executed: Line: 20 (/home/swn/aprobe/test/faq/covada/pkg-td.adb) For 3 calls to subprogram: extern:"pkg.body_elab" in main3.exe Lines in subprogram: 5 total (incl. executed 5 of 5 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.s1[1b]" in main3.exe Lines in subprogram: 1 total (incl. executed 1 of 1 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.s2[1b]" in main3.exe Lines in subprogram: 1 total (incl. executed 1 of 1 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.put[1b]" in main3.exe Lines in subprogram: 2 total (incl. executed 2 of 2 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.get[1b]" in main3.exe Lines in subprogram: 2 total (incl. executed 2 of 2 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.p1[1]" in main3.exe Lines in subprogram: 6 total (incl. executed 6 of 6 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.p2[1]" in main3.exe Lines in subprogram: 6 total (incl. executed 6 of 6 probed lines) Percent of lines executed: 100 For 3 calls to subprogram: extern:"pkg.spec_elab" in main3.exe Lines in subprogram: 4 total (incl. executed 4 of 4 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.s1[1]" in main3.exe Lines in subprogram: 1 total (incl. executed 1 of 1 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.s2[1]" in main3.exe Lines in subprogram: 1 total (incl. executed 1 of 1 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.put[1]" in main3.exe Lines in subprogram: 2 total (incl. executed 2 of 2 probed lines) Percent of lines executed: 100 For 2 calls to subprogram: extern:"pkg.get[1]" in main3.exe Lines in subprogram: 2 total (incl. executed 2 of 2 probed lines) Percent of lines executed: 100 End of Subprogram Details.
pkg-td.adb: Hit Line Source --- ---- ------ - - 0001: 0002: separate(pkg) 0003: 0004: procedure td(i : in integer) is 4 0005: i0 : integer := 0; 4 0006: i1 : integer := 1; 4 0007: f0 : float := 0.0; 4 0008: f1 : float := 1.0; 0009: begin 4 0010: case i is 0011: when 1 => 1 0012: p1(i0); 0013: when 2 => 1 0014: p1(i1); 0015: when 3 => 1 0016: p2(f0); 0017: when 4 => 1 0018: p2(f1); 0019: when others => 0 0020: Text_Io.Put_Line("bad test number"); 0021: end case; 4 0022: end td; 0023: pkg.adb: Hit Line Source --- ---- ------ - - 0001: 0002: with Text_Io; 0003: 0004: package body pkg is 0005: 15 0006: package g2 is new gen(float); 0007: 3 0008: procedure p1(I : in out integer) is 0009: begin 2 0010: g1.put(i); 2 0011: if I = 0 then 1 0012: Text_Io.Put_Line("p1, 0!"); 0013: else 1 0014: Text_Io.Put_Line("p1, not zero"); 0015: end if; 2 0016: I := g1.get + 1; 2 0017: end p1; 0018: 3 0019: procedure p2(I : in out float) is 0020: begin 2 0021: g2.put(i); 2 0022: if I < 0.000001 and then I > -0.000001 then 1 0023: Text_Io.Put_Line("p2, 0!"); 0024: else 1 0025: Text_Io.Put_Line("p2, not zero"); 0026: end if; 2 0027: I := g2.get + 1.0; 2 0028: end p2; 0029: 3 0030: procedure td(i : in integer) is separate; 0031: 3 0032: end pkg; 0033: pkg.ads: Hit Line Source --- ---- ------ - - 0001: 0002: with gen; 0003: 0004: package pkg is 0005: 15 0006: package g1 is new gen(integer); 0007: 3 0008: procedure p1(I : in out integer); 3 0009: procedure p2(I : in out float); 0010: 3 0011: procedure td(I : in integer); 0012: 0013: end pkg; 0014: