Unit test
-The means to unit testing the RMr library is contained in
-this directory. It is somewhat difficult to accurately generate
-coverage information for parts of the library because the library
-is a fair amount of static functions (to keep them from being
-visible to the user programme).
+This directory contains the unit test support for the RMr
+library. The basic test is run with the follwing command:
-To run the tests:
- ksh unit_test.sh [specific-test]
+ ksh unit_test.ksh
-If a specific test (e.g. ring_test.c) is not given on the command line,
-all *_test.c files are sussed out and an attempt to build and run them
-is made by the script.
+To run a specific test (e.g. ring_test.c) run:
+ ksh unit_test.ksh ring_test.c
-Output is an interpretation of the resulting gcov output (in more useful
-and/or easier to read format). For example:
+The script runs the unit test(s) given, and if they pass then
+runs an analysis on the .gcov files generated to generate
+coverage information. By default, pass/fail of the test is
+based only on the success or failure of the unit tests which
+are testing functionality. The unit test script can report
+an overall failure if coverage is below the indicated threshold
+when given the strict option (-s).
+
+The analysis of .gcov files generates output shown below which
+is thought to be more straight forward than the typical stuff
+gcov produces:
unit_test.ksh ring_test.c
ring_test.c --------------------------------------
[PASS] 91% ../src/common/src/ring_static.c
-The output shows, for each function, the coverage (column 2) and an
+The output shows, for each function, the coverage (column 2) and an
interpretation (ok or low) wthin an overall pass or fail.
-File Names
-The unit test script will find all files named *_test.c and assume that
-they can be compiled and executed using the local Makefile. Files
-which are needed by these programmes (e.g. common functions) are expected
-to reside in this directory as test_*.c and test_*.h files and should
-be directly included by the test programmes (not built and linked). This
-allows the unit test script to isngore the functions, and files, when
-generating coverage reports.
+Because of the static nature of the RMr library, tests with the
+intent of providing coverage information, as opposed just to providing
+functional verification, are a bit trickier. To that end, the test
+files in this directory are organised with three file name formats:
+
+ test_*.c tools for testing, not tests
+
+ *_test.c main test programmes which can be compiled in
+ a stand-alone manner (e.g. gcc foo_test.c)
+
+ *_static_test.c Test functions which are real tests and are
+ included by one or more stand-alone driver.
+
+The unit_test script will search only for *_test.c and will ignore
+*_static_test.c files when building it's list for testing.
+
+
+Use the command 'unit_test.ksh -?' to see the usage information
+and complete set of options available.
+
+
+Merging .gcov files
+As some unit test programmes may not be able/designed to cover all
+of a module completely, the script will merge each .gcov prooduced
+with the prior, matching, file after each pass. The result is a
+final .gcov file that shows the coverage of the module over all
+tests. This allows a true coverage value to be determined while
+permitting more simple tests to be used without the requirement of
+each test to cover everything.
Discounting
shows the discounted lines with a string of equal signs (====) rather
than the gcov hash string (###).
-The discount check is applied only if an entire module has a lower
-than accepted coverage rate, and can be forced for all modules with
+The discount check is applied only if an entire module has a lower
+than accepted coverage rate, and can be forced for all modules with
the -f option.
To illustrate, the following code checks the return from the system
library strdup() call which is very unlikely to fail under test without
-going to extremes and substituting for the system lib. Thus, the
+going to extremes and substituting for the system lib. Thus, the
block which checks for a nil pointer has been discounted:
-: 354:
=====: 358: return 0;
-: 359: }
+A final discount pass is made on all merged .gcov files after all unit
+tests are finished. This final pass should indicate the true discounted
+coverage
+
Target Coverage
By default, a target coverage of 80% is used. For some modules this may
-be impossible to achieve, so to prevent always failing these modules
+be impossible to achieve, so to prevent always failing these modules
may be listed in the .targets file with their expected minimum coverage.
Module names need to be qualified (e.g. ../src/common/src/foo.c.
+Sonar XML Files
+If the -x option is given on the unit test script, then two XML files
+will be generated for Sonar. These are gcov.xml and dcov.xml and are
+the coverage information as reflected in the merged .gcov files and in
+the .dcov files produced during the final pass. The XML files should
+be uploaded to Sonar only after a complete unit test run is made, otherwise
+the coverage information may be lacking (reflecting only the last test
+executed and not a full complement of tests).
+
-----------------------------------------------------------------------
A note about ksh (A.K.A Korn shell, or kshell)
-Ksh is preferred for more complex scripts such as the unit test
+Ksh is preferred for more complex scripts such as the unit test
script as it does not have some of the limitations that bash
-(and other knock-offs) have.
+(and other knock-offs) have.
Code Review / ric-plt / lib / rmr.git / blobdiff