X-Git-Url: https://gerrit.o-ran-sc.org/r/gitweb?a=blobdiff_plain;f=test%2FREADME;h=eb243debe0a60cdba5ba0bcf6384a541e7ffad82;hb=f7d44570f8de6e15f768e8e2d9b6061cd0bff11f;hp=85f96eda8f7564eb13275be07717a9c8e30c60a0;hpb=fd9cc7a5b3355146388ebdf4d558cb284c66c5f1;p=ric-plt%2Flib%2Frmr.git diff --git a/test/README b/test/README index 85f96ed..eb243de 100644 --- a/test/README +++ b/test/README @@ -1,21 +1,25 @@ Unit test -The means to unit testing the RMr library is contained in -this directory. It is somewhat difficult to accurately generate -coverage information for parts of the library because the library -is a fair amount of static functions (to keep them from being -visible to the user programme). +This directory contains the unit test support for the RMr +library. The basic test is run with the follwing command: -To run the tests: - ksh unit_test.sh [specific-test] + ksh unit_test.ksh -If a specific test (e.g. ring_test.c) is not given on the command line, -all *_test.c files are sussed out and an attempt to build and run them -is made by the script. +To run a specific test (e.g. ring_test.c) run: + ksh unit_test.ksh ring_test.c -Output is an interpretation of the resulting gcov output (in more useful -and/or easier to read format). For example: +The script runs the unit test(s) given, and if they pass then +runs an analysis on the .gcov files generated to generate +coverage information. By default, pass/fail of the test is +based only on the success or failure of the unit tests which +are testing functionality. The unit test script can report +an overall failure if coverage is below the indicated threshold +when given the strict option (-s). + +The analysis of .gcov files generates output shown below which +is thought to be more straight forward than the typical stuff +gcov produces: unit_test.ksh ring_test.c ring_test.c -------------------------------------- @@ -26,18 +30,39 @@ ring_test.c -------------------------------------- [PASS] 91% ../src/common/src/ring_static.c -The output shows, for each function, the coverage (column 2) and an +The output shows, for each function, the coverage (column 2) and an interpretation (ok or low) wthin an overall pass or fail. -File Names -The unit test script will find all files named *_test.c and assume that -they can be compiled and executed using the local Makefile. Files -which are needed by these programmes (e.g. common functions) are expected -to reside in this directory as test_*.c and test_*.h files and should -be directly included by the test programmes (not built and linked). This -allows the unit test script to isngore the functions, and files, when -generating coverage reports. +Because of the static nature of the RMr library, tests with the +intent of providing coverage information, as opposed just to providing +functional verification, are a bit trickier. To that end, the test +files in this directory are organised with three file name formats: + + test_*.c tools for testing, not tests + + *_test.c main test programmes which can be compiled in + a stand-alone manner (e.g. gcc foo_test.c) + + *_static_test.c Test functions which are real tests and are + included by one or more stand-alone driver. + +The unit_test script will search only for *_test.c and will ignore +*_static_test.c files when building it's list for testing. + + +Use the command 'unit_test.ksh -?' to see the usage information +and complete set of options available. + + +Merging .gcov files +As some unit test programmes may not be able/designed to cover all +of a module completely, the script will merge each .gcov prooduced +with the prior, matching, file after each pass. The result is a +final .gcov file that shows the coverage of the module over all +tests. This allows a true coverage value to be determined while +permitting more simple tests to be used without the requirement of +each test to cover everything. Discounting @@ -51,13 +76,13 @@ option is given, an augmented coverage listing is saved in .dcov which shows the discounted lines with a string of equal signs (====) rather than the gcov hash string (###). -The discount check is applied only if an entire module has a lower -than accepted coverage rate, and can be forced for all modules with +The discount check is applied only if an entire module has a lower +than accepted coverage rate, and can be forced for all modules with the -f option. To illustrate, the following code checks the return from the system library strdup() call which is very unlikely to fail under test without -going to extremes and substituting for the system lib. Thus, the +going to extremes and substituting for the system lib. Thus, the block which checks for a nil pointer has been discounted: -: 354: @@ -67,16 +92,29 @@ block which checks for a nil pointer has been discounted: =====: 358: return 0; -: 359: } +A final discount pass is made on all merged .gcov files after all unit +tests are finished. This final pass should indicate the true discounted +coverage + Target Coverage By default, a target coverage of 80% is used. For some modules this may -be impossible to achieve, so to prevent always failing these modules +be impossible to achieve, so to prevent always failing these modules may be listed in the .targets file with their expected minimum coverage. Module names need to be qualified (e.g. ../src/common/src/foo.c. +Sonar XML Files +If the -x option is given on the unit test script, then two XML files +will be generated for Sonar. These are gcov.xml and dcov.xml and are +the coverage information as reflected in the merged .gcov files and in +the .dcov files produced during the final pass. The XML files should +be uploaded to Sonar only after a complete unit test run is made, otherwise +the coverage information may be lacking (reflecting only the last test +executed and not a full complement of tests). + ----------------------------------------------------------------------- A note about ksh (A.K.A Korn shell, or kshell) -Ksh is preferred for more complex scripts such as the unit test +Ksh is preferred for more complex scripts such as the unit test script as it does not have some of the limitations that bash -(and other knock-offs) have. +(and other knock-offs) have.