From: YongchaoWu Date: Fri, 17 Jan 2020 12:58:51 +0000 (+0100) Subject: Improve auto test for policy agent X-Git-Tag: 1.0.1~40 X-Git-Url: https://gerrit.o-ran-sc.org/r/gitweb?a=commitdiff_plain;h=ffde6eb81f80f6ebd8e7cb48b9acacd0733dd7fe;p=nonrtric.git Improve auto test for policy agent Auto test for policy agent remote/local image Return basic health check in the printout Add test description files. Issue-ID: NONRTRIC-79 Signed-off-by: YongchaoWu Change-Id: I1dbdfbcc2568a1b46dae5d42ebcb8329c818f3ac --- diff --git a/near-rt-ric-simulator/auto-test/FTC1.sh b/near-rt-ric-simulator/auto-test/FTC1.sh index 180682b0..f49a1797 100755 --- a/near-rt-ric-simulator/auto-test/FTC1.sh +++ b/near-rt-ric-simulator/auto-test/FTC1.sh @@ -1,9 +1,11 @@ #!/usr/bin/env bash -TC_ONELINE_DESCR="Auto test for policy agent with simulator" +TC_ONELINE_DESCR="Auto test for policy agent refreshing configurations from consul/cbs" . ../common/testcase_common.sh $1 $2 +#### TEST BEGIN #### + clean_containers start_simulators diff --git a/near-rt-ric-simulator/auto-test/README.md b/near-rt-ric-simulator/auto-test/README.md new file mode 100644 index 00000000..676f079a --- /dev/null +++ b/near-rt-ric-simulator/auto-test/README.md @@ -0,0 +1,69 @@ +## Automated test Description +This auto-test repo stores test script for automated test cases for policy agent. +Each of the testcase script will bring up a containerized test enviroment for Policy Agent, +CBS, consul, and simulator(TBD) + +### Overveiw + +Right now, test cases are written in bash scripts. \ +Each test case script(ex. `FTC1.sh)` will call functions defined in `../common`. \ +The environment vriables are set in`test_env.sh`. \ +The automated test support both local build policy agent image testing and remote image stored in Nexus. +``` +# Lcal image +export POLICY_AGENT_LOCAL_IMAGE=o-ran-sc/nonrtric-policy-agent +# Remote image +export POLICY_AGENT_REMOTE_IMAGE=nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-policy-agent +``` +### Test Cases Description(more TBD) +`FTC1.sh`: Test policy-agent can refresh configurations from consul + +### Logs +All log files are stored at `logs/`. \ +The logs include the application.log and the container log from policy agent, the container logs from each simulator and the +test case log (same as the screen output). \ +In the test cases the logs are stored with a prefix so the logs can be stored at different steps during the test. +All test cases contains an entry to save all logs with prefix 'END' at the end of each test case. + +### Manual +Test case command: +``` +./.sh local | remote + +Discription: +local: test image: POLICY_AGENT_LOCAL_IMAGE=o-ran-sc/nonrtric-policy-agent +remote: test image: nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-policy-agent +``` + +### Test case file +A test case file contains a number of steps to verify a certain functionality. +A description of the test case should be given to the ``TC_ONELINE_DESCR`` var. The description will be printed in +the test result. + +The empty template for a test case files looks like this: + +(Only the parts noted with < and > shall be changed.) + +----------------------------------------------------------- +``` +#!/usr/bin/env bash + +TC_ONELINE_DESCR="" + +. ../common/testcase_common.sh $1 + +#### TEST BEGIN #### + + + + + +#### TEST COMPLETE #### + +store_logs END + +``` +----------------------------------------------------------- + +The ../common/testcase_common.sh contains all functions needed for the test case file. See the README.md file in +the ../common dir for a description of all available functions. \ No newline at end of file diff --git a/near-rt-ric-simulator/common/README.md b/near-rt-ric-simulator/common/README.md new file mode 100644 index 00000000..37d81ec3 --- /dev/null +++ b/near-rt-ric-simulator/common/README.md @@ -0,0 +1,32 @@ +## Description for common test scripts + +`test_env.sh` \ +Common env variables for test in the auto-test dir. +Used by the auto test cases/suites but could be used for other test script as well. + +`testcase_common.sh` \ +Common functions for auto test cases in the auto-test dir. +A subset of the functions could be used in other test scripts as well. + +###Descriptions of functions in testcase_common.sh + +`clean_containers` \ +Stop and remove all containers including policy agent apps and simulators + +`start_simulators` \ +Start all simulators in the simulator group + +`consul_config_app ` \ +Configure consul with json file with app config for a policy agent instance using the policy agent +instance id and the json file. + +`start_policy_agent` \ +Start the policy agent application. + +`check_policy_agent_logs` +Check the Policy agent application log for WARN and ERR messages and print the count. +`store_logs` +Store all Policy agent app and simulators log to the test case log dir. All logs get a prefix to +separate logs stored at different steps in the test script. +If logs need to be stored in several locations, use different prefix to easily identify the location +when the logs where taken. diff --git a/near-rt-ric-simulator/common/test_env.sh b/near-rt-ric-simulator/common/test_env.sh index 363f8794..b307a690 100755 --- a/near-rt-ric-simulator/common/test_env.sh +++ b/near-rt-ric-simulator/common/test_env.sh @@ -1,15 +1,18 @@ #!/usr/bin/env bash +# Set the images for the Policy agent app to use for the auto tests. Do not add the image tag. +# # Local image and tag, shall point to locally built image (non-nexus path) export POLICY_AGENT_LOCAL_IMAGE=o-ran-sc/nonrtric-policy-agent - +# Remote image +export POLICY_AGENT_REMOTE_IMAGE=nexus3.o-ran-sc.org:10004/o-ran-sc/nonrtric-policy-agent # Common env var for auto-test. POLICY_AGENT_PORT=8081 POLICY_AGENT_LOGPATH="/var/log/policy-agent/application.log" #Path the application log in the policy agent container -DOCKER_SIM_NWNAME="nonrtric-docker-net" #Name of docker private network -CONSUL_HOST="consul-server" #Host name of consul -CONSUL_PORT=8500 #Port number of consul -CONFIG_BINDING_SERVICE="config-binding-service" #Host name of CBS -PA_APP_BASE="policy-agent" +DOCKER_SIM_NWNAME="nonrtric-docker-net" #Name of docker private network +CONSUL_HOST="consul-server" #Host name of consul +CONSUL_PORT=8500 #Port number of consul +CONFIG_BINDING_SERVICE="config-binding-service" #Host name of CBS +PA_APP_BASE="policy-agent" #Base name for policy agent container diff --git a/near-rt-ric-simulator/common/testcase_common.sh b/near-rt-ric-simulator/common/testcase_common.sh index 6f2097e9..7a8d8dca 100755 --- a/near-rt-ric-simulator/common/testcase_common.sh +++ b/near-rt-ric-simulator/common/testcase_common.sh @@ -4,12 +4,16 @@ echo "Test case started as: ${BASH_SOURCE[$i+1]} "$1 $2 +# This is a script that contains all the functions needed for auto test +# Arg: local | remote + STARTED_POLICY_AGENT="" #Policy agent app names added to this var to keep track of started container in the script START_ARG=$1 IMAGE_TAG="1.0.0-SNAPSHOT" +IMAGE_TAG_REMOTE="1.0.0" if [ $# -lt 1 ] || [ $# -gt 2 ]; then - echo "Expected arg: local [] ]| remote [] ]| remote-remove []] | manual-container | manual-app" + echo "Expected arg: local [] | remote [] " exit 1 elif [ $1 == "local" ]; then if [ -z $POLICY_AGENT_LOCAL_IMAGE ]; then @@ -17,6 +21,12 @@ elif [ $1 == "local" ]; then exit 1 fi POLICY_AGENT_IMAGE=$POLICY_AGENT_LOCAL_IMAGE":"$IMAGE_TAG +elif [ $1 == "remote" ]; then + if [ -z $POLICY_AGENT_REMOTE_IMAGE ]; then + echo "POLICY_AGENT_REMOTE_IMAGE not set in test_env" + exit 1 + fi + POLICY_AGENT_IMAGE=$POLICY_AGENT_REMOTE_IMAGE":"$IMAGE_TAG_REMOTE fi # Set a description string for the test case @@ -72,17 +82,31 @@ fi echo "" if [ $1 != "manual-container" ] && [ $1 != "manual-app" ]; then - echo -e "Policy agent image tag set to: \033[1m" $IMAGE_TAG"\033[0m" - echo "Configured image for policy agent app(s) (${1}): "$POLICY_AGENT_LOCAL_IMAGE - tmp_im=$(docker images ${POLICY_AGENT_LOCAL_IMAGE} | grep -v REPOSITORY) + #echo -e "Policy agent image tag set to: \033[1m" $IMAGE_TAG"\033[0m" + echo "Configured image for policy agent app(s) (${1}): "$POLICY_AGENT_IMAGE + tmp_im=$(docker images ${POLICY_AGENT_IMAGE} | grep -v REPOSITORY) if [ $1 == "local" ]; then if [ -z "$tmp_im" ]; then - echo "Local image (non nexus) "$POLICY_AGENT_LOCAL_IMAGE" does not exist in local registry, need to be built" + echo "Local image (non nexus) "$POLICY_AGENT_IMAGE" does not exist in local registry, need to be built" exit 1 else echo -e "Policy agent local image: \033[1m"$tmp_im"\033[0m" - echo "If the policy agen image seem outdated, rebuild the image and run the test again." + echo "If the policy agent image seem outdated, rebuild the image and run the test again." + fi + elif [ $1 == "remote" ]; then + if [ -z "$tmp_im" ]; then + echo "Pulling policy agent image from nexus: "$POLICY_AGENT_IMAGE + docker pull $POLICY_AGENT_IMAGE > /dev/null + tmp_im=$(docker images ${POLICY_AGENT_IMAGE} | grep -v REPOSITORY) + if [ -z "$tmp_im" ]; then + echo "Image could not be pulled" + exit 1 + fi + echo -e "Policy Agent image: \033[1m"$tmp_im"\033[0m" + else + echo -e "Policy Agent image: \033[1m"$tmp_im"\033[0m" + echo "!! If the Policy agent image seem outdated, consider removing it from your docker registry and run the test again." fi fi fi @@ -129,7 +153,7 @@ start_policy_agent() { appname=$PA_APP_BASE - if [ $START_ARG == "local" ] ; then + if [ $START_ARG == "local" ] || [ $START_ARG == "remote" ]; then __start_policy_agent_image $appname fi } @@ -171,6 +195,7 @@ __start_policy_agent_image() { result="$(__do_curl http://127.0.0.1:${localport}/status)" if [ $? -eq 0 ]; then echo "Policy Agent ${appname} responds to service status: " $result + echo "Policy Agent is alive." pa_st=true break else diff --git a/near-rt-ric-simulator/simulator-group/consul_cbs/config.json b/near-rt-ric-simulator/simulator-group/consul_cbs/config.json index f75b2c5f..d63d8607 100644 --- a/near-rt-ric-simulator/simulator-group/consul_cbs/config.json +++ b/near-rt-ric-simulator/simulator-group/consul_cbs/config.json @@ -1,13 +1,21 @@ - { - "//description": "Application configuration", - "ric": [ - { - "name": "ric1", - "baseUrl": "http://localhost:8080/", - "managedElementIds": [ - "kista_1", - "kista_2" - ] - } - ] - } +{ + "//description": "Application configuration", + "ric": [ + { + "name": "ric1", + "baseUrl": "http://localhost:8080/", + "managedElementIds": [ + "kista_1", + "kista_2" + ] + }, + { + "name": "ric2", + "baseUrl": "http://localhost:8081/", + "managedElementIds": [ + "kista_3", + "kista_4" + ] + } + ] +}