From: Martin Skorupski Date: Fri, 14 Feb 2025 15:45:00 +0000 (+0100) Subject: update docs for solution deployment X-Git-Url: https://gerrit.o-ran-sc.org/r/gitweb?a=commitdiff_plain;h=d225e8a28659ca6def8007852714b46dcc94b5e6;p=oam.git update docs for solution deployment - update deployment page Issue-ID: OAM-430 Change-Id: I584760d96e7201c7ed2178601ea68a81a9939f35 Signed-off-by: Martin Skorupski --- diff --git a/docs/deployment.rst b/docs/deployment.rst index 4b22089..3628652 100644 --- a/docs/deployment.rst +++ b/docs/deployment.rst @@ -5,280 +5,422 @@ Deployment ========== -OAM Components are deployed in different context and for different use cases. -Please see the Kubernetes deployment within the `it/deb `__ repository and its description in `O-RAN-SC wiki `__. +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. SPDX-License-Identifier: CC-BY-4.0 +.. Copyright (C) 2025 highstreet technologies USA Corp. + +Service Management and Orchestration (SMO) OAM Deployment and Configuration +=========================================================================== + + +This document focuses on a docker-compose deployment solution for SMO/OAM +Components. Such deployment should be used for research and development. + -The main OAM Components are: +Introduction +------------ -- OAM-Controller from ONAP SDN-R using OpenDaylight for O1-NETCONF and OpenFronthaul-Management-NETCONF support. -- VES-Collector from ONAP DCAE for processing O1-VES messages. -- Message-router from ONAP DMaaP using Kafka -- Identity service from Keycloak for AAA support. +With respect to Operation and Maintenance (OAM), the SMO implements the +O1-interface and OpenFronthaul Management-Plane consumers. According to +the O-RAN OAM Architecture and the O-RAN OAM Interface Specification, the +SMO implements a NETCONF Client for configuration and a HTTP/REST/VES +server for receiving all kinds of events in VES format. +The O-RAN-SC OAM deployment contains an OpenDaylight-based NETCONF +client and an ONAP VES Collector. Kafka is used as a message router for +communication between the components. The Keycloak implementation offers +an Identity service, while traefik acts as a reverse proxy to terminate +all incoming https traffic. For storing data in a persistent way, the +implementation of the mariaDB project is used. -This page focus on a lightweight deployment using docker-compose on an Ubuntu system for experience with SMO OAM Components and/or development. +SMO OAM Components +------------------ +This docker-compose file starts a pre-configured, self-contained SDN-R +solution with the following components: + +- **Identity** + ... representing a Keycloak-based identity service for centralized + user management. Please note that the implementation does not support + IPv6. Therefore, its own network is required called ``DMZ``. + +- **Controller** single node instance + ... representing the NETCONF consumer on the Service Management and + Orchestration framework (SMO) for O-RAN O1 interface and/or O-RAN + OpenFronthaul Management Plane and/or other NETCONF/YANG schemas + implemented by the OpenDaylight project. + +- **VES collector** + ... representing the VES (REST) provider at SMO for all kinds of + events. + +- **Messages** + ... representing the SMO MessageRouter component, includes + message-router. + +- **Gateway** + ... representing a reverse proxy terminating TLS traffic (https, + NETCONF). Prerequisites ------------- -The following command are used to verify your system and it installed software package. -In general new version then display here should be fine.:: +Resources +~~~~~~~~~ + +The solution was tested on a VM with: + +- 4x Core +- 16 GBit RAM +- 50 Gbit Storage + +Operating (HOST) System +~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: console $ cat /etc/os-release | grep PRETTY_NAME - PRETTY_NAME="Ubuntu 20.04.2 LTS" + PRETTY_NAME="Ubuntu 24.04.1 LTS" + +Docker +~~~~~~ + +.. code-block:: console $ docker --version - Docker version 20.10.7, build 20.10.7-0ubuntu1~20.04.2 + Docker version 27.5.1, build 9f9e405 + +Docker configuration for IPv6 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In case you prefer IPv6 support the docker configuration must be modified. + +Please see: +https://docs.docker.com/engine/daemon/ipv6/ + +1. Edit /etc/docker/daemon.json, set the ipv6 key to true and the + fixed-cidr-v6 key to your IPv6 subnet. In this example we are setting it to 2001:db8:1::/64. + +.. code-block:: console + + { + "registry-mirrors": [ + "https://nexus3.o-ran-sc.org:10002", + "https://nexus3.onap.org:10001" + ], + "log-driver": "json-file", + "log-opts": { + "max-size": "10m", + "max-file": "3" + }, + "ipv6": true, + "fixed-cidr-v6": "2001:db8:1::/64" + } - $ docker-compose version - docker-compose version 1.29.1, build c34c88b2 - docker-py version: 5.0.0 - CPython version: 3.7.10 - OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019 +2. Reload the Docker configuration file. + +.. code-block:: console + + $ systemctl reload docker + +Docker Compose +~~~~~~~~~~~~~~ + +.. code-block:: console + + $ docker compose version + Docker Compose version v2.32.4 + +GIT +~~~ + +.. code-block:: console $ git --version - git version 2.25.1 + git version 2.43.0 -The communication between the components bases on domain-names. For a lightweight -deployment on a single laptop or virtual-machine the /etc/hosts file should be -modified as a simple DNS. -Please modify the /etc/hosts of your system. +Please clone the following repositories: +- https://gerrit.o-ran-sc.org/r/admin/repos/oam,general +- https://gerrit.o-ran-sc.org/r/admin/repos/sim/o1-ofhmp-interfaces,general -* \: is the hostname of the system, where the browser is started +Python +~~~~~~ -* \: is the IP address of the system where the solution will be deployed +.. code-block:: console -For development purposes and may reference the same system.:: + $ python3 --version + Python 3.12.3 - $ cat /etc/hosts - 127.0.0.1 localhost - 127.0.1.1 - sdnc-web - identity +A python parser package is required: -Docker Enable IPv6 -^^^^^^^^^^^^^^^^^^ +.. code-block:: console -The O-RAN Alliance specifications target the support of IPv6. -To support IPv6 by docker the docker configuration must be modified. + python3 -m venv .oam + source .oam/bin/activate + pip3 install requirements.txt -Please see: -https://docs.docker.com/engine/daemon/ipv6/ +ETC Host (DNS function) +~~~~~~~~~~~~~~~~~~~~~~~ -1. Edit /etc/docker/daemon.json, set the ipv6 key to true and the fixed-cidr-v6 key to your IPv6 subnet. In this example we are setting it to 2001:db8:1::/64. - -.. code-block:: json - - { - "dns": ["1.1.1.1"], - "registry-mirrors": [ - "https://nexus3.o-ran-sc.org:10002", - "https://nexus3.onap.org:10001" - ], - "log-driver": "json-file", - "log-opts": { - "max-size": "10m", - "max-file": "3" - }, - "ipv6": true, - "fixed-cidr-v6": "2001:db8:1::/64" - } +Your local IP and your used interface are required. Use the following +script to modify all ``.env`` and other configuration files accordingly. +The script will find automatically the interface and its IP address to +the internet. You can check its usage with the option ``-h``: -2. Reload the Docker configuration file. +.. code-block:: console -.. code-block:: bash + python3 ./adopt_to_environment.py -h + +Please run the script with your preferred fully qualified domain name +you would like to use in your browser address bar: + +.. code-block:: console + + python3 ./adopt_to_environment.py -d + +You can revert the settings in the modified ``.env`` and configuration +files using the option ``-r``: - $ systemctl reload docker +.. code-block:: console -It is beneficial (but not mandatory) adding the following line add the -end of your ~/.bashrc file. I will suppress warnings when python script -do not verify self signed certificates for HTTPS communication.:: + python3 ./adopt_to_environment.py -d -r + +It is beneficial (but not mandatory) to add the following line at the end +of your ``~/.bashrc`` file. It will suppress warnings when Python scripts +do not verify self-signed certificates for HTTPS communication. + +.. code-block:: console export PYTHONWARNINGS="ignore:Unverified HTTPS request" -Please ensure that you download and copy the required 3GPP OpenAPIs for VES-stndDefined -message validation into the folder './solution/operation-and-maintenance/smo/oam/ves-collector/externalRepo'. - -Please follow the instructions in ./solution/operation-and-maintenance/smo/oam/ves-collector/externalRepo/3gpp/rep/sa5/MnS/blob/Rel16/OpenAPI/README.md. - -The following tree shows the successfully tested folder structure. It combines different versions of the schemas ('Rel16' and 'SA88-Rel16') using 3GPP branch names.:: - - $ tree solution/operation-and-maintenance/smo/oam/ves-collector/externalRepo/ - solution/operation-and-maintenance/smo/oam/ves-collector/externalRepo/ - ├── 3gpp - │ └── rep - │ └── sa5 - │ └── MnS - │ └── blob - │ ├── Rel16 - │ │ └── OpenAPI - │ │ ├── README.md - │ │ ├── TS28532_FaultMnS.yaml - │ │ ├── TS28532_FileDataReportingMnS.yaml - │ │ ├── TS28532_HeartbeatNtf.yaml - │ │ ├── TS28532_PerfMnS.yaml - │ │ ├── TS28532_ProvMnS.yaml - │ │ ├── TS28532_StreamingDataMnS.yaml - │ │ ├── TS28536_CoslaNrm.yaml - │ │ ├── TS28541_5GcNrm.yaml - │ │ ├── TS28541_NrNrm.yaml - │ │ ├── TS28541_SliceNrm.yaml - │ │ ├── TS28550_PerfMeasJobCtrlMnS.yaml - │ │ ├── TS28623_ComDefs.yaml - │ │ ├── TS28623_GenericNrm.yaml - │ │ ├── TS29512_Npcf_SMPolicyControl.yaml - │ │ ├── TS29514_Npcf_PolicyAuthorization.yaml - │ │ └── TS29571_CommonData.yaml - │ └── SA88-Rel16 - │ └── OpenAPI - │ ├── 5gcNrm.yaml - │ ├── PerDataFileReportMnS.yaml - │ ├── PerMeasJobCtlMnS.yaml - │ ├── PerThresMonMnS.yaml - │ ├── PerfDataStreamingMnS.yaml - │ ├── README.md - │ ├── comDefs.yaml - │ ├── coslaNrm.yaml - │ ├── faultMnS.yaml - │ ├── genericNrm.yaml - │ ├── heartbeatNtf.yaml - │ ├── nrNrm.yaml - │ ├── provMnS.yaml - │ ├── sliceNrm.yaml - │ └── streamingDataMnS.yaml - -Expected Folder Structure -------------------------- - -The following figure show the expected folder structure for the different -docker-compose file and its configurations.:: - - ├── network - │ ├── .env - │ ├── config.py - │ ├── docker-compose.yaml - │ │ - │ ├── ntsim-ng-o-du - │ └── ntsim-ng-o-ru - └── smo - ├── common - │ ├── .env - │ ├── docker-compose.yaml - │ │ - │ ├── dmaap - │ ├── docker - │ ├── identity - │ ├── kafka - │ └── zookeeper - └── oam - ├── .env - ├── docker-compose.yaml - │ - ├── web - ├── controller - └── ves-collector +Please modify the ``/etc/hosts`` of your system or modify the DNS of your +environment: + +- ````: hostname of the system where the browser is started +- ````: IP address of the system where the solution + will be deployed + +.. code-block:: none + + $ cat /etc/hosts + 127.0.0.1 localhost + 127.0.1.1 + + # SMO OAM development system + smo.o-ran-sc.org + gateway.smo.o-ran-sc.org + identity.smo.o-ran-sc.org + messages.smo.o-ran-sc.org + kafka-bridge.smo.o-ran-sc.org + kafka-ui.smo.o-ran-sc.org + odlux.oam.smo.o-ran-sc.org + flows.oam.smo.o-ran-sc.org + tests.oam.smo.o-ran-sc.org + controller.dcn.smo.o-ran-sc.org + ves-collector.dcn.smo.o-ran-sc.org Usage ----- Bring Up Solution +~~~~~~~~~~~~~~~~~ + +Short story +^^^^^^^^^^^ + +The following commands should be invoked. More details can be found in +the next chapters. + +.. code-block:: bash + + source .oam/bin/activate + ./setup.sh + +Simulated network ^^^^^^^^^^^^^^^^^ -Please check and adjust, if required the environment variables:: +Before starting the simulated network, you need to locally build the +docker images. This is because of copyright issues with the 3GPP YANG +models. + +The build should be straightforward. The repository containing the PyNTS +code needs to be cloned, and then you run a command to build the images. +Run this in another terminal, in another folder (not in this repo): + +.. code-block:: bash + + git clone "https://gerrit.o-ran-sc.org/r/sim/o1-ofhmp-interfaces" + cd o1-ofhmp-interfaces + make build-all + +After everything is built successfully, you can return to your solution +folder here and start the network: + +.. code-block:: bash + + docker compose -f network/docker-compose.yaml up -d + docker compose -f network/docker-compose.yaml restart pynts-o-du-o1 + +Check (adjust if required) environment variables +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. code-block:: console nano smo/common/.env nano smo/oam/.env nano network/.env -The tested configuration uses the following external https ports: - -* 8443 for the ves-collector -* 8453 for web access to ODLUX (SDNC_WEB_PORT) -* 8463 for the keyclock web administrator user interface. +Startup solution +^^^^^^^^^^^^^^^^ -Please note that it is necessary to configure first the identity service, -before starting further docker images. +Please note that it is necessary to configure the identity service first, +before starting additional docker images. -The several docker-compose yaml files must be started in the right order as listed below: +The several docker-compose yaml files must be started in the right order +as listed below: -First the SMO common services must be started:: +.. code-block:: bash - docker-compose -f smo/common/docker-compose.yaml up -d + docker compose -f infra/docker-compose.yaml up -d + docker compose -f smo/common/docker-compose.yaml up -d python smo/common/identity/config.py -The python script configure the users within the identity service (keycloak). -A system user (%USER) is also created with administration rights. The initial password (Default4SDN!). -The identity service implemented by Keycloak is configured in a way that the user must change its initial password after the first successful login. +The python script will configure the users within the identity service +(Keycloak). A system user (%USER) is also created with administration +rights. -In a second step the OAM specific service can be started: :: +.. code-block:: bash - docker-compose -f smo/oam/docker-compose.yaml up -d + docker compose -f smo/oam/docker-compose.yaml up -d -Looking into the ONAP SDN-R logs will give you the startup procedure.:: +Looking into the ONAP SDN-R logs will show the startup procedure: - docker logs -f sdnr +.. code-block:: console -The startup was successful when you see the following line: :: + docker logs -f controller - Everything OK in Certificate Installation +If you see the login page (``https://odlux.oam.smo.o-ran-sc.org``), you +are good to go and can start the (simulated) network: +.. code-block:: bash -If you see the login page (https://sdnc-web:8453) you are good to go and can start the (simulated) network.:: + docker compose -f network/docker-compose.yaml up -d - docker-compose -f network/docker-compose.yaml up -d +Usually the first ``ves:event`` gets lost. Please restart the O-DU Docker +container(s) to send a second ``ves:pnfRegistration``: +.. code-block:: bash -Usually the first ves:event gets lost. Please restart the O-DU docker container(s) to send a second ves:pnfRegistration.:: + docker compose -f network/docker-compose.yaml restart pynts-o-du-o1 - docker-compose -f network/docker-compose.yaml restart ntsim-ng-o-du-1122 - python network/config.py +The simulated O-DU and O-RUs are pre-configured according to O-RAN hybrid +architecture. +- **O-RU** - NETCONF Call HOME and NETCONF notifications +- **O-DU** - ``ves:pnfRegistration`` and ``ves:fault``, ``ves:heartbeat`` -The python script configures the simulated O-DU and O-RU according to O-RAN hybrid architecture. +.. image:: _static/nstim-ng-connected-after-ves-pnf-registration-in-odlux.png + :alt: ves:pnfRegistration in ODLUX + :align: center + :width: 70% + :target: _static/nstim-ng-connected-after-ves-pnf-registration-in-odlux.png -O-DU - NETCONF Call HOME and NETCONF notifications -O-RU - ves:pnfRegistration and ves:fault, ves:heartbeat +``ves:fault`` events are processed and finally visible in ODLUX: -'True' indicated that the settings through SDN-R to the NETCONF server were -successful. +.. image:: _static/ves-fault-in-odlux.png + :alt: ves:fault in ODLUX + :align: center + :width: 70% + :target: _static/ves-fault-in-odlux.png -SDN-R reads the fault events from DMaaP and processes them. -Finally the fault events are visible in ODLUX. +Log files and karaf console +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +ODL karaf.logs +^^^^^^^^^^^^^^ -Log files and karaf console -^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. code-block:: console -OpenDaylight/Apache Karaf logs:: + docker exec -it controller tail -f /opt/opendaylight/data/log/karaf.log - docker exec -it sdnr tail -f /opt/opendaylight/data/log/karaf.log +ves-collector logs +^^^^^^^^^^^^^^^^^^ -VES-collector logs:: +.. code-block:: console docker logs -f ves-collector +Customizing Solution +~~~~~~~~~~~~~~~~~~~~ + +``.env`` file contains various customizing parameters. + +Verification Solution +~~~~~~~~~~~~~~~~~~~~~ + +Access to SDN-R ODLUX +^^^^^^^^^^^^^^^^^^^^^ + +Access the following URL in your browser: -Verification of Solution +``https://odlux.oam.smo.o-ran-sc.org`` + +- User: ``admin`` +- Password: *(see .env file)* + +In case of trouble, please update your customized ``.env`` file. + +Access to Node Red Flows ^^^^^^^^^^^^^^^^^^^^^^^^ -On the web user interface https://sdnc-web:8453 you can login with the system user ($USER) and the default password mentioned above. -You should see 4 network function connected via IPv6 and also alarm notification via NETCONF and VES. +``https://flows.oam.smo.o-ran-sc.org`` +- User: ``admin`` +- Password: *(see .env file)* + +Again, update the commands with your customized ``.env`` file if needed. Terminate solution -^^^^^^^^^^^^^^^^^^ +~~~~~~~~~~~~~~~~~~ -To stop all container please respect the following order:: +To stop all containers, please respect the following order: - docker-compose -f network/docker-compose.yaml down - docker-compose -f smo/oam/docker-compose.yaml down - docker-compose -f smo/common/docker-compose.yaml down +.. code-block:: bash + + docker compose -f network/docker-compose.yaml down + docker compose -f smo/apps/docker-compose.yaml down + docker compose -f smo/oam/docker-compose.yaml down + docker compose -f smo/common/docker-compose.yaml down + docker compose -f infra/docker-compose.yaml down + +Alternatively: + +.. code-block:: bash + + ./teardown.sh Cleanup -^^^^^^^ +~~~~~~~ + +.. warning:: + Be careful if other stopped containers are on the same system. -Please be very careful with the following command, if other stopped containers are on the same system:: +.. code-block:: console docker system prune -a -f + +Troubleshooting +~~~~~~~~~~~~~~~ + +In most cases, the ``.env`` settings do not fit the environment and need +to be adjusted. Please ensure that the network settings do not overlap +with other networks. + +Use commands like: + +.. code-block:: console + + docker ps -a + docker compose ps + docker rm -f $(docker ps -aq)