solution/README.md

   1 # Service Management and Orchestration (SMO)
   2
   3 This project focus on a docker-compose deployment solution for SMO/OAM Components.
   4
   5 ## Introduction
   6
   7 With respect to OAM the SMO implements the O1-interface consumers.
   8 According to the O-RAN OAM Architecture and the O-RAN OAM Interface Specification,
   9 the SMO implements a NETCONF Client for configuration and a HTTP/REST/VES server
  10 for receiving all kind of events in VES format.
  11
  12 The setup contains an OpenDaylight based NETCONF client and an ONAP VES Collector.
  13
  14 ## SMO OAM Components
  15
  16 This docker-compose file starts a pre-configured, self-contained SDN-R solution
  17 with the following components.
  18
  19   * **Identity**
  20     ... representing an KeyCloak based identity service for centralized user
  21     management. Please note that the implementation does not support IPv6.
  22     Therefore, its own network is required called 'DMZ'.
  23
  24   * **Controller** single node instance
  25
  26     ... representing the NETCONF consumer on the Service Management and
  27     Orchestration framework (SMO) for O-RAN O1 interface and/or O-RAN OpenFronthaul Management Plane and/or other NETCONF/YANG schemas implemented by the OpenDaylight project.
  28
  29   * **VES collector**
  30
  31     ... representing the VES (REST) provider at SMO for all kind of events.
  32
  33   * **Messages**
  34     ... representing SMO MessageRouter component, includes message-router
  35
  36 ## Prerequisites
  37
  38 ### Resources
  39
  40 The solution was tested on a VM with
  41
  42 - 4x Core
  43 - 16 GBit RAM
  44 - 50 Gbit Storage
  45
  46 ### Operating (HOST) System
  47
  48 ```
  49 $ cat /etc/os-release | grep PRETTY_NAME
  50 PRETTY_NAME="Ubuntu 22.04.2 LTS"
  51 ```
  52
  53 ### Docker
  54
  55 ```
  56 $ docker --version
  57 Docker version 23.0.1, build a5ee5b1
  58 ```
  59 Please follow the required docker daemon configuration as documented in the following README.md:
  60 - [./smo/common/docker/README.md](./smo/common/docker/README.md)
  61
  62 ### Docker Compose
  63
  64 ```
  65 $ docker compose version
  66 Docker Compose version v2.17.2
  67 ```
  68
  69 ### GIT
  70
  71 ```
  72 $ git --version
  73 git version 2.34.1
  74 ```
  75
  76 ### Python
  77
  78 ```
  79 $ python3 --version
  80 Python 3.10.6
  81 ```
  82
  83 A python parser package is required.
  84 ```
  85 sudo apt install python3-pip
  86 pip install jproperties
  87 ```
  88
  89 It is beneficial (but not mandatory) adding the following line add the
  90 end of your ~/.bashrc file. I will suppress warnings when python script
  91 do not verify self signed certificates for HTTPS communication.
  92
  93 ```
  94 export PYTHONWARNINGS="ignore:Unverified HTTPS request"
  95 ```
  96
  97 ### ETC Host (DNS function)
  98
  99 Please change in the different .env files the environment variable 'HOST_IP'
 100 to the IP address of the system where you deploy the solution - search for
 101 'aaa.bbb.ccc.ddd' and replace it.
 102
 103 Please modify the /etc/hosts of your system.
 104
 105 * \<your-system>: is the hostname of the system, where the browser is started
 106
 107 * \<deployment-system-ipv4>: is the IP address of the system where the solution will be deployed
 108
 109 For development purposes <your-system> and <deployment-system> may reference the same system.
 110
 111 ```
 112 $ cat /etc/hosts
 113 127.0.0.1                      localhost
 114 127.0.1.1                      <your-system>
 115
 116 # SMO OAM development system
 117 <deployment-system-ipv4>                   smo.o-ran-sc.org
 118 <deployment-system-ipv4>           gateway.smo.o-ran-sc.org
 119 <deployment-system-ipv4>          identity.smo.o-ran-sc.org
 120 <deployment-system-ipv4>          messages.smo.o-ran-sc.org
 121 <deployment-system-ipv4>      kafka-bridge.smo.o-ran-sc.org
 122 <deployment-system-ipv4>         odlux.oam.smo.o-ran-sc.org
 123 <deployment-system-ipv4>         flows.oam.smo.o-ran-sc.org
 124 <deployment-system-ipv4>         tests.oam.smo.o-ran-sc.org
 125 <deployment-system-ipv4>    controller.dcn.smo.o-ran-sc.org
 126 <deployment-system-ipv4> ves-collector.dcn.smo.o-ran-sc.org
 127
 128 ```
 129
 130 ## Usage
 131
 132 ### Bring Up Solution
 133
 134 #### Short story
 135
 136 The following commands should be invoked. More detailed can be found in the
 137 next chapters.
 138
 139 ```
 140 docker compose -f smo/common/docker-compose.yaml up -d
 141 python smo/common/identity/config.py
 142
 143 docker compose -f smo/oam/docker-compose.yaml up -d
 144 docker compose -f smo/apps/docker-compose.yaml up -d
 145
 146 # wait until the cpu load is low again
 147
 148 docker compose -f network/docker-compose.yaml up -d
 149 docker compose -f network/docker-compose.yaml restart ntsim-ng-o-du-1122 ntsim-ng-o-du-1123
 150 python network/config.py
 151 ```
 152
 153 #### Check (adjust if required) environment variables
 154
 155 ```
 156 nano smo/common/.env
 157 nano smo/oam/.env
 158 nano network/.env
 159 ```
 160
 161 #### Startup solution
 162
 163 Please note that it is necessary to configure first the identity service,
 164 before starting further docker images.
 165
 166 The several docker-compose yaml files must be started in the right order as listed below:
 167
 168 ```
 169 docker compose -f smo/common/docker-compose.yaml up -d
 170 python smo/common/identity/config.py
 171 ```
 172
 173 The python script configure the users within the identity service (keycloak).
 174 A system user (%USER) is also created with administration rights.
 175
 176
 177 ```
 178 docker compose -f smo/oam/docker-compose.yaml up -d
 179 ```
 180
 181 Looking into the ONAP SDN-R logs will give you the startup procedure.
 182
 183 ```
 184 docker logs -f controller
 185 ```
 186
 187 If you see the login page (https://odlux.oam.smo.o-ran-sc.org) you are good to go and can start the (simulated) network.
 188
 189 ```
 190 docker compose -f network/docker-compose.yaml up -d
 191 ```
 192
 193 Usually the first ves:event gets lost. Please restart the O-DU docker container(s) to send a second ves:pnfRegistration.
 194
 195 ```
 196 docker compose -f network/docker-compose.yaml restart ntsim-ng-o-du-1122
 197 python network/config.py
 198 ```
 199
 200 The python script configures the simulated O-DU and O-RU according to O-RAN hybrid architecture.
 201
 202 O-RU - NETCONF Call HOME and NETCONF notifications
 203 O-DU - ves:pnfRegistration and ves:fault, ves:heartbeat
 204
 205 ![ves:pnfRegistration in ODLUX](docs/nstim-ng-connected-after-ves-pnf-registration-in-odlux.png "ves:pnfRegistration in ODLUX")
 206
 207 'True' indicated that the settings through SDN-R to the NETCONF server were
 208 successful.
 209
 210 SDN-R reads the fault events from DMaaP and processes them.
 211 Finally the fault events are visible in ODLUX.
 212
 213 ![ves:fault in ODLUX](docs/ves-fault-in-odlux.png "ves:fault in ODLUX")
 214
 215
 216 ### Log files and karaf console
 217
 218 #### ODL karaf.logs
 219
 220 ```
 221 docker exec -it controller tail -f /opt/opendaylight/data/log/karaf.log
 222 ```
 223
 224 #### ves-collector logs
 225
 226 ```
 227 docker logs -f ves-collector
 228 ```
 229
 230 ### Customizing Solution
 231
 232 '.env' file contains customizing parameters
 233
 234 ### Verification Solution
 235
 236 #### Access to SDN-R ODLUX
 237
 238     https://odlux.oam.smo.o-ran-sc.org
 239
 240     User: admin
 241
 242     Password: // see .env file
 243
 244 In case of trouble, please update the commands with your customized '.env' file.
 245
 246 #### Access to Node Red Flows
 247
 248     https://flows.oam.smo.o-ran-sc.org
 249
 250     User: admin
 251
 252     Password: // see .env file
 253
 254 In case of trouble, please update the commands with your customized '.env' file.
 255
 256 ### Terminate solution
 257
 258 To stop all container please respect the following order
 259
 260 ```
 261 docker compose -f network/docker-compose.yaml down
 262 docker compose -f smo/apps/docker-compose.yaml down
 263 docker compose -f smo/oam/docker-compose.yaml down
 264 docker compose -f smo/common/docker-compose.yaml down
 265 ```
 266
 267 ### Cleanup
 268
 269 !!! be careful if other stopped containers are on the same system
 270 ```
 271 docker system prune -a -f
 272 ```
 273 ### Troubleshooting
 274
 275 In most cases the .env setting do not fit to the environment and need to be
 276 adjusted.
 277
 278 Please make sure that the network settings to not overlap with other networks.
 279
 280 The commands ...
 281 ```
 282 docker ps -a
 283 docker-compose ps
 284 docker rm -f $(docker ps -aq)
 285
 286 ## Commands in action
 287
 288 ```
 289 $ docker ps -a --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
 290 NAMES     IMAGE     STATUS
 291 $ docker compose -f smo/common/docker-compose.yaml up -d
 292 [+] Running 9/9
 293  ✔ Network smo            Created                                                                                                                                                                  0.1s
 294  ✔ Network dmz            Created                                                                                                                                                                  0.1s
 295  ✔ Container zookeeper    Started                                                                                                                                                                  1.6s
 296  ✔ Container persistence  Started                                                                                                                                                                  1.5s
 297  ✔ Container identitydb   Started                                                                                                                                                                  1.2s
 298  ✔ Container gateway      Healthy                                                                                                                                                                 12.1s
 299  ✔ Container kafka        Started                                                                                                                                                                  2.2s
 300  ✔ Container identity     Started                                                                                                                                                                 13.4s
 301  ✔ Container messages     Started                                                                                                                                                                 13.4s
 302 $ python3 smo/common/identity/config.py
 303 Got token!
 304 User leia.organa created!
 305 User r2.d2 created!
 306 User luke.skywalker created!
 307 User jargo.fett created!
 308 User role jargo.fett supervision created!
 309 User role leia.organa administration created!
 310 User role luke.skywalker provision created!
 311 User role r2.d2 administration created!
 312 $ docker compose -f smo/oam/docker-compose.yaml up -d
 313 [+] Running 4/4
 314  ✔ Network oam              Created                                                                                                                                                                0.1s
 315  ✔ Container controller     Healthy                                                                                                                                                               83.4s
 316  ✔ Container ves-collector  Started                                                                                                                                                                1.2s
 317  ✔ Container odlux          Started                                                                                                                                                               84.0s
 318 $ docker compose -f smo/apps/docker-compose.yaml up -d
 319 WARN[0000] Found orphan containers ([odlux controller ves-collector]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
 320 [+] Running 1/1
 321  ✔ Container flows  Started                                                                                                                                                                        0.9s
 322 $ docker compose -f network/docker-compose.yaml up -d
 323 WARN[0000] Found orphan containers ([flows odlux controller ves-collector]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
 324 [+] Running 6/6
 325  ✔ Container ntsim-ng-o-du-1123   Started                                                                                                                                                          2.6s
 326  ✔ Container ntsim-ng-o-ru-11223  Started                                                                                                                                                          2.2s
 327  ✔ Container ntsim-ng-o-ru-11221  Started                                                                                                                                                          1.9s
 328  ✔ Container ntsim-ng-o-ru-11224  Started                                                                                                                                                          1.9s
 329  ✔ Container ntsim-ng-o-du-1122   Started                                                                                                                                                          2.4s
 330  ✔ Container ntsim-ng-o-ru-11222  Started                                                                                                                                                          2.3s
 331 $ docker compose -f network/docker-compose.yaml restart ntsim-ng-o-du-1122 ntsim-ng-o-du-1123
 332 [+] Running 2/2
 333  ✔ Container ntsim-ng-o-du-1122  Started                                                                                                                                                           2.8s
 334  ✔ Container ntsim-ng-o-du-1123  Started                                                                                                                                                           2.9s
 335 $ python3 network/config.py
 336 Set O-RU-11221 True
 337 Set O-RU-11224 True
 338 Set O-RU-11222 True
 339 Set O-DU-1123 True
 340 Set O-DU-1122 True
 341 Set O-RU-11223 True
 342 $ docker ps -a --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
 343 NAMES                 IMAGE                                                                                        STATUS
 344 ntsim-ng-o-ru-11221   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 345 ntsim-ng-o-ru-11224   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 346 ntsim-ng-o-ru-11222   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 347 ntsim-ng-o-du-1123    o-ran-sc/nts-ng-o-ran-du-rel-18:1.6.2                                                        Up 54 seconds
 348 ntsim-ng-o-du-1122    nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-du:1.6.2                                     Up About a minute
 349 ntsim-ng-o-ru-11223   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 350 flows                 nodered/node-red:latest-configured                                                           Up 4 minutes (healthy)
 351 odlux                 nexus3.onap.org:10001/onap/sdnc-web-image:2.4.2                                              Up 7 minutes
 352 controller            nexus3.onap.org:10001/onap/sdnc-image:2.4.2                                                  Up 8 minutes (healthy)
 353 ves-collector         nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.10.1-configured   Up 8 minutes (healthy)
 354 messages              nexus3.onap.org:10001/onap/dmaap/dmaap-mr:1.1.18                                             Up 11 minutes
 355 identity              bitnami/keycloak:18.0.2                                                                      Up 11 minutes
 356 kafka                 nexus3.onap.org:10001/onap/dmaap/kafka111:1.0.4                                              Up 11 minutes
 357 zookeeper             nexus3.onap.org:10001/onap/dmaap/zookeeper:6.0.3                                             Up 11 minutes
 358 identitydb            bitnami/postgresql:13                                                                        Up 11 minutes
 359 persistence           docker.elastic.co/elasticsearch/elasticsearch-oss:7.9.3                                      Up 11 minutes
 360 gateway               traefik:v2.9                                                                                 Up 11 minutes (healthy)
 361 $
 362 ```