solution/README.md

   1 # Service Management and Orchestration (SMO)
   2
   3 This project focus on a docker-compose deployment solution for SMO/OAM Components.
   4
   5 ## Introduction
   6
   7 With respect to OAM the SMO implements the O1-interface consumers.
   8 According to the O-RAN OAM Architecture and the O-RAN OAM Interface Specification,
   9 the SMO implements a NETCONF Client for configuration and a HTTP/REST/VES server
  10 for receiving all kind of events in VES format.
  11
  12 The setup contains an OpenDaylight based NETCONF client and an ONAP VES Collector.
  13
  14 ## SMO OAM Components
  15
  16 This docker-compose file starts a pre-configured, self-contained SDN-R solution
  17 with the following components.
  18
  19   * **Identity**
  20     ... representing an KeyCloak based identity service for centralized user
  21     management. Please note that the implementation does not support IPv6.
  22     Therefore, its own network is required called 'DMZ'.
  23
  24   * **Controller** single node instance
  25
  26     ... representing the NETCONF consumer on the Service Management and
  27     Orchestration framework (SMO) for O-RAN O1 interface and/or O-RAN OpenFronthaul Management Plane and/or other NETCONF/YANG schemas implemented by the OpenDaylight project.
  28
  29   * **VES collector**
  30
  31     ... representing the VES (REST) provider at SMO for all kind of events.
  32
  33   * **Messages**
  34     ... representing SMO MessageRouter component, includes message-router
  35
  36 ## Prerequisites
  37
  38 ### Resources
  39
  40 The solution was tested on a VM with
  41
  42 - 4x Core
  43 - 16 GBit RAM
  44 - 50 Gbit Storage
  45
  46 ### Operating (HOST) System
  47
  48 ```
  49 $ cat /etc/os-release | grep PRETTY_NAME
  50 PRETTY_NAME="Ubuntu 22.04.2 LTS"
  51 ```
  52
  53 ### Docker
  54
  55 ```
  56 $ docker --version
  57 Docker version 23.0.1, build a5ee5b1
  58 ```
  59 Please follow the required docker daemon configuration as documented in the following README.md:
  60 - [./smo/common/docker/README.md](./smo/common/docker/README.md)
  61
  62 ### Docker Compose
  63
  64 ```
  65 $ docker compose version
  66 Docker Compose version v2.17.2
  67 ```
  68
  69 ### GIT
  70
  71 ```
  72 $ git --version
  73 git version 2.34.1
  74 ```
  75
  76 ### Python
  77
  78 ```
  79 $ python3 --version
  80 Python 3.10.6
  81 ```
  82
  83 A python parser package is required.
  84 ```
  85 sudo apt install python3-pip
  86 pip install jproperties
  87 ```
  88
  89 It is beneficial (but not mandatory) adding the following line add the
  90 end of your ~/.bashrc file. I will suppress warnings when python script
  91 do not verify self signed certificates for HTTPS communication.
  92
  93 ```
  94 export PYTHONWARNINGS="ignore:Unverified HTTPS request"
  95 ```
  96
  97 ### ETC Host (DNS function)
  98
  99 Please change in the different .env files the environment variable 'HOST_IP'
 100 to the IP address of the system where you deploy the solution - search for
 101 'aaa.bbb.ccc.ddd' and replace it.
 102
 103 Please modify the /etc/hosts of your system.
 104
 105 * \<your-system>: is the hostname of the system, where the browser is started
 106
 107 * \<deployment-system-ipv4>: is the IP address of the system where the solution will be deployed
 108
 109 For development purposes <your-system> and <deployment-system> may reference the same system.
 110
 111 ```
 112 $ cat /etc/hosts
 113 127.0.0.1                      localhost
 114 127.0.1.1                      <your-system>
 115
 116 # SMO OAM development system
 117 <deployment-system-ipv4>                   smo.o-ran-sc.org
 118 <deployment-system-ipv4>           gateway.smo.o-ran-sc.org
 119 <deployment-system-ipv4>          identity.smo.o-ran-sc.org
 120 <deployment-system-ipv4>          messages.smo.o-ran-sc.org
 121 <deployment-system-ipv4> ves-collector.oam.smo.o-ran-sc.org
 122 <deployment-system-ipv4>         odlux.oam.smo.o-ran-sc.org
 123 <deployment-system-ipv4>    controller.oam.smo.o-ran-sc.org
 124
 125 ```
 126
 127 ## Usage
 128
 129 ### Bring Up Solution
 130
 131 #### Short story
 132
 133 The following commands should be invoked. More detailed can be found in the
 134 next chapters.
 135
 136 ```
 137 docker compose -f smo/common/docker-compose.yml up -d
 138 python smo/common/identity/config.py
 139
 140 docker compose -f smo/oam/docker-compose.yml up -d
 141 docker compose -f smo/apps/docker-compose.yml up -d
 142
 143 # wait until the cpu load is low again
 144
 145 docker compose -f network/docker-compose.yml up -d
 146 docker compose -f network/docker-compose.yml restart ntsim-ng-o-du-1122 ntsim-ng-o-du-1123
 147 python network/config.py
 148 ```
 149
 150 #### Check (adjust if required) environment variables
 151
 152 ```
 153 nano smo/common/.env
 154 nano smo/oam/.env
 155 nano network/.env
 156 ```
 157
 158 #### Startup solution
 159
 160 Please note that it is necessary to configure first the identity service,
 161 before starting further docker images.
 162
 163 The several docker-compose yml files must be started in the right order as listed below:
 164
 165 ```
 166 docker compose -f smo/common/docker-compose.yml up -d
 167 python smo/common/identity/config.py
 168 ```
 169
 170 The python script configure the users within the identity service (keycloak).
 171 A system user (%USER) is also created with administration rights.
 172
 173
 174 ```
 175 docker compose -f smo/oam/docker-compose.yml up -d
 176 ```
 177
 178 Looking into the ONAP SDN-R logs will give you the startup procedure.
 179
 180 ```
 181 docker logs -f controller
 182 ```
 183
 184 If you see the login page (https://odlux.oam.smo.o-ran-sc.org) you are good to go and can start the (simulated) network.
 185
 186 ```
 187 docker compose -f network/docker-compose.yml up -d
 188 ```
 189
 190 Usually the first ves:event gets lost. Please restart the O-DU docker container(s) to send a second ves:pnfRegistration.
 191
 192 ```
 193 docker compose -f network/docker-compose.yml restart ntsim-ng-o-du-1122
 194 python network/config.py
 195 ```
 196
 197 The python script configures the simulated O-DU and O-RU according to O-RAN hybrid architecture.
 198
 199 O-RU - NETCONF Call HOME and NETCONF notifications
 200 O-DU - ves:pnfRegistration and ves:fault, ves:heartbeat
 201
 202 ![ves:pnfRegistration in ODLUX](docs/nstim-ng-connected-after-ves-pnf-registration-in-odlux.png "ves:pnfRegistration in ODLUX")
 203
 204 'True' indicated that the settings through SDN-R to the NETCONF server were
 205 successful.
 206
 207 SDN-R reads the fault events from DMaaP and processes them.
 208 Finally the fault events are visible in ODLUX.
 209
 210 ![ves:fault in ODLUX](docs/ves-fault-in-odlux.png "ves:fault in ODLUX")
 211
 212
 213 ### Log files and karaf console
 214
 215 #### ODL karaf.logs
 216
 217 ```
 218 docker exec -it controller tail -f /opt/opendaylight/data/log/karaf.log
 219 ```
 220
 221 #### ves-collector logs
 222
 223 ```
 224 docker logs -f ves-collector
 225 ```
 226
 227 ### Customizing Solution
 228
 229 '.env' file contains customizing parameters
 230
 231 ### Verification Solution
 232
 233 #### Access to SDN-R ODLUX
 234
 235     https://odlux.oam.smo.o-ran-sc.org
 236
 237     User: admin
 238
 239     Password: // see .env file
 240
 241 In case of trouble, please update the commands with your customized '.env' file.
 242
 243 #### Access to Node Red Flows
 244
 245     https://flows.oam.smo.o-ran-sc.org
 246
 247     User: admin
 248
 249     Password: // see .env file
 250
 251 In case of trouble, please update the commands with your customized '.env' file.
 252
 253 ### Terminate solution
 254
 255 To stop all container please respect the following order
 256
 257 ```
 258 docker compose -f network/docker-compose.yml down
 259 docker compose -f smo/apps/docker-compose.yml down
 260 docker compose -f smo/oam/docker-compose.yml down
 261 docker compose -f smo/common/docker-compose.yml down
 262 ```
 263
 264 ### Cleanup
 265
 266 !!! be careful if other stopped containers are on the same system
 267 ```
 268 docker system prune -a -f
 269 ```
 270 ### Troubleshooting
 271
 272 In most cases the .env setting do not fit to the environment and need to be
 273 adjusted.
 274
 275 Please make sure that the network settings to not overlap with other networks.
 276
 277 The commands ...
 278 ```
 279 docker ps -a
 280 docker-compose ps
 281 docker rm -f $(docker ps -aq)
 282
 283 ## Commands in action
 284
 285 ```
 286 $ docker ps -a --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
 287 NAMES     IMAGE     STATUS
 288 $ docker compose -f smo/common/docker-compose.yml up -d
 289 [+] Running 9/9
 290  ✔ Network smo            Created                                                                                                                                                                  0.1s
 291  ✔ Network dmz            Created                                                                                                                                                                  0.1s
 292  ✔ Container zookeeper    Started                                                                                                                                                                  1.6s
 293  ✔ Container persistence  Started                                                                                                                                                                  1.5s
 294  ✔ Container identitydb   Started                                                                                                                                                                  1.2s
 295  ✔ Container gateway      Healthy                                                                                                                                                                 12.1s
 296  ✔ Container kafka        Started                                                                                                                                                                  2.2s
 297  ✔ Container identity     Started                                                                                                                                                                 13.4s
 298  ✔ Container messages     Started                                                                                                                                                                 13.4s
 299 $ python3 smo/common/identity/config.py
 300 Got token!
 301 User leia.organa created!
 302 User r2.d2 created!
 303 User luke.skywalker created!
 304 User jargo.fett created!
 305 User role jargo.fett supervision created!
 306 User role leia.organa administration created!
 307 User role luke.skywalker provision created!
 308 User role r2.d2 administration created!
 309 $ docker compose -f smo/oam/docker-compose.yml up -d
 310 [+] Running 4/4
 311  ✔ Network oam              Created                                                                                                                                                                0.1s
 312  ✔ Container controller     Healthy                                                                                                                                                               83.4s
 313  ✔ Container ves-collector  Started                                                                                                                                                                1.2s
 314  ✔ Container odlux          Started                                                                                                                                                               84.0s
 315 $ docker compose -f smo/apps/docker-compose.yml up -d
 316 WARN[0000] Found orphan containers ([odlux controller ves-collector]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
 317 [+] Running 1/1
 318  ✔ Container flows  Started                                                                                                                                                                        0.9s
 319 $ docker compose -f network/docker-compose.yml up -d
 320 WARN[0000] Found orphan containers ([flows odlux controller ves-collector]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
 321 [+] Running 6/6
 322  ✔ Container ntsim-ng-o-du-1123   Started                                                                                                                                                          2.6s
 323  ✔ Container ntsim-ng-o-ru-11223  Started                                                                                                                                                          2.2s
 324  ✔ Container ntsim-ng-o-ru-11221  Started                                                                                                                                                          1.9s
 325  ✔ Container ntsim-ng-o-ru-11224  Started                                                                                                                                                          1.9s
 326  ✔ Container ntsim-ng-o-du-1122   Started                                                                                                                                                          2.4s
 327  ✔ Container ntsim-ng-o-ru-11222  Started                                                                                                                                                          2.3s
 328 $ docker compose -f network/docker-compose.yml restart ntsim-ng-o-du-1122 ntsim-ng-o-du-1123
 329 [+] Running 2/2
 330  ✔ Container ntsim-ng-o-du-1122  Started                                                                                                                                                           2.8s
 331  ✔ Container ntsim-ng-o-du-1123  Started                                                                                                                                                           2.9s
 332 $ python3 network/config.py
 333 Set O-RU-11221 True
 334 Set O-RU-11224 True
 335 Set O-RU-11222 True
 336 Set O-DU-1123 True
 337 Set O-DU-1122 True
 338 Set O-RU-11223 True
 339 $ docker ps -a --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
 340 NAMES                 IMAGE                                                                                        STATUS
 341 ntsim-ng-o-ru-11221   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 342 ntsim-ng-o-ru-11224   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 343 ntsim-ng-o-ru-11222   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 344 ntsim-ng-o-du-1123    o-ran-sc/nts-ng-o-ran-du-rel-18:1.6.2                                                        Up 54 seconds
 345 ntsim-ng-o-du-1122    nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-du:1.6.2                                     Up About a minute
 346 ntsim-ng-o-ru-11223   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 347 flows                 nodered/node-red:latest-configured                                                           Up 4 minutes (healthy)
 348 odlux                 nexus3.onap.org:10001/onap/sdnc-web-image:2.4.2                                              Up 7 minutes
 349 controller            nexus3.onap.org:10001/onap/sdnc-image:2.4.2                                                  Up 8 minutes (healthy)
 350 ves-collector         nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.10.1-configured   Up 8 minutes (healthy)
 351 messages              nexus3.onap.org:10001/onap/dmaap/dmaap-mr:1.1.18                                             Up 11 minutes
 352 identity              bitnami/keycloak:18.0.2                                                                      Up 11 minutes
 353 kafka                 nexus3.onap.org:10001/onap/dmaap/kafka111:1.0.4                                              Up 11 minutes
 354 zookeeper             nexus3.onap.org:10001/onap/dmaap/zookeeper:6.0.3                                             Up 11 minutes
 355 identitydb            bitnami/postgresql:13                                                                        Up 11 minutes
 356 persistence           docker.elastic.co/elasticsearch/elasticsearch-oss:7.9.3                                      Up 11 minutes
 357 gateway               traefik:v2.9                                                                                 Up 11 minutes (healthy)
 358 $
 359 ```