solution/README.md

   1 # Service Management and Orchestration (SMO)
   2
   3 This project focus on a docker-compose deployment solution for SMO/OAM Components.
   4
   5 ## Introduction
   6
   7 With respect to OAM the SMO implements the O1-interface consumers.
   8 According to the O-RAN OAM Architecture and the O-RAN OAM Interface Specification,
   9 the SMO implements a NETCONF Client for configuration and a HTTP/REST/VES server
  10 for receiving all kind of events in VES format.
  11
  12 The setup contains an OpenDaylight based NETCONF client and an ONAP VES Collector.
  13
  14 ## SMO OAM Components
  15
  16 This docker-compose file starts a pre-configured, self-contained SDN-R solution
  17 with the following components.
  18
  19   * **Identity**
  20     ... representing an KeyCloak based identity service for centralized user
  21     management. Please note that the implementation does not support IPv6.
  22     Therefore, its own network is required called 'DMZ'.
  23
  24   * **Controller** single node instance
  25
  26     ... representing the NETCONF consumer on the Service Management and
  27     Orchestration framework (SMO) for O-RAN O1 interface and/or O-RAN OpenFronthaul Management Plane and/or other NETCONF/YANG schemas implemented by the OpenDaylight project.
  28
  29   * **VES collector**
  30
  31     ... representing the VES (REST) provider at SMO for all kind of events.
  32
  33   * **Messages**
  34     ... representing SMO MessageRouter component, includes message-router
  35
  36 ## Prerequisites
  37
  38 ### Resources
  39
  40 The solution was tested on a VM with
  41
  42 - 4x Core
  43 - 16 GBit RAM
  44 - 50 Gbit Storage
  45
  46 ### Operating (HOST) System
  47
  48 ```
  49 $ cat /etc/os-release | grep PRETTY_NAME
  50 PRETTY_NAME="Ubuntu 22.04.2 LTS"
  51 ```
  52
  53 ### Docker
  54
  55 ```
  56 $ docker --version
  57 Docker version 23.0.1, build a5ee5b1
  58 ```
  59 Please follow the required docker daemon configuration as documented in the following README.md:
  60 - [./smo/common/docker/README.md](./smo/common/docker/README.md)
  61
  62 ### Docker Compose
  63
  64 ```
  65 $ docker compose version
  66 Docker Compose version v2.17.2
  67 ```
  68
  69 ### GIT
  70
  71 ```
  72 $ git --version
  73 git version 2.34.1
  74 ```
  75
  76 ### Python
  77
  78 ```
  79 $ python3 --version
  80 Python 3.10.6
  81 ```
  82
  83 A python parser package is required.
  84 ```
  85 sudo apt install python3-pip
  86 pip install jproperties
  87 ```
  88
  89 It is beneficial (but not mandatory) adding the following line add the
  90 end of your ~/.bashrc file. I will suppress warnings when python script
  91 do not verify self signed certificates for HTTPS communication.
  92
  93 ```
  94 export PYTHONWARNINGS="ignore:Unverified HTTPS request"
  95 ```
  96
  97 ### ETC Host (DNS function)
  98
  99 Please change in the different .env files the environment variable 'HOST_IP'
 100 to the IP address of the system where you deploy the solution - search for
 101 'aaa.bbb.ccc.ddd' and replace it.
 102
 103 Please modify the /etc/hosts of your system.
 104
 105 * \<your-system>: is the hostname of the system, where the browser is started
 106
 107 * \<deployment-system-ipv4>: is the IP address of the system where the solution will be deployed
 108
 109 For development purposes <your-system> and <deployment-system> may reference the same system.
 110
 111 ```
 112 $ cat /etc/hosts
 113 127.0.0.1                      localhost
 114 127.0.1.1                      <your-system>
 115
 116 # SMO OAM development system
 117 <deployment-system-ipv4>                   smo.o-ran-sc.org
 118 <deployment-system-ipv4>           gateway.smo.o-ran-sc.org
 119 <deployment-system-ipv4>          identity.smo.o-ran-sc.org
 120 <deployment-system-ipv4>          messages.smo.o-ran-sc.org
 121 <deployment-system-ipv4>         odlux.oam.smo.o-ran-sc.org
 122 <deployment-system-ipv4>         flows.oam.smo.o-ran-sc.org
 123 <deployment-system-ipv4>         tests.oam.smo.o-ran-sc.org
 124 <deployment-system-ipv4>    controller.dcn.smo.o-ran-sc.org
 125 <deployment-system-ipv4> ves-collector.dcn.smo.o-ran-sc.org
 126
 127 ```
 128
 129 ## Usage
 130
 131 ### Bring Up Solution
 132
 133 #### Short story
 134
 135 The following commands should be invoked. More detailed can be found in the
 136 next chapters.
 137
 138 ```
 139 docker compose -f smo/common/docker-compose.yml up -d
 140 python smo/common/identity/config.py
 141
 142 docker compose -f smo/oam/docker-compose.yml up -d
 143 docker compose -f smo/apps/docker-compose.yml up -d
 144
 145 # wait until the cpu load is low again
 146
 147 docker compose -f network/docker-compose.yml up -d
 148 docker compose -f network/docker-compose.yml restart ntsim-ng-o-du-1122 ntsim-ng-o-du-1123
 149 python network/config.py
 150 ```
 151
 152 #### Check (adjust if required) environment variables
 153
 154 ```
 155 nano smo/common/.env
 156 nano smo/oam/.env
 157 nano network/.env
 158 ```
 159
 160 #### Startup solution
 161
 162 Please note that it is necessary to configure first the identity service,
 163 before starting further docker images.
 164
 165 The several docker-compose yml files must be started in the right order as listed below:
 166
 167 ```
 168 docker compose -f smo/common/docker-compose.yml up -d
 169 python smo/common/identity/config.py
 170 ```
 171
 172 The python script configure the users within the identity service (keycloak).
 173 A system user (%USER) is also created with administration rights.
 174
 175
 176 ```
 177 docker compose -f smo/oam/docker-compose.yml up -d
 178 ```
 179
 180 Looking into the ONAP SDN-R logs will give you the startup procedure.
 181
 182 ```
 183 docker logs -f controller
 184 ```
 185
 186 If you see the login page (https://odlux.oam.smo.o-ran-sc.org) you are good to go and can start the (simulated) network.
 187
 188 ```
 189 docker compose -f network/docker-compose.yml up -d
 190 ```
 191
 192 Usually the first ves:event gets lost. Please restart the O-DU docker container(s) to send a second ves:pnfRegistration.
 193
 194 ```
 195 docker compose -f network/docker-compose.yml restart ntsim-ng-o-du-1122
 196 python network/config.py
 197 ```
 198
 199 The python script configures the simulated O-DU and O-RU according to O-RAN hybrid architecture.
 200
 201 O-RU - NETCONF Call HOME and NETCONF notifications
 202 O-DU - ves:pnfRegistration and ves:fault, ves:heartbeat
 203
 204 ![ves:pnfRegistration in ODLUX](docs/nstim-ng-connected-after-ves-pnf-registration-in-odlux.png "ves:pnfRegistration in ODLUX")
 205
 206 'True' indicated that the settings through SDN-R to the NETCONF server were
 207 successful.
 208
 209 SDN-R reads the fault events from DMaaP and processes them.
 210 Finally the fault events are visible in ODLUX.
 211
 212 ![ves:fault in ODLUX](docs/ves-fault-in-odlux.png "ves:fault in ODLUX")
 213
 214
 215 ### Log files and karaf console
 216
 217 #### ODL karaf.logs
 218
 219 ```
 220 docker exec -it controller tail -f /opt/opendaylight/data/log/karaf.log
 221 ```
 222
 223 #### ves-collector logs
 224
 225 ```
 226 docker logs -f ves-collector
 227 ```
 228
 229 ### Customizing Solution
 230
 231 '.env' file contains customizing parameters
 232
 233 ### Verification Solution
 234
 235 #### Access to SDN-R ODLUX
 236
 237     https://odlux.oam.smo.o-ran-sc.org
 238
 239     User: admin
 240
 241     Password: // see .env file
 242
 243 In case of trouble, please update the commands with your customized '.env' file.
 244
 245 #### Access to Node Red Flows
 246
 247     https://flows.oam.smo.o-ran-sc.org
 248
 249     User: admin
 250
 251     Password: // see .env file
 252
 253 In case of trouble, please update the commands with your customized '.env' file.
 254
 255 ### Terminate solution
 256
 257 To stop all container please respect the following order
 258
 259 ```
 260 docker compose -f network/docker-compose.yml down
 261 docker compose -f smo/apps/docker-compose.yml down
 262 docker compose -f smo/oam/docker-compose.yml down
 263 docker compose -f smo/common/docker-compose.yml down
 264 ```
 265
 266 ### Cleanup
 267
 268 !!! be careful if other stopped containers are on the same system
 269 ```
 270 docker system prune -a -f
 271 ```
 272 ### Troubleshooting
 273
 274 In most cases the .env setting do not fit to the environment and need to be
 275 adjusted.
 276
 277 Please make sure that the network settings to not overlap with other networks.
 278
 279 The commands ...
 280 ```
 281 docker ps -a
 282 docker-compose ps
 283 docker rm -f $(docker ps -aq)
 284
 285 ## Commands in action
 286
 287 ```
 288 $ docker ps -a --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
 289 NAMES     IMAGE     STATUS
 290 $ docker compose -f smo/common/docker-compose.yml up -d
 291 [+] Running 9/9
 292  ✔ Network smo            Created                                                                                                                                                                  0.1s
 293  ✔ Network dmz            Created                                                                                                                                                                  0.1s
 294  ✔ Container zookeeper    Started                                                                                                                                                                  1.6s
 295  ✔ Container persistence  Started                                                                                                                                                                  1.5s
 296  ✔ Container identitydb   Started                                                                                                                                                                  1.2s
 297  ✔ Container gateway      Healthy                                                                                                                                                                 12.1s
 298  ✔ Container kafka        Started                                                                                                                                                                  2.2s
 299  ✔ Container identity     Started                                                                                                                                                                 13.4s
 300  ✔ Container messages     Started                                                                                                                                                                 13.4s
 301 $ python3 smo/common/identity/config.py
 302 Got token!
 303 User leia.organa created!
 304 User r2.d2 created!
 305 User luke.skywalker created!
 306 User jargo.fett created!
 307 User role jargo.fett supervision created!
 308 User role leia.organa administration created!
 309 User role luke.skywalker provision created!
 310 User role r2.d2 administration created!
 311 $ docker compose -f smo/oam/docker-compose.yml up -d
 312 [+] Running 4/4
 313  ✔ Network oam              Created                                                                                                                                                                0.1s
 314  ✔ Container controller     Healthy                                                                                                                                                               83.4s
 315  ✔ Container ves-collector  Started                                                                                                                                                                1.2s
 316  ✔ Container odlux          Started                                                                                                                                                               84.0s
 317 $ docker compose -f smo/apps/docker-compose.yml up -d
 318 WARN[0000] Found orphan containers ([odlux controller ves-collector]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
 319 [+] Running 1/1
 320  ✔ Container flows  Started                                                                                                                                                                        0.9s
 321 $ docker compose -f network/docker-compose.yml up -d
 322 WARN[0000] Found orphan containers ([flows odlux controller ves-collector]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
 323 [+] Running 6/6
 324  ✔ Container ntsim-ng-o-du-1123   Started                                                                                                                                                          2.6s
 325  ✔ Container ntsim-ng-o-ru-11223  Started                                                                                                                                                          2.2s
 326  ✔ Container ntsim-ng-o-ru-11221  Started                                                                                                                                                          1.9s
 327  ✔ Container ntsim-ng-o-ru-11224  Started                                                                                                                                                          1.9s
 328  ✔ Container ntsim-ng-o-du-1122   Started                                                                                                                                                          2.4s
 329  ✔ Container ntsim-ng-o-ru-11222  Started                                                                                                                                                          2.3s
 330 $ docker compose -f network/docker-compose.yml restart ntsim-ng-o-du-1122 ntsim-ng-o-du-1123
 331 [+] Running 2/2
 332  ✔ Container ntsim-ng-o-du-1122  Started                                                                                                                                                           2.8s
 333  ✔ Container ntsim-ng-o-du-1123  Started                                                                                                                                                           2.9s
 334 $ python3 network/config.py
 335 Set O-RU-11221 True
 336 Set O-RU-11224 True
 337 Set O-RU-11222 True
 338 Set O-DU-1123 True
 339 Set O-DU-1122 True
 340 Set O-RU-11223 True
 341 $ docker ps -a --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
 342 NAMES                 IMAGE                                                                                        STATUS
 343 ntsim-ng-o-ru-11221   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 344 ntsim-ng-o-ru-11224   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 345 ntsim-ng-o-ru-11222   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 346 ntsim-ng-o-du-1123    o-ran-sc/nts-ng-o-ran-du-rel-18:1.6.2                                                        Up 54 seconds
 347 ntsim-ng-o-du-1122    nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-du:1.6.2                                     Up About a minute
 348 ntsim-ng-o-ru-11223   nexus3.o-ran-sc.org:10004/o-ran-sc/nts-ng-o-ran-ru-fh:1.6.2                                  Up 4 minutes
 349 flows                 nodered/node-red:latest-configured                                                           Up 4 minutes (healthy)
 350 odlux                 nexus3.onap.org:10001/onap/sdnc-web-image:2.4.2                                              Up 7 minutes
 351 controller            nexus3.onap.org:10001/onap/sdnc-image:2.4.2                                                  Up 8 minutes (healthy)
 352 ves-collector         nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.10.1-configured   Up 8 minutes (healthy)
 353 messages              nexus3.onap.org:10001/onap/dmaap/dmaap-mr:1.1.18                                             Up 11 minutes
 354 identity              bitnami/keycloak:18.0.2                                                                      Up 11 minutes
 355 kafka                 nexus3.onap.org:10001/onap/dmaap/kafka111:1.0.4                                              Up 11 minutes
 356 zookeeper             nexus3.onap.org:10001/onap/dmaap/zookeeper:6.0.3                                             Up 11 minutes
 357 identitydb            bitnami/postgresql:13                                                                        Up 11 minutes
 358 persistence           docker.elastic.co/elasticsearch/elasticsearch-oss:7.9.3                                      Up 11 minutes
 359 gateway               traefik:v2.9                                                                                 Up 11 minutes (healthy)
 360 $
 361 ```