docs/installation-guide.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3 .. Copyright (C) 2019 Wind River Systems, Inc.
   4
   5
   6 Installation Guide
   7 ==================
   8
   9 .. contents::
  10    :depth: 3
  11    :local:
  12
  13 Abstract
  14 --------
  15
  16 This document describes how to install O-RAN INF image, example configuration for better
  17 real time performance, and example deployment of Kubernetes cluster and plugins.
  18
  19 The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
  20 and container technology.
  21
  22 Version history
  23
  24 +--------------------+--------------------+--------------------+--------------------+
  25 | **Date**           | **Ver.**           | **Author**         | **Comment**        |
  26 |                    |                    |                    |                    |
  27 +--------------------+--------------------+--------------------+--------------------+
  28 | 2019-11-02         | 1.0.0              | Jackie Huang       | Initail version    |
  29 |                    |                    |                    |                    |
  30 +--------------------+--------------------+--------------------+--------------------+
  31 |                    |                    |                    |                    |
  32 |                    |                    |                    |                    |
  33 +--------------------+--------------------+--------------------+--------------------+
  34 |                    |                    |                    |                    |
  35 |                    |                    |                    |                    |
  36 +--------------------+--------------------+--------------------+--------------------+
  37
  38
  39 Preface
  40 -------
  41
  42 Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
  43
  44
  45 Hardware Requirements
  46 ---------------------
  47
  48 Following minimum hardware requirements must be met for installation of O-RAN INF image:
  49
  50 +--------------------+----------------------------------------------------+
  51 | **HW Aspect**      | **Requirement**                                    |
  52 |                    |                                                    |
  53 +--------------------+----------------------------------------------------+
  54 | **# of servers**   | 1                                                  |
  55 +--------------------+----------------------------------------------------+
  56 | **CPU**            | 2                                                  |
  57 |                    |                                                    |
  58 +--------------------+----------------------------------------------------+
  59 | **RAM**            | 4G                                                 |
  60 |                    |                                                    |
  61 +--------------------+----------------------------------------------------+
  62 | **Disk**           | 20G                                                |
  63 |                    |                                                    |
  64 +--------------------+----------------------------------------------------+
  65 | **NICs**           | 1                                                  |
  66 |                    |                                                    |
  67 +--------------------+----------------------------------------------------+
  68
  69
  70
  71 Software Installation and Deployment
  72 ------------------------------------
  73
  74 1. Installation from the O-RAN INF ISO image
  75 ````````````````````````````````````````````
  76
  77 - Please see the README.md file for how to build the image.
  78 - The Image is a live ISO image with CLI installer: oran-image-inf-host-intel-x86-64.iso
  79
  80 1.1 Burn the image to the USB device
  81 ''''''''''''''''''''''''''''''''''''
  82
  83 - Assume the the usb device is /dev/sdX here
  84
  85 ::
  86
  87   $ sudo dd if=/path/to/oran-image-inf-host-intel-x86-64.iso of=/dev/sdX bs=1M
  88
  89 1.2 Insert the USB device in the target to be booted.
  90 '''''''''''''''''''''''''''''''''''''''''''''''''''''
  91
  92 1.3 Reboot the target from the USB device.
  93 ''''''''''''''''''''''''''''''''''''''''''
  94
  95 1.4 Select "Graphics console install" or "Serial console install" and press ENTER
  96 '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
  97
  98 1.5 Select the hard disk and press ENTER
  99 ''''''''''''''''''''''''''''''''''''''''
 100
 101 Notes: In this installer, you can only select which hard disk to install, the whole disk will be used and partitioned automatically.
 102
 103 - e.g. insert "sda" and press ENTER
 104
 105 1.6 Remove the USB device and press ENTER to reboot
 106 '''''''''''''''''''''''''''''''''''''''''''''''''''
 107
 108 2. Configuration for better real time performance
 109 `````````````````````````````````````````````````
 110
 111 Notes: Some of the tuning options are machine specific or depend on use cases,
 112 like the hugepages, isolcpus, rcu_nocbs, kthread_cpus, irqaffinity, nohz_full and
 113 so on, please do not just copy and past.
 114
 115 - Edit the grub.cfg with the following example tuning options
 116
 117 ::
 118
 119   # Notes: the grub.cfg file path is different for legacy and UEFI mode
 120   #   For legacy mode: /boot/grub/grub.cfg
 121   #   For UEFI mode: /boot/EFI/BOOT/grub.cfg
 122
 123   grub_cfg="/boot/grub/grub.cfg"
 124   #grub_cfg="/boot/EFI/BOOT/grub.cfg"
 125
 126   # In this example, 1-16 cores are isolated for real time processes
 127   root@intel-x86-64:~# rt_tuning="crashkernel=auto biosdevname=0 iommu=pt usbcore.autosuspend=-1 nmi_watchdog=0 softlockup_panic=0 intel_iommu=on cgroup_enable=memory skew_tick=1 hugepagesz=1G hugepages=4 default_hugepagesz=1G isolcpus=1-16 rcu_nocbs=1-16 kthread_cpus=0 irqaffinity=0 nohz=on nohz_full=1-16 intel_idle.max_cstate=0 processor.max_cstate=1 intel_pstate=disable nosoftlockup idle=poll mce=ignore_ce"
 128
 129   # optional to add the console setting
 130   root@intel-x86-64:~# console="console=ttyS0,115200"
 131
 132   root@intel-x86-64:~# sed -i "/linux / s/$/ $console $rt_tuning/" $grub_cfg
 133
 134
 135 - Reboot the target
 136
 137 ::
 138
 139   root@intel-x86-64:~# reboot
 140
 141 3. Kubernetes cluster and plugins deployment instructions (All-in-one)
 142 ``````````````````````````````````````````````````````````````````````
 143 This instruction will show you how to deploy kubernetes cluster and plugins in an all-in-one example scenario after the above installation.
 144
 145 3.1 Change the hostname (Optional)
 146 ''''''''''''''''''''''''''''''''''
 147
 148 ::
 149
 150   # Assuming the hostname is oran-aio, ip address is <aio_host_ip>
 151   # please DO NOT copy and paste, use your actaul hostname and ip address
 152   root@intel-x86-64:~# echo oran-aio > /etc/hostname
 153   root@intel-x86-64:~# export AIO_HOST_IP="<aio_host_ip>"
 154   root@intel-x86-64:~# echo "$AIO_HOST_IP oran-aio" >> /etc/hosts
 155
 156 3.2 Disable swap for Kubernetes
 157 '''''''''''''''''''''''''''''''
 158
 159 ::
 160
 161   root@intel-x86-64:~# sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
 162   root@intel-x86-64:~# systemctl mask dev-sda4.swap
 163
 164 3.3 Set the proxy for docker (Optional)
 165 '''''''''''''''''''''''''''''''''''''''
 166
 167 - If you are under a firewall, you may need to set the proxy for docker to pull images
 168
 169 ::
 170
 171   root@intel-x86-64:~# HTTP_PROXY="http://<your_proxy_server_ip>:<port>"
 172   root@intel-x86-64:~# mkdir /etc/systemd/system/docker.service.d/
 173   root@intel-x86-64:~# cat << EOF > /etc/systemd/system/docker.service.d/http-proxy.conf
 174   [Service]
 175   Environment="HTTP_PROXY=$HTTP_PROXY" "NO_PROXY=localhost,127.0.0.1,localaddress,.localdomain.com,$AIO_HOST_IP,10.244.0.0/16"
 176   EOF
 177
 178 3.4 Reboot the target
 179 '''''''''''''''''''''
 180
 181 ::
 182
 183   root@intel-x86-64:~# reboot
 184
 185 3.5 Initialize kubernetes cluster master
 186 ''''''''''''''''''''''''''''''''''''''''
 187
 188 ::
 189
 190   root@oran-aio:~# kubeadm init --kubernetes-version v1.15.2 --pod-network-cidr=10.244.0.0/16
 191   root@oran-aio:~# mkdir -p $HOME/.kube
 192   root@oran-aio:~# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 193   root@oran-aio:~# chown $(id -u):$(id -g) $HOME/.kube/config
 194
 195 3.6 Make the master also works as a worker node
 196 '''''''''''''''''''''''''''''''''''''''''''''''
 197
 198 ::
 199
 200   root@oran-aio:~# kubectl taint nodes oran-aio node-role.kubernetes.io/master-
 201
 202 3.7 Deploy flannel
 203 ''''''''''''''''''
 204
 205 ::
 206
 207   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/flannel/kube-flannel.yml
 208
 209 Check that the aio node is ready after flannel is successfully deployed and running
 210
 211 ::
 212
 213   root@oran-aio:~# kubectl get pods --all-namespaces |grep flannel
 214   kube-system   kube-flannel-ds-amd64-bwt52        1/1     Running   0          3m24s
 215
 216   root@oran-aio:~# kubectl get nodes
 217   NAME       STATUS   ROLES    AGE     VERSION
 218   oran-aio   Ready    master   3m17s   v1.15.2-dirty
 219
 220 3.8 Deploy kubernetes dashboard
 221 '''''''''''''''''''''''''''''''
 222
 223 Deploy kubernetes dashboard
 224
 225 ::
 226
 227   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/kubernetes-dashboard/kubernetes-dashboard-admin.rbac.yaml
 228   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/kubernetes-dashboard/kubernetes-dashboard.yaml
 229
 230 Verify that the dashboard is up and running
 231
 232 ::
 233
 234   # Check the pod for dashboard
 235   root@oran-aio:~# kubectl get pods --all-namespaces |grep dashboard
 236   kube-system   kubernetes-dashboard-5b67bf4d5f-ghg4f   1/1     Running   0          64s
 237
 238 Access the dashboard UI in a web browser with the https url, port number is 30443.
 239
 240 - For detail usage, please refer to `Doc for dashboard`_
 241
 242 .. _`Doc for dashboard`: https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
 243
 244 3.9 Deploy Multus-CNI
 245 '''''''''''''''''''''
 246
 247 ::
 248
 249   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/multus-cni/multus-daemonset.yml
 250
 251 Verify that the multus-cni is up and running
 252
 253 ::
 254
 255   root@oran-aio:~# kubectl get pods --all-namespaces | grep -i multus
 256   kube-system   kube-multus-ds-amd64-hjpk4              1/1     Running   0          7m34s
 257
 258 - For further validating, please refer to the `Multus-CNI quick start`_
 259
 260 .. _`Multus-CNI quick start`: https://github.com/intel/multus-cni/blob/master/doc/quickstart.md
 261
 262 3.10 Deploy NFD (node-feature-discovery)
 263 ''''''''''''''''''''''''''''''''''''''''
 264
 265 ::
 266
 267   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/node-feature-discovery/nfd-master.yaml
 268   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/node-feature-discovery/nfd-worker-daemonset.yaml
 269
 270 Verify that nfd-master and nfd-worker are up and running
 271
 272 ::
 273
 274   root@oran-aio:~# kubectl get pods --all-namespaces |grep nfd
 275   default       nfd-master-7v75k                        1/1     Running   0          91s
 276   default       nfd-worker-xn797                        1/1     Running   0          24s
 277
 278 Verify that the node is labeled by nfd:
 279
 280 ::
 281
 282   root@oran-aio:~# kubectl describe nodes|grep feature.node.kubernetes
 283                      feature.node.kubernetes.io/cpu-cpuid.AESNI=true
 284                      feature.node.kubernetes.io/cpu-cpuid.AVX=true
 285                      feature.node.kubernetes.io/cpu-cpuid.AVX2=true
 286                      (...snip...)
 287
 288 3.11 Deploy CMK (CPU-Manager-for-Kubernetes)
 289 ''''''''''''''''''''''''''''''''''''''''''''
 290
 291 Build the CMK docker image
 292
 293 ::
 294
 295   root@oran-aio:~# cd /opt/kubernetes_plugins/cpu-manager-for-kubernetes/
 296   root@oran-aio:/opt/kubernetes_plugins/cpu-manager-for-kubernetes# make
 297
 298 Verify that the cmk docker images is built successfully
 299
 300 ::
 301
 302   root@oran-aio:/opt/kubernetes_plugins/cpu-manager-for-kubernetes# docker images|grep cmk
 303   cmk          v1.3.1              3fec5f753b05        44 minutes ago      765MB
 304
 305 Edit the template yaml file for your deployment:
 306   - The template file is: /etc/kubernetes/plugins/cpu-manager-for-kubernetes/cmk-cluster-init-pod-template.yaml
 307   - The options you may need to change:
 308
 309 ::
 310
 311   # You can change the value for the following env:
 312   env:
 313   - name: HOST_LIST
 314     # Change this to modify the the host list to be initialized
 315     value: "oran-aio"
 316   - name: NUM_EXCLUSIVE_CORES
 317     # Change this to modify the value passed to `--num-exclusive-cores` flag
 318     value: "4"
 319   - name: NUM_SHARED_CORES
 320     # Change this to modify the value passed to `--num-shared-cores` flag
 321     value: "1"
 322   - name: CMK_IMG
 323     # Change his ONLY if you built the docker images with a different tag name
 324     value: "cmk:v1.3.1"
 325
 326 Or you can also refer to `CMK operator manual`_
 327
 328 .. _`CMK operator manual`: https://github.com/intel/CPU-Manager-for-Kubernetes/blob/master/docs/operator.md
 329
 330
 331 Depoly CMK from yaml files
 332
 333 ::
 334
 335   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/cpu-manager-for-kubernetes/cmk-rbac-rules.yaml
 336   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/cpu-manager-for-kubernetes/cmk-serviceaccount.yaml
 337   root@oran-aio:~# kubectl apply -f /etc/kubernetes/plugins/cpu-manager-for-kubernetes/cmk-cluster-init-pod-template.yaml
 338
 339 Verify that the cmk cluster init completed and the pods for nodereport and webhook deployment are up and running
 340
 341 ::
 342
 343   root@oran-aio:/opt/kubernetes_plugins/cpu-manager-for-kubernetes# kubectl get pods --all-namespaces |grep cmk
 344   default       cmk-cluster-init-pod                         0/1     Completed   0          11m
 345   default       cmk-init-install-discover-pod-oran-aio       0/2     Completed   0          10m
 346   default       cmk-reconcile-nodereport-ds-oran-aio-qbdqb   2/2     Running     0          10m
 347   default       cmk-webhook-deployment-6f9dd7dfb6-2lj2p      1/1     Running     0          10m
 348
 349 - For detail usage, please refer to `CMK user manual`_
 350
 351 .. _`CMK user manual`: https://github.com/intel/CPU-Manager-for-Kubernetes/blob/master/docs/user.md
 352
 353 References
 354 ----------
 355
 356 - `Flannel`_
 357 - `Doc for dashboard`_
 358 - `Multus-CNI quick start`_
 359 - `CMK operator manual`_
 360 - `CMK user manual`_
 361
 362 .. _`Flannel`: https://github.com/coreos/flannel/blob/master/README.md