.. Copyright (C) 2019 Wind River Systems, Inc.
-Installation Guide
-==================
+INF Installation Guide
+======================
.. contents::
:depth: 3
:local:
Abstract
---------
+********
-This document describes how to install O-RAN INF image, example configuration for better
-real time performance, and example deployment of Kubernetes cluster and plugins.
+This document describes how to install O-RAN INF image, example configuration (All-in-one Duplex)
+for better real time performance, and example deployment of Kubernetes cluster and plugins.
The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
and container technology.
-Version history
-
-+--------------------+--------------------+--------------------+--------------------+
-| **Date** | **Ver.** | **Author** | **Comment** |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-| 2019-11-02 | 1.0.0 | Jackie Huang | Initail version |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-| 2020-06-03 | 2.0.0 (Bronze) | Xiaohua Zhang | Bronze version |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-| 2020-11-23 | 3.0.0 (Cherry) | Xiaohua Zhang | Cherry version |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-
Preface
--------
+*******
Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
Hardware Requirements
----------------------
-
-Following minimum hardware requirements must be met for installation of O-RAN INF image:
-
-+--------------------+----------------------------------------------------+
-| **HW Aspect** | **Requirement** |
-| | |
-+--------------------+----------------------------------------------------+
-| **# of servers** | 1 |
-+--------------------+----------------------------------------------------+
-| **CPU** | 4 |
-| | |
-+--------------------+----------------------------------------------------+
-| **RAM** | 16G |
-| | |
-+--------------------+----------------------------------------------------+
-| **Disk** | 500G * 2 |
-| | |
-+--------------------+----------------------------------------------------+
-| **NICs** | 2 |
-| | |
-+--------------------+----------------------------------------------------+
-
-If want to enable the HA mode which is available in Cherry release, 2 servers are needed.
-
-
-ORAN INF Cherry test on Dell R740
-=================================
+*********************
+
+Following minimum hardware requirements must be met for installation of O-RAN INF image with AIO-DX:
+
++-------------------+----------------------------------------------------------------------------+
+| **HW Aspect** | **Requirement** |
+| | |
++-------------------+----------------------------------------------------------------------------+
+| **# of servers** | 2 |
++-------------------+----------------------------------------------------------------------------+
+| **CPU** | Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket |
+| | or |
+| | Single-CPU Intel® Xeon® D-15xx family, 8 cores (low-power/low-cost option) |
++-------------------+----------------------------------------------------------------------------+
+| **RAM** | 32G |
+| | |
++-------------------+----------------------------------------------------------------------------+
+| **Disk** | * Disk 1: 500G(It's better to be SSD) |
+| | * Disk 2: 1 or more 500 GB for Ceph OSD |
++-------------------+----------------------------------------------------------------------------+
+| **NICs** | * OAM: 1x1GE |
+| | * Data: 1 or more x 10GE (optional) |
++-------------------+----------------------------------------------------------------------------+
+| **BIOS settings** | * Hyper-Threading technology enabled |
+| | * Virtualization technology enabled |
+| | * VT for directed I/O enabled |
+| | * CPU power and performance policy set to performance |
+| | * CPU C state control disabled |
+| | * Plug & play BMC detection disabled |
++-------------------+----------------------------------------------------------------------------+
+
+Installation example (All-in-one Duplex) on HP ProLiant DL380p Gen8
+*******************************************************************
1. Installation for the first server from the O-RAN INF ISO image
-----------------------------------------------------------------
inf-image-aio-installer-intel-corei7-64.iso
1.1 Burn the image to the USB device
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+++++++++++++++++++++++++++++++++++++
- Assume the the usb device is /dev/sdX here
$ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
-1.2 Install the first Dell-R740 server (controller-0)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+1.2 Install the first server (controller-0)
++++++++++++++++++++++++++++++++++++++++++++
- Reboot the target from the USB device.
---------------------------------------------
2.1 First Login with sysadmin/sysadmin and change password
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
2.2 Set OAM network before bootstrap
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+++++++++++++++++++++++++++++++++++++
::
sudo ip route add default via $DEFAULT_OAM_GATEWAY dev $OAM_DEV
2.3 Login the server through SSH with sysadmin
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+++++++++++++++++++++++++++++++++++++++++++++++
2.4 Prepare the localhost.yml for bootstrap
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++++++++++++++++++++++
::
EOF
2.5 Run the ansible bootstrap
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++++++++
::
localhost : ok=257 changed=151 unreachable=0 failed=0 skipped=214 rescued=0 ignored=0
2.6 Congiure controller-0
-~~~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++++
Acquire admin credentials:
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-17T00:30:45.265032+00:00 |
- | updated_at | 2020-11-17T01:03:39.031612+00:00 |
+ | created_at | 2021-11-17T00:30:45.265032+00:00 |
+ | updated_at | 2021-11-17T01:03:39.031612+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
+-----------------+--------------------------------------+
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-17T00:28:32.365863+00:00 |
- | updated_at | 2020-11-17T01:03:45.090904+00:00 |
+ | created_at | 2021-11-17T00:28:32.365863+00:00 |
+ | updated_at | 2021-11-17T01:03:45.090904+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| accelerated | [True] |
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-17T00:28:32.612230+00:00 |
- | updated_at | 2020-11-17T01:03:47.341003+00:00 |
+ | created_at | 2021-11-17T00:28:32.612230+00:00 |
+ | updated_at | 2021-11-17T01:03:47.341003+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| accelerated | [True] |
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-17T00:28:32.365863+00:00 |
- | updated_at | 2020-11-17T01:03:49.368879+00:00 |
+ | created_at | 2021-11-17T00:28:32.365863+00:00 |
+ | updated_at | 2021-11-17T01:03:49.368879+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| accelerated | [True] |
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-17T00:28:32.612230+00:00 |
- | updated_at | 2020-11-17T01:03:53.143795+00:00 |
+ | created_at | 2021-11-17T00:28:32.612230+00:00 |
+ | updated_at | 2021-11-17T01:03:53.143795+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| accelerated | [True] |
| vlan_id | 18 |
| uses | [u'pxeboot0'] |
| used_by | [] |
- | created_at | 2020-11-17T01:03:57.303000+00:00 |
+ | created_at | 2021-11-17T01:03:57.303000+00:00 |
| updated_at | None |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| vlan_id | 19 |
| uses | [u'pxeboot0'] |
| used_by | [] |
- | created_at | 2020-11-17T01:04:02.613518+00:00 |
+ | created_at | 2021-11-17T01:04:02.613518+00:00 |
| updated_at | None |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| uuid | 3206cf01-c64a-457e-ac66-b8224c9684c3 |
| ntpservers | 0.pool.ntp.org,1.pool.ntp.org |
| isystem_uuid | cc79b616-d24e-4432-a953-85c9b242cb3a |
- | created_at | 2020-11-17T00:27:23.529571+00:00 |
+ | created_at | 2021-11-17T00:27:23.529571+00:00 |
| updated_at | None |
+--------------+--------------------------------------+
| idisk_uuid | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 |
| tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
| tier_name | storage |
- | created_at | 2020-11-17T01:05:04.063823+00:00 |
+ | created_at | 2021-11-17T01:05:04.063823+00:00 |
| updated_at | None |
+------------------+-------------------------------------------------------+
[sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
| idisk_uuid | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 |
| tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
| tier_name | storage |
- | created_at | 2020-11-17T01:05:06.939798+00:00 |
+ | created_at | 2021-11-17T01:05:06.939798+00:00 |
| updated_at | None |
+------------------+-------------------------------------------------------+
[sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
+--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
2.7 Unlock controller-0
-~~~~~~~~~~~~~~~~~~~~~~~
++++++++++++++++++++++++
::
| config_status | None |
| config_target | c6ae9b2d-a3c4-4751-a79e-5487ba81ed82 |
| console | ttyS0,115200 |
- | created_at | 2020-11-17T00:28:01.983673+00:00 |
+ | created_at | 2021-11-17T00:28:01.983673+00:00 |
| hostname | controller-0 |
| id | 1 |
| install_output | graphical |
| reserved | False |
| rootfs_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
| serialid | None |
- | software_load | 19.12 |
+ | software_load | 21.05 |
| subfunction_avail | online |
| subfunction_oper | disabled |
| subfunctions | controller,worker,lowlatency |
| task | Unlocking |
| tboot | false |
| ttys_dcd | None |
- | updated_at | 2020-11-17T01:05:07.015414+00:00 |
+ | updated_at | 2021-11-17T01:05:07.015414+00:00 |
| uptime | 3496 |
| uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
| vim_progress_status | None |
| 1 | controller-0 | controller | unlocked | enabled | available |
+----+--------------+-------------+----------------+-------------+--------------+
-2. Installation for the second server (controller-1)
+3. Installation for the second server (controller-1)
----------------------------------------------------
-2.1 Power on the controller-1 server and force it to network boot
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+3.1 Power on the controller-1 server and force it to network boot
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-2.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+3.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-2.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+3.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
::
| 2 | None | None | locked | disabled | offline |
+----+--------------+-------------+----------------+-------------+--------------+
-2.4 Using the host id, set the personality of this host to 'controller’:
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+3.4 Using the host id, set the personality of this host to 'controller':
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
::
| config_status | None |
| config_target | None |
| console | ttyS0,115200 |
- | created_at | 2020-11-17T10:17:44.387813+00:00 |
+ | created_at | 2021-11-17T10:17:44.387813+00:00 |
| hostname | controller-1 |
| id | 2 |
| install_output | text |
| reserved | False |
| rootfs_device | /dev/sda |
| serialid | None |
- | software_load | 19.12 |
+ | software_load | 21.05 |
| subfunction_avail | not-installed |
| subfunction_oper | disabled |
| subfunctions | controller,worker,lowlatency |
| vim_progress_status | None |
+-----------------------+--------------------------------------+
-2.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+3.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
This can take 5-10 minutes, depending on the performance of the host
machine.
| 2 | controller-1 | controller | locked | disabled | online |
+----+--------------+-------------+----------------+-------------+--------------+
-2.6 Configure controller-1
-~~~~~~~~~~~~~~~~~~~~~~~~~~
+3.6 Configure controller-1
+++++++++++++++++++++++++++
::
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-22T14:01:32.365863+00:00 |
- | updated_at | 2020-11-22T15:04:45.090904+00:00 |
+ | created_at | 2021-11-22T14:01:32.365863+00:00 |
+ | updated_at | 2021-11-22T15:04:45.090904+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| accelerated | [True] |
| vlan_id | None |
| uses | [] |
| used_by | [] |
- | created_at | 2020-11-22T14:05:16.052229+00:00 |
- | updated_at | 2020-11-22T15:08:35.324634+00:00 |
+ | created_at | 2021-11-22T14:05:16.052229+00:00 |
+ | updated_at | 2021-11-22T15:08:35.324634+00:00 |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| accelerated | [True] |
| vlan_id | 19 |
| uses | [u'pxeboot0'] |
| used_by | [] |
- | created_at | 2020-11-22T15:08:43.932209+00:00 |
+ | created_at | 2021-11-22T15:08:43.932209+00:00 |
| updated_at | None |
| sriov_numvfs | 0 |
| sriov_vf_driver | None |
| idisk_uuid | 1a3f0a36-5961-42e5-a271-e71db1c25d42 |
| tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
| tier_name | storage |
- | created_at | 2020-11-22T15:11:55.641193+00:00 |
+ | created_at | 2021-11-22T15:11:55.641193+00:00 |
| updated_at | None |
+------------------+-------------------------------------------------------+
| idisk_uuid | eddd732f-2cea-49b3-86db-b722c0b1a1ae |
| tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
| tier_name | storage |
- | created_at | 2020-11-22T15:12:04.274839+00:00 |
+ | created_at | 2021-11-22T15:12:04.274839+00:00 |
| updated_at | None |
+------------------+-------------------------------------------------------+
| 5be88c7a-3a94-4b97-9da5-b247bb89406c | osd | 3 | configuring-on-unlock | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
+--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
-2.7 Unlock controller-1
-~~~~~~~~~~~~~~~~~~~~~~~
+3.7 Unlock controller-1
++++++++++++++++++++++++
Unlock controller-1 in order to bring it into service:
| config_status | Config out-of-date |
| config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
| console | ttyS0,115200 |
- | created_at | 2020-11-22T12:58:11.630526+00:00 |
+ | created_at | 2021-11-22T12:58:11.630526+00:00 |
| hostname | controller-1 |
| id | 3 |
| install_output | text |
| reserved | False |
| rootfs_device | /dev/sda |
| serialid | None |
- | software_load | 19.12 |
+ | software_load | 21.05 |
| subfunction_avail | online |
| subfunction_oper | disabled |
| subfunctions | controller,worker,lowlatency |
| task | Unlocking |
| tboot | false |
| ttys_dcd | None |
- | updated_at | 2020-11-22T15:13:09.716324+00:00 |
+ | updated_at | 2021-11-22T15:13:09.716324+00:00 |
| uptime | 752 |
| uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
| vim_progress_status | None |
| config_status | None |
| config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
| console | ttyS0,115200 |
- | created_at | 2020-11-22T12:58:11.630526+00:00 |
+ | created_at | 2021-11-22T12:58:11.630526+00:00 |
| hostname | controller-1 |
| id | 2 |
| install_output | text |
| reserved | False |
| rootfs_device | /dev/sda |
| serialid | None |
- | software_load | 19.12 |
+ | software_load | 21.05 |
| subfunction_avail | available |
| subfunction_oper | enabled |
| subfunctions | controller,worker,lowlatency |
| task | |
| tboot | false |
| ttys_dcd | None |
- | updated_at | 2020-11-22T23:59:07.787759+00:00 |
+ | updated_at | 2021-11-22T23:59:07.787759+00:00 |
| uptime | 31008 |
| uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
| vim_progress_status | services-enabled |
+-----------------------+-----------------------------------------------------------------------+
-3. Simple use case for sriov
-````````````````````````````
+4. Simple use case for sriov
+----------------------------
-3.1 After controller-0 is rebooted and up running, download the DPDK
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+4.1 After controller-0 is rebooted and up running, download the DPDK
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
::
[sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
[sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
Password:
- --2020-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
+ --2021-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
[========================================================================================>] 9.78M
1.48MB/s in 6.8s
- 2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
+ 2021-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
sudo tar xvf dpdk-17.11.10.tar.xz
sudo ln -s dpdk-stable-17.11.10 dpdk-stable
-3.2 Prepare the yaml file for the network assignment container
-''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
+4.2 Prepare the yaml file for the network assignment container
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
The following the exmaple of the yaml file:
> }'
> EOF
-3.3 Run the network assignent container for the 2 VFs
-'''''''''''''''''''''''''''''''''''''''''''''''''''''
+4.3 Run the network assignent container for the 2 VFs
++++++++++++++++++++++++++++++++++++++++++++++++++++++
::
networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
-3.4 Prepare the VF container yaml file
-''''''''''''''''''''''''''''''''''''''
+4.4 Prepare the VF container yaml file
+++++++++++++++++++++++++++++++++++++++
::
> path: /usr/src/
> EOF
-3.5 Run the VF container
-''''''''''''''''''''''''
+4.5 Run the VF container
+++++++++++++++++++++++++
Start the VF container.
References
-----------
+**********
- `StarlingX`_
Code Review / pti / rtp.git / blobdiff