:depth: 3
:local:
-Abstract
+Overview
********
-This document describes how to install O-RAN INF image, example configuration (All-in-one Duplex)
-for better real time performance, and example deployment of Kubernetes cluster and plugins.
+O-RAN INF is a downstream project of `StarlingX`_, and use the same installation and deployment methods.
-The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
-and container technology.
+Please see the detail of all the supported `Deployment Configurations`_.
+Notes: For Yocto based image, only "All-in-one Simplex" and "All-in-one Duplex (up to 50 worker nodes)" are supported.
+
+.. _`Deployment Configurations`: https://docs.starlingx.io/r/stx.7.0/deploy/index-deploy-da06a98b83b1.html
Preface
*******
-Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
+Before starting the installation and deployment of O-RAN INF, you need to download the released ISO image or build from source as described in developer-guide.
+
+The INF project supports Multi OS and the latest released images for each based OS can be dwonloaded in:
+ - CentOS 7: `inf-image-centos-all-x86-64.iso`_
+ - Debian 11 (bullseye): `inf-image-debian-all-x86-64.iso`_
+ - Yocto 2.7 (warrior): `inf-image-yocto-aio-x86-64.iso`_
+.. _`inf-image-debian-all-x86-64.iso`: https://nexus.o-ran-sc.org/content/sites/images/org/o-ran-sc/pti/rtp/g-release/inf-image-debian-all-x86-64.iso
+.. _`inf-image-centos-all-x86-64.iso`: https://nexus.o-ran-sc.org/content/sites/images/org/o-ran-sc/pti/rtp/g-release/inf-image-centos-all-x86-64.iso
+.. _`inf-image-yocto-aio-x86-64.iso`: https://nexus.o-ran-sc.org/content/sites/images/org/o-ran-sc/pti/rtp/g-release/inf-image-yocto-aio-x86-64.iso
Hardware Requirements
*********************
-Following minimum hardware requirements must be met for installation of O-RAN INF image with AIO-DX:
-
-+-------------------+----------------------------------------------------------------------------+
-| **HW Aspect** | **Requirement** |
-| | |
-+-------------------+----------------------------------------------------------------------------+
-| **# of servers** | 2 |
-+-------------------+----------------------------------------------------------------------------+
-| **CPU** | Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket |
-| | or |
-| | Single-CPU Intel® Xeon® D-15xx family, 8 cores (low-power/low-cost option) |
-+-------------------+----------------------------------------------------------------------------+
-| **RAM** | 32G |
-| | |
-+-------------------+----------------------------------------------------------------------------+
-| **Disk** | * Disk 1: 500G(It's better to be SSD) |
-| | * Disk 2: 1 or more 500 GB for Ceph OSD |
-+-------------------+----------------------------------------------------------------------------+
-| **NICs** | * OAM: 1x1GE |
-| | * Data: 1 or more x 10GE (optional) |
-+-------------------+----------------------------------------------------------------------------+
-| **BIOS settings** | * Hyper-Threading technology enabled |
-| | * Virtualization technology enabled |
-| | * VT for directed I/O enabled |
-| | * CPU power and performance policy set to performance |
-| | * CPU C state control disabled |
-| | * Plug & play BMC detection disabled |
-+-------------------+----------------------------------------------------------------------------+
-
-Installation example (All-in-one Duplex) on HP ProLiant DL380p Gen8
-*******************************************************************
-
-1. Installation for the first server from the O-RAN INF ISO image
------------------------------------------------------------------
-
-- Please see the README.md file for how to build the image.
-- The Image is a live ISO image with CLI installer:
- inf-image-aio-installer-intel-corei7-64.iso
-
-1.1 Burn the image to the USB device
-++++++++++++++++++++++++++++++++++++
-
-- Assume the the usb device is /dev/sdX here
-
-::
-
- $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
-
-1.2 Install the first server (controller-0)
-+++++++++++++++++++++++++++++++++++++++++++
-
-- Reboot the target from the USB device.
-
-- Select “All-in-one Graphics console” or “All-in-one Serial console
- install” and press ENTER
-
-- Start the auto installation
-
-- It will reboot aotumatically after installation
-
-2. Configuration and initialize the bootstrap
----------------------------------------------
-
-2.1 First Login with sysadmin/sysadmin and change password
-++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-2.2 Set OAM network before bootstrap
-++++++++++++++++++++++++++++++++++++
-
-::
-
- export OAM_DEV=eno3
- export CONTROLLER0_OAM_CIDR=128.224.210.110/24
- export DEFAULT_OAM_GATEWAY=128.224.210.1
- sudo ip address add $CONTROLLER0_OAM_CIDR dev $OAM_DEV
- sudo ip link set up dev $OAM_DEV
- sudo ip route add default via $DEFAULT_OAM_GATEWAY dev $OAM_DEV
-
-2.3 Login the server through SSH with sysadmin
-++++++++++++++++++++++++++++++++++++++++++++++
-
-2.4 Prepare the localhost.yml for bootstrap
-+++++++++++++++++++++++++++++++++++++++++++
-
-::
-
- cat << EOF > localhost.yml
- system_mode: duplex
- management_subnet: 192.168.18.0/24
- management_start_address: 192.168.18.2
- management_end_address: 192.168.18.50
- management_gateway_address: 192.168.18.1
- external_oam_subnet: 128.224.210.0/24
- external_oam_gateway_address: 128.224.210.1
- external_oam_floating_address: 128.224.210.110
- external_oam_node_0_address: 128.224.210.111
- external_oam_node_1_address: 128.224.210.112
- EOF
-
-2.5 Run the ansible bootstrap
-+++++++++++++++++++++++++++++
-
-::
-
- ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
-
-After the bootstrap successfully finish, it will show as following:
-
-::
-
- PLAY RECAP *************************************************************************************************************
- localhost : ok=257 changed=151 unreachable=0 failed=0 skipped=214 rescued=0 ignored=0
-
-2.6 Congiure controller-0
-+++++++++++++++++++++++++
-
-Acquire admin credentials:
-
-::
-
- controller-0:~$ source /etc/platform/openrc
- [sysadmin@controller-0 ~(keystone_admin)]$
-
-Configure the OAM and MGMT interfaces of controller-0 and specify the
-attached networks:
-
-::
-
- OAM_IF=eno3
- MGMT_IF=eno1
- system host-if-modify controller-0 lo -c none
- IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
- for UUID in $IFNET_UUIDS; do
- system interface-network-remove ${UUID}
- done
-
- system host-if-modify controller-0 $OAM_IF -n oam0
- system host-if-modify controller-0 $MGMT_IF -n pxeboot0
-
- system host-if-modify controller-0 oam0 -c platform
- system interface-network-assign controller-0 oam0 oam
-
- system host-if-modify controller-0 pxeboot0 -c platform
- system interface-network-assign controller-0 pxeboot0 pxeboot
-
- system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
- system interface-network-assign controller-0 mgmt0 mgmt
-
- system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
- system interface-network-assign controller-0 cluster0 cluster-host
-
-Example output:
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
- [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 lo -c none
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | lo |
- | iftype | virtual |
- | ports | [] |
- | imac | 00:00:00:00:00:00 |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | 08c95952-892b-40b5-b17a-7d2ad46e725c |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-17T00:30:45.265032+00:00 |
- | updated_at | 2021-11-17T01:03:39.031612+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
- [sysadmin@controller-0 ~(keystone_admin)]$ for UUID in $IFNET_UUIDS; do
- > system interface-network-remove ${UUID}
- > done
- Deleted Interface Network: 0bf11f1b-4fc6-4e97-b896-3d6393a3744e
- Deleted Interface Network: a62d95f6-ad4e-4779-bfc0-6a885067f8d8
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -n oam0
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | oam0 |
- | iftype | ethernet |
- | ports | [u'eno3'] |
- | imac | 24:6e:96:5d:0c:b2 |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | d8a048fa-67ef-43ac-8166-671be93caa30 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-17T00:28:32.365863+00:00 |
- | updated_at | 2021-11-17T01:03:45.090904+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $MGMT_IF -n pxeboot0
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | pxeboot0 |
- | iftype | ethernet |
- | ports | [u'eno1'] |
- | imac | 24:6e:96:5d:0c:92 |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-17T00:28:32.612230+00:00 |
- | updated_at | 2021-11-17T01:03:47.341003+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 oam0 -c platform
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | oam0 |
- | iftype | ethernet |
- | ports | [u'eno3'] |
- | imac | 24:6e:96:5d:0c:b2 |
- | imtu | 1500 |
- | ifclass | platform |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | d8a048fa-67ef-43ac-8166-671be93caa30 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-17T00:28:32.365863+00:00 |
- | updated_at | 2021-11-17T01:03:49.368879+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 oam
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-0 |
- | uuid | 3c8bd181-d3f3-4e14-8e89-75a3432db1e4 |
- | ifname | oam0 |
- | network_name | oam |
- +--------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 pxeboot0 -c platform
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | pxeboot0 |
- | iftype | ethernet |
- | ports | [u'eno1'] |
- | imac | 24:6e:96:5d:0c:92 |
- | imtu | 1500 |
- | ifclass | platform |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-17T00:28:32.612230+00:00 |
- | updated_at | 2021-11-17T01:03:53.143795+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 pxeboot0 pxeboot
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-0 |
- | uuid | 6c55622d-2da4-4f4e-ab5e-f8e06e03af7c |
- | ifname | pxeboot0 |
- | network_name | pxeboot |
- +--------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | mgmt0 |
- | iftype | vlan |
- | ports | [] |
- | imac | 24:6e:96:5d:0c:92 |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | 18 |
- | uses | [u'pxeboot0'] |
- | used_by | [] |
- | created_at | 2021-11-17T01:03:57.303000+00:00 |
- | updated_at | None |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 mgmt0 mgmt
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-0 |
- | uuid | 2e93ef03-e9ee-457a-8667-05b52b7109a5 |
- | ifname | mgmt0 |
- | network_name | mgmt |
- +--------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | cluster0 |
- | iftype | vlan |
- | ports | [] |
- | imac | 24:6e:96:5d:0c:92 |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vlan_id | 19 |
- | uses | [u'pxeboot0'] |
- | used_by | [] |
- | created_at | 2021-11-17T01:04:02.613518+00:00 |
- | updated_at | None |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 cluster0 cluster-host
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-0 |
- | uuid | fb8b6be6-1618-4662-b063-b1e8d340aa48 |
- | ifname | cluster0 |
- | network_name | cluster-host |
- +--------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-0
- +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
- | uuid | name | class | type | vlan id | ports | uses i/f | used by i/f | attributes |
- +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
- | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 | mgmt0 | platform | vlan | 18 | [] | [u'pxeboot0'] | [] | MTU=1500 |
- | 23b5e923-1e53-4e70-a975-542d8380b7f2 | pxeboot0 | platform | ethernet | None | [u'eno1'] | [] | [u'mgmt0', u'cluster0'] | MTU=1500 |
- | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 | cluster0 | platform | vlan | 19 | [] | [u'pxeboot0'] | [] | MTU=1500 |
- | d8a048fa-67ef-43ac-8166-671be93caa30 | oam0 | platform | ethernet | None | [u'eno3'] | [] | [] | MTU=1500 |
- +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
-
-Configure NTP Servers for network time synchronization:
-
-::
-
- system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
-
-Output
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | uuid | 3206cf01-c64a-457e-ac66-b8224c9684c3 |
- | ntpservers | 0.pool.ntp.org,1.pool.ntp.org |
- | isystem_uuid | cc79b616-d24e-4432-a953-85c9b242cb3a |
- | created_at | 2021-11-17T00:27:23.529571+00:00 |
- | updated_at | None |
- +--------------+--------------------------------------+
-
-Add an OSD on controller-0 for Ceph:
-
-::
-
- system host-disk-list controller-0
- system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
- system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
- system host-stor-list controller-0
-
-Output
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
- | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path |
- | | de | num | type | gib | gib | | | |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
- | 8e2a719a-fa5a-4c25-89af-70a23fb7b238 | /dev/sda | 2048 | HDD | 893. | 644.726 | Undetermined | 00c66a07604fa8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
- | | | | | 75 | | | | |
- | | | | | | | | | |
- | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/sdb | 2064 | HDD | 1117. | 1117.247 | Undetermined | 00c6b9139b76a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
- | | | | | 25 | | | | |
- | | | | | | | | | |
- | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/sdc | 2080 | HDD | 1117. | 1117.247 | Undetermined | 0053be63c794a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
- | | | | | 25 | | | | |
- | | | | | | | | | |
- | 4879b381-8e9f-48f3-84e2-f9c6a94bbfe0 | /dev/sdd | 2096 | HDD | 1117. | 0.0 | Undetermined | 0065482503bca8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
- | | | | | 25 | | | | |
- | | | | | | | | | |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
- +------------------+-------------------------------------------------------+
- | Property | Value |
- +------------------+-------------------------------------------------------+
- | osdid | 0 |
- | function | osd |
- | state | configuring-on-unlock |
- | journal_location | 0816c72f-a4f0-49ea-9a95-0f02c880717c |
- | journal_size_gib | 1024 |
- | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
- | journal_node | /dev/sdb2 |
- | uuid | 0816c72f-a4f0-49ea-9a95-0f02c880717c |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | idisk_uuid | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 |
- | tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
- | tier_name | storage |
- | created_at | 2021-11-17T01:05:04.063823+00:00 |
- | updated_at | None |
- +------------------+-------------------------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
- +------------------+-------------------------------------------------------+
- | Property | Value |
- +------------------+-------------------------------------------------------+
- | osdid | 1 |
- | function | osd |
- | state | configuring-on-unlock |
- | journal_location | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 |
- | journal_size_gib | 1024 |
- | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
- | journal_node | /dev/sdc2 |
- | uuid | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 |
- | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | idisk_uuid | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 |
- | tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
- | tier_name | storage |
- | created_at | 2021-11-17T01:05:06.939798+00:00 |
- | updated_at | None |
- +------------------+-------------------------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
- | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
- | 0816c72f-a4f0-49ea-9a95-0f02c880717c | osd | 0 | configuring-on-unlock | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2 | 1 | storage |
- | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 | osd | 1 | configuring-on-unlock | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
-
-2.7 Unlock controller-0
-+++++++++++++++++++++++
-
-::
-
- system host-unlock controller-0
-
-Output:
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
- +-----------------------+-------------------------------------------------+
- | Property | Value |
- +-----------------------+-------------------------------------------------+
- | action | none |
- | administrative | locked |
- | availability | online |
- | bm_ip | None |
- | bm_type | none |
- | bm_username | None |
- | boot_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
- | capabilities | {u'stor_function': u'monitor'} |
- | clock_synchronization | ntp |
- | config_applied | 6aa15fb4-8cb3-494e-b94e-95f85b560f22 |
- | config_status | None |
- | config_target | c6ae9b2d-a3c4-4751-a79e-5487ba81ed82 |
- | console | ttyS0,115200 |
- | created_at | 2021-11-17T00:28:01.983673+00:00 |
- | hostname | controller-0 |
- | id | 1 |
- | install_output | graphical |
- | install_state | None |
- | install_state_info | None |
- | inv_state | inventoried |
- | invprovision | provisioning |
- | location | {} |
- | mgmt_ip | 192.168.18.3 |
- | mgmt_mac | 24:6e:96:5d:0c:92 |
- | operational | disabled |
- | personality | controller |
- | reserved | False |
- | rootfs_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
- | serialid | None |
- | software_load | 21.05 |
- | subfunction_avail | online |
- | subfunction_oper | disabled |
- | subfunctions | controller,worker,lowlatency |
- | task | Unlocking |
- | tboot | false |
- | ttys_dcd | None |
- | updated_at | 2021-11-17T01:05:07.015414+00:00 |
- | uptime | 3496 |
- | uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
- | vim_progress_status | None |
- +-----------------------+-------------------------------------------------+
-
-Controller-0 will reboot to apply configuration changes and come into
-service. This can take 5-10 minutes, depending on the performance of the
-host machine.
-
-Once the controller comes back up, check the status of controller-0
-
-::
-
- controller-0:~$ source /etc/platform/openrc
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
- +----+--------------+-------------+----------------+-------------+--------------+
- | id | hostname | personality | administrative | operational | availability |
- +----+--------------+-------------+----------------+-------------+--------------+
- | 1 | controller-0 | controller | unlocked | enabled | available |
- +----+--------------+-------------+----------------+-------------+--------------+
-
-3. Installation for the second server (controller-1)
-----------------------------------------------------
-
-3.1 Power on the controller-1 server and force it to network boot
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-3.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
-++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-3.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
-++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
- +----+--------------+-------------+----------------+-------------+--------------+
- | id | hostname | personality | administrative | operational | availability |
- +----+--------------+-------------+----------------+-------------+--------------+
- | 1 | controller-0 | controller | unlocked | enabled | degraded |
- | 2 | None | None | locked | disabled | offline |
- +----+--------------+-------------+----------------+-------------+--------------+
-
-3.4 Using the host id, set the personality of this host to 'controller':
-++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller
- +-----------------------+--------------------------------------+
- | Property | Value |
- +-----------------------+--------------------------------------+
- | action | none |
- | administrative | locked |
- | availability | offline |
- | bm_ip | None |
- | bm_type | None |
- | bm_username | None |
- | boot_device | /dev/sda |
- | capabilities | {} |
- | clock_synchronization | ntp |
- | config_applied | None |
- | config_status | None |
- | config_target | None |
- | console | ttyS0,115200 |
- | created_at | 2021-11-17T10:17:44.387813+00:00 |
- | hostname | controller-1 |
- | id | 2 |
- | install_output | text |
- | install_state | None |
- | install_state_info | None |
- | inv_state | None |
- | invprovision | None |
- | location | {} |
- | mgmt_ip | 192.168.18.4 |
- | mgmt_mac | 24:6e:96:5d:38:ee |
- | operational | disabled |
- | personality | controller |
- | reserved | False |
- | rootfs_device | /dev/sda |
- | serialid | None |
- | software_load | 21.05 |
- | subfunction_avail | not-installed |
- | subfunction_oper | disabled |
- | subfunctions | controller,worker,lowlatency |
- | task | None |
- | tboot | false |
- | ttys_dcd | None |
- | updated_at | None |
- | uptime | 0 |
- | uuid | f069381d-9743-49cc-bf8b-eb4bd3972203 |
- | vim_progress_status | None |
- +-----------------------+--------------------------------------+
+* For INF platform Hardware Requirements, refer to `System Hardware Requirements`_.
+* For INF Openstack Hardware Requirements, refer to `Openstack Hardware Requirements`_.
-3.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
-+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+* And you can also refer to the `Verified Commercial Hardware`_.
-This can take 5-10 minutes, depending on the performance of the host
-machine.
+.. _`System Hardware Requirements`: https://docs.starlingx.io/planning/kubernetes/starlingx-hardware-requirements.html
+.. _`Verified Commercial Hardware`: https://docs.starlingx.io/planning/kubernetes/verified-commercial-hardware.html
+.. _`Openstack Hardware Requirements`: https://docs.starlingx.io/planning/openstack/hardware-requirements.html
-::
+Installation
+************
- [root@controller-0 hieradata(keystone_admin)]$ system host-list
- +----+--------------+-------------+----------------+-------------+--------------+
- | id | hostname | personality | administrative | operational | availability |
- +----+--------------+-------------+----------------+-------------+--------------+
- | 1 | controller-0 | controller | unlocked | enabled | available |
- | 2 | controller-1 | controller | locked | disabled | online |
- +----+--------------+-------------+----------------+-------------+--------------+
+Platform Installation
+---------------------
-3.6 Configure controller-1
-++++++++++++++++++++++++++
+INF uses the same installation and deployment methods of StarlingX, please refer to `StarlingX Installation`_ for the detail installation steps.
-::
+.. _`StarlingX Installation`: https://docs.starlingx.io/r/stx.7.0/deploy_install_guides/r7_release/index-install-r7-8966076f0e81.html
- OAM_IF=eno3
- MGMT_IF=eno1
- system host-if-modify controller-1 $OAM_IF -n oam0
- system host-if-modify controller-1 oam0 -c platform
- system interface-network-assign controller-1 oam0 oam
- system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
- system interface-network-assign controller-1 cluster0 cluster-host
+Applications Installation
+-------------------------
- system host-if-list controller-1
+Here are the example applications installations:
+* `Install O-RAN O2`_
+* `Install FlexRAN`_
- system host-disk-list controller-1
- system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
- system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
- system host-stor-list controller-1
+.. _`Install O-RAN O2`: https://docs.starlingx.io/admintasks/kubernetes/oran-o2-application-b50a0c899e66.html
+.. _`Install FlexRAN`: https://docs.starlingx.io/sample_apps/flexran/deploy-flexran-2203-on-starlingx-1d1b15ecb16f.html
-Output:
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
- [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 $OAM_IF -n oam0
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | oam0 |
- | iftype | ethernet |
- | ports | [u'eno3'] |
- | imac | 24:6e:96:5d:39:0e |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | c2473511-d0d6-445d-9739-4d43dc029de9 |
- | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-22T14:01:32.365863+00:00 |
- | updated_at | 2021-11-22T15:04:45.090904+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 oam0 -c platform
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | oam0 |
- | iftype | ethernet |
- | ports | [u'eno3'] |
- | imac | 24:6e:96:5d:39:0e |
- | imtu | 1500 |
- | ifclass | platform |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | c2473511-d0d6-445d-9739-4d43dc029de9 |
- | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2021-11-22T14:05:16.052229+00:00 |
- | updated_at | 2021-11-22T15:08:35.324634+00:00 |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 oam0 oam
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-1 |
- | uuid | f2e7f088-0dd0-4adc-8348-4e3cef23bc47 |
- | ifname | oam0 |
- | network_name | oam |
- +--------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | cluster0 |
- | iftype | vlan |
- | ports | [] |
- | imac | 24:6e:96:5d:38:ee |
- | imtu | 1500 |
- | ifclass | None |
- | aemode | None |
- | schedpolicy | None |
- | txhashpolicy | None |
- | uuid | b6783682-b2aa-4135-90d2-676e1db41ae8 |
- | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | vlan_id | 19 |
- | uses | [u'pxeboot0'] |
- | used_by | [] |
- | created_at | 2021-11-22T15:08:43.932209+00:00 |
- | updated_at | None |
- | sriov_numvfs | 0 |
- | sriov_vf_driver | None |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 cluster0 cluster-host
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-1 |
- | uuid | 8fc64805-b54b-45a4-b88a-e13b236abfe8 |
- | ifname | cluster0 |
- | network_name | cluster-host |
- +--------------+--------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-1
- +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
- | uuid | name | class | type | vlan id | ports | uses i/f | used by i/f | attributes |
- +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
- | b6783682-b2aa-4135-90d2-676e1db41ae8 | cluster0 | platform | vlan | 19 | [] | [u'pxeboot0'] | [] | MTU=1500 |
- | b8921960-fde5-44c3-960d-2aebf42ea400 | pxeboot0 | platform | ethernet | None | [u'eno1'] | [] | [u'mgmt0', u'cluster0'] | MTU=1500 |
- | c103275b-2b75-4568-865f-ac6be32ecb2d | mgmt0 | platform | vlan | 18 | [] | [u'pxeboot0'] | [] | MTU=1500 |
- | c2473511-d0d6-445d-9739-4d43dc029de9 | oam0 | platform | ethernet | None | [u'eno3'] | [] | [] | MTU=1500 |
- +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
- | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path |
- | | de | num | type | gib | gib | | | |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
- | 5b8fade4-b048-48fa-b906-9dcbdbed8e96 | /dev/sda | 2048 | HDD | 893. | 644.726 | Undetermined | 00cbd97f3e36ccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
- | | | | | 75 | | | | |
- | | | | | | | | | |
- | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/sdb | 2064 | HDD | 1117. | 1117.247 | Undetermined | 006d0e977b5fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
- | | | | | 25 | | | | |
- | | | | | | | | | |
- | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/sdc | 2080 | HDD | 1117. | 1117.247 | Undetermined | 003a2377ac7fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
- | | | | | 25 | | | | |
- | | | | | | | | | |
- | 774c3cd0-1178-4145-9573-f0d6dee2ba06 | /dev/sdd | 2096 | HDD | 1117. | 1117.247 | Undetermined | 00d7093ef0adccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
- | | | | | 25 | | | | |
- | | | | | | | | | |
- | 00361302-8d55-4730-855c-b0098c73ab7e | /dev/sde | 2112 | SSD | 223. | 223.568 | N/A | PHDW730104QM240E | /dev/disk/by-path/pci-0000:d8:00.0-ata-1 |
- | | | | | 57 | | | | |
- | | | | | | | | | |
- | 7ce735e6-920f-4424-a890-a7a7f48d7632 | /dev/sdf | 2128 | SSD | 223. | 223.568 | N/A | PHDW730104LL240E | /dev/disk/by-path/pci-0000:d8:00.0-ata-2 |
- | | | | | 57 | | | | |
- | | | | | | | | | |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
- +------------------+-------------------------------------------------------+
- | Property | Value |
- +------------------+-------------------------------------------------------+
- | osdid | 2 |
- | function | osd |
- | state | configuring-on-unlock |
- | journal_location | 54a218d8-0466-4366-9ef0-3ec5a952fde7 |
- | journal_size_gib | 1024 |
- | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
- | journal_node | /dev/sdb2 |
- | uuid | 54a218d8-0466-4366-9ef0-3ec5a952fde7 |
- | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | idisk_uuid | 1a3f0a36-5961-42e5-a271-e71db1c25d42 |
- | tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
- | tier_name | storage |
- | created_at | 2021-11-22T15:11:55.641193+00:00 |
- | updated_at | None |
- +------------------+-------------------------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
- +------------------+-------------------------------------------------------+
- | Property | Value |
- +------------------+-------------------------------------------------------+
- | osdid | 3 |
- | function | osd |
- | state | configuring-on-unlock |
- | journal_location | 5be88c7a-3a94-4b97-9da5-b247bb89406c |
- | journal_size_gib | 1024 |
- | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
- | journal_node | /dev/sdc2 |
- | uuid | 5be88c7a-3a94-4b97-9da5-b247bb89406c |
- | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | idisk_uuid | eddd732f-2cea-49b3-86db-b722c0b1a1ae |
- | tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
- | tier_name | storage |
- | created_at | 2021-11-22T15:12:04.274839+00:00 |
- | updated_at | None |
- +------------------+-------------------------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-1
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
- | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
- | 54a218d8-0466-4366-9ef0-3ec5a952fde7 | osd | 2 | configuring-on-unlock | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2 | 1 | storage |
- | 5be88c7a-3a94-4b97-9da5-b247bb89406c | osd | 3 | configuring-on-unlock | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
-
-3.7 Unlock controller-1
-+++++++++++++++++++++++
-
-Unlock controller-1 in order to bring it into service:
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-1
- +-----------------------+--------------------------------------+
- | Property | Value |
- +-----------------------+--------------------------------------+
- | action | none |
- | administrative | locked |
- | availability | online |
- | bm_ip | None |
- | bm_type | None |
- | bm_username | None |
- | boot_device | /dev/sda |
- | capabilities | {u'stor_function': u'monitor'} |
- | clock_synchronization | ntp |
- | config_applied | None |
- | config_status | Config out-of-date |
- | config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
- | console | ttyS0,115200 |
- | created_at | 2021-11-22T12:58:11.630526+00:00 |
- | hostname | controller-1 |
- | id | 3 |
- | install_output | text |
- | install_state | None |
- | install_state_info | None |
- | inv_state | inventoried |
- | invprovision | unprovisioned |
- | location | {} |
- | mgmt_ip | 192.168.18.4 |
- | mgmt_mac | 24:6e:96:5d:38:ee |
- | operational | disabled |
- | personality | controller |
- | reserved | False |
- | rootfs_device | /dev/sda |
- | serialid | None |
- | software_load | 21.05 |
- | subfunction_avail | online |
- | subfunction_oper | disabled |
- | subfunctions | controller,worker,lowlatency |
- | task | Unlocking |
- | tboot | false |
- | ttys_dcd | None |
- | updated_at | 2021-11-22T15:13:09.716324+00:00 |
- | uptime | 752 |
- | uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | vim_progress_status | None |
- +-----------------------+--------------------------------------+
-
-Controller-1 will reboot in order to apply configuration changes and
-come into service. This can take 5-10 minutes, depending on the
-performance of the host machine.
-
-::
-
- [root@controller-0 hieradata(keystone_admin)]$ system host-list
- +----+--------------+-------------+----------------+-------------+--------------+
- | id | hostname | personality | administrative | operational | availability |
- +----+--------------+-------------+----------------+-------------+--------------+
- | 1 | controller-0 | controller | unlocked | enabled | available |
- | 2 | controller-1 | controller | unlocked | enabled | available |
- +----+--------------+-------------+----------------+-------------+--------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-1
- +-----------------------+-----------------------------------------------------------------------+
- | Property | Value |
- +-----------------------+-----------------------------------------------------------------------+
- | action | none |
- | administrative | unlocked |
- | availability | available |
- | bm_ip | None |
- | bm_type | None |
- | bm_username | None |
- | boot_device | /dev/sda |
- | capabilities | {u'stor_function': u'monitor', u'Personality': u'Controller-Standby'} |
- | clock_synchronization | ntp |
- | config_applied | 9747e0ce-2319-409d-b75c-2475bc5065ac |
- | config_status | None |
- | config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
- | console | ttyS0,115200 |
- | created_at | 2021-11-22T12:58:11.630526+00:00 |
- | hostname | controller-1 |
- | id | 2 |
- | install_output | text |
- | install_state | None |
- | install_state_info | None |
- | inv_state | inventoried |
- | invprovision | provisioned |
- | location | {} |
- | mgmt_ip | 192.168.18.4 |
- | mgmt_mac | 24:6e:96:5d:38:ee |
- | operational | enabled |
- | personality | controller |
- | reserved | False |
- | rootfs_device | /dev/sda |
- | serialid | None |
- | software_load | 21.05 |
- | subfunction_avail | available |
- | subfunction_oper | enabled |
- | subfunctions | controller,worker,lowlatency |
- | task | |
- | tboot | false |
- | ttys_dcd | None |
- | updated_at | 2021-11-22T23:59:07.787759+00:00 |
- | uptime | 31008 |
- | uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
- | vim_progress_status | services-enabled |
- +-----------------------+-----------------------------------------------------------------------+
-
-
-4. Simple use case for sriov
-----------------------------
-
-4.1 After controller-0 is rebooted and up running, download the DPDK
-++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
- [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
- Password:
- --2021-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
- Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
- Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
-
- HTTP request sent, awaiting response... 200 OK
- Length: 10251680 (9.8M) [application/octet-stream]
- Saving to: ‘dpdk-17.11.10.tar.xz’
-
- dpdk-17.11.10.tar.xz 100%
- [========================================================================================>] 9.78M
- 1.48MB/s in 6.8s
-
- 2021-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
-
- sudo tar xvf dpdk-17.11.10.tar.xz
-
- sudo ln -s dpdk-stable-17.11.10 dpdk-stable
-
-4.2 Prepare the yaml file for the network assignment container
-++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-The following the exmaple of the yaml file:
-
-::
-
- [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
- > apiVersion: "k8s.cni.cncf.io/v1"
- > kind: NetworkAttachmentDefinition
- > metadata:
- > name: sriov-data-dpdk-0
- > annotations:
- > k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
- > spec:
- > config: '{
- > "type": "sriov",
- > "name": "sriov-data-dpdk-0"
- > }'
- >
- > ---
- > apiVersion: "k8s.cni.cncf.io/v1"
- > kind: NetworkAttachmentDefinition
- > metadata:
- > name: sriov-data-dpdk-1
- > annotations:
- > k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
- > spec:
- > config: '{
- > "type": "sriov",
- > "name": "sriov-data-dpdk-1"
- > }'
- > EOF
-
-4.3 Run the network assignent container for the 2 VFs
-+++++++++++++++++++++++++++++++++++++++++++++++++++++
-
-::
-
- [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
- networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
- networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
-
-4.4 Prepare the VF container yaml file
-++++++++++++++++++++++++++++++++++++++
-
-::
-
- [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
- > apiVersion: v1
- > kind: Pod
- metadata:
- > metadata:
- > name: pod-with-dpdk-vfs-0
- > annotations:
- > k8s.v1.cni.cncf.io/networks: '[
- > { "name": "sriov-data-dpdk-0" },
- { "name": "sriov-data-dpdk-1" }
- > { "name": "sriov-data-dpdk-1" }
- > ]'
- > spec:
- > restartPolicy: Never
- > containers:
- > - name: pod-with-dpdk-vfs-0
- > image: wrsnfv/ubuntu-dpdk-build:v0.3
- > env:
- > - name: RTE_SDK
- > value: "/usr/src/dpdk"
- > command:
- > - sleep
- > - infinity
- > stdin: true
- > tty: true
- > securityContext:
- > privileged: true
- > capabilities:
- > add:
- > - ALL
- > resources:
- > requests:
- > cpu: 4
- > memory: 4Gi
- > intel.com/pci_sriov_net_physnet0: '1'
- > intel.com/pci_sriov_net_physnet1: '1'
- > limits:
- > cpu: 4
- > hugepages-1Gi: 2Gi
- > memory: 4Gi
- > intel.com/pci_sriov_net_physnet0: '1'
- > intel.com/pci_sriov_net_physnet1: '1'
- > volumeMounts:
- > - mountPath: /mnt/huge-1048576kB
- > name: hugepage
- > - name: dpdk-volume
- > mountPath: /usr/src/dpdk
- > - name: lib-volume
- > mountPath: /lib/modules
- > - name: src-volume
- > mountPath: /usr/src/
- > volumes:
- > - name: hugepage
- > emptyDir:
- > medium: HugePages
- > - name: dpdk-volume
- > hostPath:
- > path: /opt/dpdk-stable/
- > - name: lib-volume
- > hostPath:
- > path: /lib/modules
- > - name: src-volume
- > hostPath:
- > path: /usr/src/
- > EOF
-
-4.5 Run the VF container
-++++++++++++++++++++++++
-
-Start the VF container.
-
-::
-
- [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
- pod/pod-with-dpdk-vfs-0 created
-
- [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
- NAME READY STATUS RESTARTS AGE
- pod-with-dpdk-vfs-0 1/1 Running 0 6m40s
-
-Login the VF container
-
-::
-
- kubectl exec -it pod-with-dpdk-vfs-0 -- bash
-
-Build the DPDK
-
-::
-
- cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
-
- root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
- HOSTCC scripts/basic/fixdep
- HOSTCC scripts/kconfig/conf.o
- HOSTCC scripts/kconfig/confdata.o
- HOSTCC scripts/kconfig/expr.o
- HOSTCC scripts/kconfig/symbol.o
- HOSTCC scripts/kconfig/preprocess.o
- HOSTCC scripts/kconfig/zconf.lex.o
- HOSTCC scripts/kconfig/zconf.tab.o
- HOSTLD scripts/kconfig/conf
- scripts/kconfig/conf --syncconfig Kconfig
- HOSTCC arch/x86/tools/relocs_32.o
- HOSTCC arch/x86/tools/relocs_64.o
- HOSTCC arch/x86/tools/relocs_common.o
- HOSTLD arch/x86/tools/relocs
- HOSTCC scripts/genksyms/genksyms.o
- YACC scripts/genksyms/parse.tab.c
- HOSTCC scripts/genksyms/parse.tab.o
- LEX scripts/genksyms/lex.lex.c
- YACC scripts/genksyms/parse.tab.h
- HOSTCC scripts/genksyms/lex.lex.o
- HOSTLD scripts/genksyms/genksyms
- HOSTCC scripts/bin2c
- HOSTCC scripts/kallsyms
- HOSTCC scripts/conmakehash
- HOSTCC scripts/recordmcount
- HOSTCC scripts/sortextable
- HOSTCC scripts/asn1_compiler
- HOSTCC scripts/sign-file
- HOSTCC scripts/extract-cert
- CC scripts/mod/empty.o
- HOSTCC scripts/mod/mk_elfconfig
- MKELF scripts/mod/elfconfig.h
- HOSTCC scripts/mod/modpost.o
- CC scripts/mod/devicetable-offsets.s
- UPD scripts/mod/devicetable-offsets.h
- HOSTCC scripts/mod/file2alias.o
- HOSTCC scripts/mod/sumversion.o
- HOSTLD scripts/mod/modpost
- CC kernel/bounds.s
- CC arch/x86/kernel/asm-offsets.s
- CALL scripts/checksyscalls.sh
-
-Build the test_pmd application
-
-::
-
- cd $RTE_SDK
- ./usertools/dpdk-setup.sh
- Option: 14
- CC config.o
- CC iofwd.o
- CC macfwd.o
- CC macswap.o
- CC flowgen.o
- CC rxonly.o
- CC txonly.o
- CC csumonly.o
- CC icmpecho.o
- CC tm.o
- LD testpmd
- INSTALL-APP testpmd
- INSTALL-MAP testpmd.map
- == Build app/proc_info
- CC main.o
- LD dpdk-procinfo
- INSTALL-APP dpdk-procinfo
- INSTALL-MAP dpdk-procinfo.map
- == Build app/pdump
- CC main.o
- LD dpdk-pdump
- INSTALL-APP dpdk-pdump
- INSTALL-MAP dpdk-pdump.map
- == Build app/test-crypto-perf
- CC main.o
- CC cperf_ops.o
- CC cperf_options_parsing.o
- CC cperf_test_vectors.o
- CC cperf_test_throughput.o
- CC cperf_test_latency.o
- CC cperf_test_pmd_cyclecount.o
- CC cperf_test_verify.o
- CC cperf_test_vector_parsing.o
- CC cperf_test_common.o
- LD dpdk-test-crypto-perf
- INSTALL-APP dpdk-test-crypto-perf
- INSTALL-MAP dpdk-test-crypto-perf.map
- == Build app/test-eventdev
- CC evt_main.o
- CC evt_options.o
- CC evt_test.o
- CC parser.o
- CC test_order_common.o
- CC test_order_queue.o
- CC test_order_atq.o
- CC test_perf_common.o
- CC test_perf_queue.o
- CC test_perf_atq.o
- LD dpdk-test-eventdev
- INSTALL-APP dpdk-test-eventdev
- INSTALL-MAP dpdk-test-eventdev.map
- Build complete [x86_64-native-linuxapp-gcc]
- Installation cannot run with T defined and DESTDIR undefined
- ------------------------------------------------------------------------------
- RTE_TARGET exported as x86_64-native-linuxapp-gcc
- ------------------------------------------------------------------------------
-
- Press enter to continue ...
-
-Check the VF PCI information:
-
-::
-
- root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
- PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
- PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
-
-Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
-
-::
-
- [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
-
- [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
- lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
-
- [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
- [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
-
-
-Get back to pod and run the test_pmd
-
-::
-
- cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
-
- taskset -p --cpu-list 1
- pid 1's current affinity list: 2,3,18,19
-
- ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
- start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
- peer=1,"9e:fd:e6:dd:c1:01"
- EAL: Detected 32 lcore(s)
- EAL: No free hugepages reported in hugepages-2048kB
- EAL: Probing VFIO support...
- EAL: VFIO support initialized
- EAL: PCI device 0000:05:11.0 on NUMA socket 0
- EAL: probe driver: 8086:10ed net_ixgbe_vf
- EAL: using IOMMU type 1 (Type 1)
- EAL: PCI device 0000:05:11.1 on NUMA socket 0
- EAL: probe driver: 8086:10ed net_ixgbe_vf
- Auto-start selected
- Ports to start sending a burst of packets first
- Warning: lsc_interrupt needs to be off when using tx_first. Disabling.
- USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
- Configuring Port 0 (socket 0)
- Port 0: 9E:FD:E6:DD:C1:01
- Configuring Port 1 (socket 0)
- Port 1: 9E:FD:E6:DD:C1:02
- Checking link statuses...
- Port0 Link Up. speed 10000 Mbps- full-duplex
- Port1 Link Up. speed 10000 Mbps- full-duplex
- Done
- No commandline core given, start packet forwarding
- io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
- disabled
- Logical Core 3 (socket 0) forwards packets on 2 streams:
- RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
- RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
-
- io packet forwarding packets/burst=32
- nb forwarding cores=1 - nb forwarding ports=2
- port 0:
- CRC stripping enabled
- RX queues=1 - RX desc=128 - RX free threshold=32
- RX threshold registers: pthresh=8 hthresh=8 wthresh=0
- TX queues=1 - TX desc=512 - TX free threshold=32
- TX threshold registers: pthresh=32 hthresh=0 wthresh=0
- TX RS bit threshold=32 - TXQ flags=0xf01
- port 1:
- CRC stripping enabled
- RX queues=1 - RX desc=128 - RX free threshold=32
- RX threshold registers: pthresh=8 hthresh=8 wthresh=0
- TX queues=1 - TX desc=512 - TX free threshold=32
- TX threshold registers: pthresh=32 hthresh=0 wthresh=0
- TX RS bit threshold=32 - TXQ flags=0xf01
-
- Port statistics ====================================
- ######################## NIC statistics for port 0 ########################
- RX-packets: 56 RX-missed: 0 RX-bytes: 4096
- RX-errors: 0
- RX-nombuf: 0
- TX-packets: 64 TX-errors: 0 TX-bytes: 4096
-
- Throughput (since last show)
- Rx-pps: 0
- Tx-pps: 0
- ############################################################################
-
- ######################## NIC statistics for port 1 ########################
- RX-packets: 432 RX-missed: 0 RX-bytes: 27712
- RX-errors: 0
- RX-nombuf: 0
- TX-packets: 461 TX-errors: 0 TX-bytes: 30080
-
- Throughput (since last show)
- Rx-pps: 0
- Tx-pps: 0
- ############################################################################
-
- Port statistics ====================================
- ######################## NIC statistics for port 0 ########################
- RX-packets: 14124641 RX-missed: 0 RX-bytes: 903977344
- RX-errors: 0
- RX-nombuf: 0
- TX-packets: 14170205 TX-errors: 0 TX-bytes: 906893376
-
- Throughput (since last show)
- Rx-pps: 7068409
- Tx-pps: 7091206
- ############################################################################
-
-
-
-
-
-
References
**********
Code Review / pti / rtp.git / commitdiff