Abstract
--------
-This document describes how to install O-RAN INF image, example configuration for better
-real time performance, and example deployment of Kubernetes cluster and plugins.
+This document describes how to install O-RAN INF image, example configuration (All-in-one Duplex)
+for better real time performance, and example deployment of Kubernetes cluster and plugins.
The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
and container technology.
-Version history
-
-+--------------------+--------------------+--------------------+--------------------+
-| **Date** | **Ver.** | **Author** | **Comment** |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-| 2019-11-02 | 1.0.0 | Jackie Huang | Initail version |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-| 2020-06-03 | 2.0.0 (Bronze) | Xiaohua Zhang | Bronze version |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-| | | | |
-| | | | |
-+--------------------+--------------------+--------------------+--------------------+
-
Preface
-------
Hardware Requirements
---------------------
-Following minimum hardware requirements must be met for installation of O-RAN INF image:
+Following minimum hardware requirements must be met for installation of O-RAN INF image with AIO-DX:
+
++-------------------+----------------------------------------------------------------------------+
+| **HW Aspect** | **Requirement** |
+| | |
++-------------------+----------------------------------------------------------------------------+
+| **# of servers** | 2 |
++-------------------+----------------------------------------------------------------------------+
+| **CPU** | Dual-CPU Intel® Xeon® E5 26xx family (SandyBridge) 8 cores/socket |
+| | or |
+| | Single-CPU Intel® Xeon® D-15xx family, 8 cores (low-power/low-cost option) |
++-------------------+----------------------------------------------------------------------------+
+| **RAM** | 32G |
+| | |
++-------------------+----------------------------------------------------------------------------+
+| **Disk** | * Disk 1: 500G(It's better to be SSD) |
+| | * Disk 2: 1 or more 500 GB for Ceph OSD |
++-------------------+----------------------------------------------------------------------------+
+| **NICs** | * OAM: 1x1GE |
+| | * Data: 1 or more x 10GE (optional) |
++-------------------+----------------------------------------------------------------------------+
+| **BIOS settings** | * Hyper-Threading technology enabled |
+| | * Virtualization technology enabled |
+| | * VT for directed I/O enabled |
+| | * CPU power and performance policy set to performance |
+| | * CPU C state control disabled |
+| | * Plug & play BMC detection disabled |
++-------------------+----------------------------------------------------------------------------+
+
+ORAN INF E Release tested on HP ProLiant DL380p Gen8
+====================================================
+
+1. Installation for the first server from the O-RAN INF ISO image
+-----------------------------------------------------------------
+
+- Please see the README.md file for how to build the image.
+- The Image is a live ISO image with CLI installer:
+ inf-image-aio-installer-intel-corei7-64.iso
-+--------------------+----------------------------------------------------+
-| **HW Aspect** | **Requirement** |
-| | |
-+--------------------+----------------------------------------------------+
-| **# of servers** | 1 |
-+--------------------+----------------------------------------------------+
-| **CPU** | 4 |
-| | |
-+--------------------+----------------------------------------------------+
-| **RAM** | 16G |
-| | |
-+--------------------+----------------------------------------------------+
-| **Disk** | 500G * 2 |
-| | |
-+--------------------+----------------------------------------------------+
-| **NICs** | 2 |
-| | |
-+--------------------+----------------------------------------------------+
+1.1 Burn the image to the USB device
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+- Assume the the usb device is /dev/sdX here
+::
-Software Installation and Deployment
-------------------------------------
+ $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
-1. Installation from the O-RAN INF ISO image
-````````````````````````````````````````````
+1.2 Install the first server (controller-0)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Please see the README.md file for how to build the image.
-- The Image is a live ISO image with CLI installer: inf-image-aio-installer-intel-corei7-64.iso
+- Reboot the target from the USB device.
-1.1 Burn the image to the USB device
-''''''''''''''''''''''''''''''''''''
+- Select “All-in-one Graphics console” or “All-in-one Serial console
+ install” and press ENTER
+
+- Start the auto installation
+
+- It will reboot aotumatically after installation
-- Assume the the usb device is /dev/sdX here
+2. Configuration and initialize the bootstrap
+---------------------------------------------
+
+2.1 First Login with sysadmin/sysadmin and change password
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+2.2 Set OAM network before bootstrap
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
- $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
+ export OAM_DEV=eno3
+ export CONTROLLER0_OAM_CIDR=128.224.210.110/24
+ export DEFAULT_OAM_GATEWAY=128.224.210.1
+ sudo ip address add $CONTROLLER0_OAM_CIDR dev $OAM_DEV
+ sudo ip link set up dev $OAM_DEV
+ sudo ip route add default via $DEFAULT_OAM_GATEWAY dev $OAM_DEV
-1.2 Install the server
-''''''''''''''''''''''
+2.3 Login the server through SSH with sysadmin
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Reboot the target from the USB device.
+2.4 Prepare the localhost.yml for bootstrap
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Select "Graphics console install" or "Serial console install" and press ENTER
+::
-- Remove the USB device and press ENTER to reboot
+ cat << EOF > localhost.yml
+ system_mode: duplex
+ management_subnet: 192.168.18.0/24
+ management_start_address: 192.168.18.2
+ management_end_address: 192.168.18.50
+ management_gateway_address: 192.168.18.1
+ external_oam_subnet: 128.224.210.0/24
+ external_oam_gateway_address: 128.224.210.1
+ external_oam_floating_address: 128.224.210.110
+ external_oam_node_0_address: 128.224.210.111
+ external_oam_node_1_address: 128.224.210.112
+ EOF
+
+2.5 Run the ansible bootstrap
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+::
-2. Configuration and initialize the bootstrap
-`````````````````````````````````````````````
+ ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
-2.1 First Login
-```````````````
-Example:
+After the bootstrap successfully finish, it will show as following:
::
- ifconfig eno1 128.224.180.14/24 up
+ PLAY RECAP *************************************************************************************************************
+ localhost : ok=257 changed=151 unreachable=0 failed=0 skipped=214 rescued=0 ignored=0
- ip route add default via 128.224.180.1
+2.6 Congiure controller-0
+~~~~~~~~~~~~~~~~~~~~~~~~~
- ping 8.8.8.8
+Acquire admin credentials:
-2.3 Login the server through SSH
-````````````````````````````````
+::
-2.4 Prepare the localhost.yml for bootstrap
-```````````````````````````````````````````
+ controller-0:~$ source /etc/platform/openrc
+ [sysadmin@controller-0 ~(keystone_admin)]$
-Example:
+Configure the OAM and MGMT interfaces of controller-0 and specify the
+attached networks:
::
- system_mode: simplex
+ OAM_IF=eno3
+ MGMT_IF=eno1
+ system host-if-modify controller-0 lo -c none
+ IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
+ for UUID in $IFNET_UUIDS; do
+ system interface-network-remove ${UUID}
+ done
- external_oam_subnet: 128.224.180.0/24
+ system host-if-modify controller-0 $OAM_IF -n oam0
+ system host-if-modify controller-0 $MGMT_IF -n pxeboot0
- external_oam_gateway_address: 128.224.180.1
+ system host-if-modify controller-0 oam0 -c platform
+ system interface-network-assign controller-0 oam0 oam
- external_oam_floating_address: 128.224.180.14
+ system host-if-modify controller-0 pxeboot0 -c platform
+ system interface-network-assign controller-0 pxeboot0 pxeboot
- admin_username: admin
+ system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
+ system interface-network-assign controller-0 mgmt0 mgmt
- admin_password: Wind123!
+ system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
+ system interface-network-assign controller-0 cluster0 cluster-host
- ansible_become_pass: Wind123!
+Example output:
- docker_http_proxy: http://xxx.xxx.xxx.xxx:8080
+::
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
+ [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 lo -c none
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | lo |
+ | iftype | virtual |
+ | ports | [] |
+ | imac | 00:00:00:00:00:00 |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | 08c95952-892b-40b5-b17a-7d2ad46e725c |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-17T00:30:45.265032+00:00 |
+ | updated_at | 2021-11-17T01:03:39.031612+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
+ [sysadmin@controller-0 ~(keystone_admin)]$ for UUID in $IFNET_UUIDS; do
+ > system interface-network-remove ${UUID}
+ > done
+ Deleted Interface Network: 0bf11f1b-4fc6-4e97-b896-3d6393a3744e
+ Deleted Interface Network: a62d95f6-ad4e-4779-bfc0-6a885067f8d8
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -n oam0
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | oam0 |
+ | iftype | ethernet |
+ | ports | [u'eno3'] |
+ | imac | 24:6e:96:5d:0c:b2 |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | d8a048fa-67ef-43ac-8166-671be93caa30 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-17T00:28:32.365863+00:00 |
+ | updated_at | 2021-11-17T01:03:45.090904+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $MGMT_IF -n pxeboot0
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | pxeboot0 |
+ | iftype | ethernet |
+ | ports | [u'eno1'] |
+ | imac | 24:6e:96:5d:0c:92 |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-17T00:28:32.612230+00:00 |
+ | updated_at | 2021-11-17T01:03:47.341003+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 oam0 -c platform
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | oam0 |
+ | iftype | ethernet |
+ | ports | [u'eno3'] |
+ | imac | 24:6e:96:5d:0c:b2 |
+ | imtu | 1500 |
+ | ifclass | platform |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | d8a048fa-67ef-43ac-8166-671be93caa30 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-17T00:28:32.365863+00:00 |
+ | updated_at | 2021-11-17T01:03:49.368879+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 oam
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | hostname | controller-0 |
+ | uuid | 3c8bd181-d3f3-4e14-8e89-75a3432db1e4 |
+ | ifname | oam0 |
+ | network_name | oam |
+ +--------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 pxeboot0 -c platform
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | pxeboot0 |
+ | iftype | ethernet |
+ | ports | [u'eno1'] |
+ | imac | 24:6e:96:5d:0c:92 |
+ | imtu | 1500 |
+ | ifclass | platform |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-17T00:28:32.612230+00:00 |
+ | updated_at | 2021-11-17T01:03:53.143795+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 pxeboot0 pxeboot
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | hostname | controller-0 |
+ | uuid | 6c55622d-2da4-4f4e-ab5e-f8e06e03af7c |
+ | ifname | pxeboot0 |
+ | network_name | pxeboot |
+ +--------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | mgmt0 |
+ | iftype | vlan |
+ | ports | [] |
+ | imac | 24:6e:96:5d:0c:92 |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | 18 |
+ | uses | [u'pxeboot0'] |
+ | used_by | [] |
+ | created_at | 2021-11-17T01:03:57.303000+00:00 |
+ | updated_at | None |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 mgmt0 mgmt
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | hostname | controller-0 |
+ | uuid | 2e93ef03-e9ee-457a-8667-05b52b7109a5 |
+ | ifname | mgmt0 |
+ | network_name | mgmt |
+ +--------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | cluster0 |
+ | iftype | vlan |
+ | ports | [] |
+ | imac | 24:6e:96:5d:0c:92 |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vlan_id | 19 |
+ | uses | [u'pxeboot0'] |
+ | used_by | [] |
+ | created_at | 2021-11-17T01:04:02.613518+00:00 |
+ | updated_at | None |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 cluster0 cluster-host
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | hostname | controller-0 |
+ | uuid | fb8b6be6-1618-4662-b063-b1e8d340aa48 |
+ | ifname | cluster0 |
+ | network_name | cluster-host |
+ +--------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-0
+ +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
+ | uuid | name | class | type | vlan id | ports | uses i/f | used by i/f | attributes |
+ +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
+ | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 | mgmt0 | platform | vlan | 18 | [] | [u'pxeboot0'] | [] | MTU=1500 |
+ | 23b5e923-1e53-4e70-a975-542d8380b7f2 | pxeboot0 | platform | ethernet | None | [u'eno1'] | [] | [u'mgmt0', u'cluster0'] | MTU=1500 |
+ | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 | cluster0 | platform | vlan | 19 | [] | [u'pxeboot0'] | [] | MTU=1500 |
+ | d8a048fa-67ef-43ac-8166-671be93caa30 | oam0 | platform | ethernet | None | [u'eno3'] | [] | [] | MTU=1500 |
+ +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
+
+Configure NTP Servers for network time synchronization:
- docker_https_proxy: https://xxx.xxx.xxx.xxx:8080
+::
-2.5 Run the bootstrap
-`````````````````````
+ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
-Use the following command:
+Output
::
- ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
+ [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | uuid | 3206cf01-c64a-457e-ac66-b8224c9684c3 |
+ | ntpservers | 0.pool.ntp.org,1.pool.ntp.org |
+ | isystem_uuid | cc79b616-d24e-4432-a953-85c9b242cb3a |
+ | created_at | 2021-11-17T00:27:23.529571+00:00 |
+ | updated_at | None |
+ +--------------+--------------------------------------+
-After the bootstrap successfully finish, it will show as following:
+Add an OSD on controller-0 for Ceph:
::
- PLAY RECAP ******************************************************************************************************************************************
- localhost : ok=256 changed=135 unreachable=0 failed=0 skipped=220 rescued=0 ignored=0
+ system host-disk-list controller-0
+ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
+ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
+ system host-stor-list controller-0
-2.6 Configure the OAM network
-`````````````````````````````
+Output
+
+::
-Example:
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
+ +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
+ | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path |
+ | | de | num | type | gib | gib | | | |
+ +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
+ | 8e2a719a-fa5a-4c25-89af-70a23fb7b238 | /dev/sda | 2048 | HDD | 893. | 644.726 | Undetermined | 00c66a07604fa8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
+ | | | | | 75 | | | | |
+ | | | | | | | | | |
+ | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/sdb | 2064 | HDD | 1117. | 1117.247 | Undetermined | 00c6b9139b76a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
+ | | | | | 25 | | | | |
+ | | | | | | | | | |
+ | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/sdc | 2080 | HDD | 1117. | 1117.247 | Undetermined | 0053be63c794a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
+ | | | | | 25 | | | | |
+ | | | | | | | | | |
+ | 4879b381-8e9f-48f3-84e2-f9c6a94bbfe0 | /dev/sdd | 2096 | HDD | 1117. | 0.0 | Undetermined | 0065482503bca8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
+ | | | | | 25 | | | | |
+ | | | | | | | | | |
+ +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
+ +------------------+-------------------------------------------------------+
+ | Property | Value |
+ +------------------+-------------------------------------------------------+
+ | osdid | 0 |
+ | function | osd |
+ | state | configuring-on-unlock |
+ | journal_location | 0816c72f-a4f0-49ea-9a95-0f02c880717c |
+ | journal_size_gib | 1024 |
+ | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
+ | journal_node | /dev/sdb2 |
+ | uuid | 0816c72f-a4f0-49ea-9a95-0f02c880717c |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | idisk_uuid | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 |
+ | tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
+ | tier_name | storage |
+ | created_at | 2021-11-17T01:05:04.063823+00:00 |
+ | updated_at | None |
+ +------------------+-------------------------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
+ +------------------+-------------------------------------------------------+
+ | Property | Value |
+ +------------------+-------------------------------------------------------+
+ | osdid | 1 |
+ | function | osd |
+ | state | configuring-on-unlock |
+ | journal_location | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 |
+ | journal_size_gib | 1024 |
+ | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
+ | journal_node | /dev/sdc2 |
+ | uuid | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 |
+ | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | idisk_uuid | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 |
+ | tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
+ | tier_name | storage |
+ | created_at | 2021-11-17T01:05:06.939798+00:00 |
+ | updated_at | None |
+ +------------------+-------------------------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
+ +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
+ +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
+ | 0816c72f-a4f0-49ea-9a95-0f02c880717c | osd | 0 | configuring-on-unlock | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2 | 1 | storage |
+ | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 | osd | 1 | configuring-on-unlock | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
+ +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
+
+2.7 Unlock controller-0
+~~~~~~~~~~~~~~~~~~~~~~~
::
- controller-0:~$ source /etc/platform/openrc
-
- [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno1
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -c platform
-
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | eno1 |
- +-----------------+--------------------------------------+
- | iftype | ethernet |
- +-----------------+--------------------------------------+
- | ports | [u'eno1'] |
- +-----------------+--------------------------------------+
- | imac | 3c:a8:2a:1d:31:d0 |
- +-----------------+--------------------------------------+
- | imtu | 1500 |
- +-----------------+--------------------------------------+
- | ifclass | platform |
- +-----------------+--------------------------------------+
- | aemode | None |
- +-----------------+--------------------------------------+
- | schedpolicy | None |
- +-----------------+--------------------------------------+
- | txhashpolicy | None |
- +-----------------+--------------------------------------+
- | uuid | 95a32d13-01fc-43d2-90ab-801970fb3859 |
- +-----------------+--------------------------------------+
- | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-----------------+--------------------------------------+
- | vlan_id | None |
- +-----------------+--------------------------------------+
- | uses | [] |
- +-----------------+--------------------------------------+
- | used_by | [] |
- +-----------------+--------------------------------------+
- | created_at | 2020-06-03T08:21:17.526303+00:00 |
- +-----------------+--------------------------------------+
- | updated_at | 2020-06-03T10:02:54.682501+00:00 |
- +-----------------+--------------------------------------+
- | sriov_numvfs | 0 |
- +-----------------+--------------------------------------+
- | sriov_vf_driver | None |
- +-----------------+--------------------------------------+
- | accelerated | [False] |
- +-----------------+--------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 $OAM_IF oam
-
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | hostname | controller-0 |
- +--------------+--------------------------------------+
- | uuid | 32592cc3-9d66-4961-a51e-23686da00290 |
- +--------------+--------------------------------------+
- | ifname | eno1 |
- +--------------+--------------------------------------+
- | network_name | oam |
- +--------------+--------------------------------------+
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-list controller-0
-
- +--------------+--------------------------------------+--------+--------------+
- | hostname | uuid | ifname | network_name |
- +--------------+--------------------------------------+--------+--------------+
- | controller-0 | 2478f4c0-a7b4-4656-aa2a-97a0e8891955 | lo | mgmt |
- +--------------+--------------------------------------+--------+--------------+
- | controller-0 | 32592cc3-9d66-4961-a51e-23686da00290 | eno1 | oam |
- +--------------+--------------------------------------+--------+--------------+
- | controller-0 | 43721e4d-4f52-4b4b-bd02-0efa00ef2124 | lo | cluster-host |
- +--------------+--------------------------------------+--------+--------------+
-
-2.7 Configure the NTP server
-````````````````````````````
+ system host-unlock controller-0
-Example:
+Output:
::
-
- [sysadmin@controller-0 ~(keystone_admin)]$
- [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
-
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | uuid | 8fbe5712-6ba0-4e61-b174-ed60f152f616 |
- +--------------+--------------------------------------+
- | ntpservers | 0.pool.ntp.org,1.pool.ntp.org |
- +--------------+--------------------------------------+
- | isystem_uuid | 802ee0be-3728-4df4-b9f7-1533c67fd960 |
- +--------------+--------------------------------------+
- | created_at | 2020-06-03T08:19:47.812249+00:00 |
- +--------------+--------------------------------------+
- | updated_at | None |
- +--------------+--------------------------------------+
-
-2.8 Configure the Storage by using the second hard drive as ceph backend
-````````````````````````````````````````````````````````````````````````
-Example:
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
+ +-----------------------+-------------------------------------------------+
+ | Property | Value |
+ +-----------------------+-------------------------------------------------+
+ | action | none |
+ | administrative | locked |
+ | availability | online |
+ | bm_ip | None |
+ | bm_type | none |
+ | bm_username | None |
+ | boot_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
+ | capabilities | {u'stor_function': u'monitor'} |
+ | clock_synchronization | ntp |
+ | config_applied | 6aa15fb4-8cb3-494e-b94e-95f85b560f22 |
+ | config_status | None |
+ | config_target | c6ae9b2d-a3c4-4751-a79e-5487ba81ed82 |
+ | console | ttyS0,115200 |
+ | created_at | 2021-11-17T00:28:01.983673+00:00 |
+ | hostname | controller-0 |
+ | id | 1 |
+ | install_output | graphical |
+ | install_state | None |
+ | install_state_info | None |
+ | inv_state | inventoried |
+ | invprovision | provisioning |
+ | location | {} |
+ | mgmt_ip | 192.168.18.3 |
+ | mgmt_mac | 24:6e:96:5d:0c:92 |
+ | operational | disabled |
+ | personality | controller |
+ | reserved | False |
+ | rootfs_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
+ | serialid | None |
+ | software_load | 21.05 |
+ | subfunction_avail | online |
+ | subfunction_oper | disabled |
+ | subfunctions | controller,worker,lowlatency |
+ | task | Unlocking |
+ | tboot | false |
+ | ttys_dcd | None |
+ | updated_at | 2021-11-17T01:05:07.015414+00:00 |
+ | uptime | 3496 |
+ | uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
+ | vim_progress_status | None |
+ +-----------------------+-------------------------------------------------+
+
+Controller-0 will reboot to apply configuration changes and come into
+service. This can take 5-10 minutes, depending on the performance of the
+host machine.
+
+Once the controller comes back up, check the status of controller-0
::
- [sysadmin@controller-0 ~(keystone_admin)]$
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
-
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
- | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_ | device_path |
- | | de | num | type | gib | gib | | id | |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
- | 3884f906-cb9b-4052-bcb8-8be70e599b9f | /dev/sda | 2048 | HDD | 1117. | 868.759 | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
- | | | | | 782 | | | RH8W4QQ | 03:00.0-scsi-0:1:0:0 |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
- | | | | | | | | | |
- | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/sdb | 2064 | HDD | 1117. | 0.0 | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
- | | | | | 782 | | | RH8W4QQ | 03:00.0-scsi-0:1:0:1 |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
- | | | | | | | | | |
- +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
-
- +------------------+-------------------------------------------------------+
- | Property | Value |
- +------------------+-------------------------------------------------------+
- | osdid | 0 |
- +------------------+-------------------------------------------------------+
- | function | osd |
- +------------------+-------------------------------------------------------+
- | state | configuring-on-unlock |
- +------------------+-------------------------------------------------------+
- | journal_location | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf |
- +------------------+-------------------------------------------------------+
- | journal_size_gib | 1024 |
- +------------------+-------------------------------------------------------+
- | journal_path | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 |
- +------------------+-------------------------------------------------------+
- | journal_node | /dev/sdb2 |
- +------------------+-------------------------------------------------------+
- | uuid | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf |
- +------------------+-------------------------------------------------------+
- | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +------------------+-------------------------------------------------------+
- | idisk_uuid | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 |
- +------------------+-------------------------------------------------------+
- | tier_uuid | db6eabf9-9922-4d8e-9740-c0f9878e692d |
- +------------------+-------------------------------------------------------+
- | tier_name | storage |
- +------------------+-------------------------------------------------------+
- | created_at | 2020-06-03T10:03:55.645744+00:00 |
- +------------------+-------------------------------------------------------+
- | updated_at | None |
- +------------------+-------------------------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
-
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
- | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
- | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf | osd | 0 | configuring-on-unlock | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 | /dev/sdb2 | 1 | storage |
- +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
-
-2.9 Configure the SR-IOV enabled network
-````````````````````````````````````````
-It need :
- - assign the label of SR-IOV
- - change the CPU policy to static
- - change the hugepage setting
- - create the data network
- - assign the SR-IOV interface to data network
-
-Example:
+ controller-0:~$ source /etc/platform/openrc
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | id | hostname | personality | administrative | operational | availability |
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | 1 | controller-0 | controller | unlocked | enabled | available |
+ +----+--------------+-------------+----------------+-------------+--------------+
+
+2. Installation for the second server (controller-1)
+----------------------------------------------------
+
+2.1 Power on the controller-1 server and force it to network boot
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+2.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+2.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
- [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
-
- [sysadmin@controller-0 ~(keystone_admin)]$
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER sriovdp=enabled
-
- +-------------+--------------------------------------+
- | Property | Value |
- +-------------+--------------------------------------+
- | uuid | 7101010d-1897-4cc8-8c87-7dffaebf848b |
- +-------------+--------------------------------------+
- | host_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-------------+--------------------------------------+
- | label_key | sriovdp |
- +-------------+--------------------------------------+
- | label_value | enabled |
- +-------------+--------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER kube-cpu-mgr-policy=static
-
- +-------------+--------------------------------------+
- | Property | Value |
- +-------------+--------------------------------------+
- | uuid | 4936f32b-6939-4115-b643-846bd50e7738 |
- +-------------+--------------------------------------+
- | host_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-------------+--------------------------------------+
- | label_key | kube-cpu-mgr-policy |
- +-------------+--------------------------------------+
- | label_value | static |
- +-------------+--------------------------------------+
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7 0
-
- +-------------------------------------+--------------------------------------+
- | Property | Value |
- +-------------------------------------+--------------------------------------+
- | Memory: Usable Total (MiB) | 0 |
- +-------------------------------------+--------------------------------------+
- | Platform (MiB) | 7000 |
- +-------------------------------------+--------------------------------------+
- | Available (MiB) | 0 |
- +-------------------------------------+--------------------------------------+
- | Huge Pages Configured | True |
- +-------------------------------------+--------------------------------------+
- | vSwitch Huge Pages: Size (MiB) | 1024 |
- +-------------------------------------+--------------------------------------+
- | Total | 0 |
- +-------------------------------------+--------------------------------------+
- | Available | 0 |
- +-------------------------------------+--------------------------------------+
- | Required | None |
- +-------------------------------------+--------------------------------------+
- | Application Pages (4K): Total | None |
- +-------------------------------------+--------------------------------------+
- | Application Huge Pages (2M): Total | 0 |
- +-------------------------------------+--------------------------------------+
- | Available | 0 |
- +-------------------------------------+--------------------------------------+
- | Application Huge Pages (1G): Total | 0 |
- +-------------------------------------+--------------------------------------+
- | Total Pending | 7 |
- +-------------------------------------+--------------------------------------+
- | Available | 0 |
- +-------------------------------------+--------------------------------------+
- | uuid | 15702f44-2281-41f3-8c83-a1e9a300ad01 |
- +-------------------------------------+--------------------------------------+
- | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-------------------------------------+--------------------------------------+
- | inode_uuid | d52648bc-dfa6-4214-afa3-5d2148ee4180 |
- +-------------------------------------+--------------------------------------+
- | created_at | 2020-06-03T08:21:27.468357+00:00 |
- +-------------------------------------+--------------------------------------+
- | updated_at | 2020-06-03T10:10:49.159080+00:00 |
- +-------------------------------------+--------------------------------------+
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7 1
-
- +-------------------------------------+--------------------------------------+
- | Property | Value |
- +-------------------------------------+--------------------------------------+
- | Memory: Usable Total (MiB) | 0 |
- +-------------------------------------+--------------------------------------+
- | Platform (MiB) | 1000 |
- +-------------------------------------+--------------------------------------+
- | Available (MiB) | 0 |
- +-------------------------------------+--------------------------------------+
- | Huge Pages Configured | True |
- +-------------------------------------+--------------------------------------+
- | vSwitch Huge Pages: Size (MiB) | 1024 |
- +-------------------------------------+--------------------------------------+
- | Total | 0 |
- +-------------------------------------+--------------------------------------+
- | Available | 0 |
- +-------------------------------------+--------------------------------------+
- | Required | None |
- +-------------------------------------+--------------------------------------+
- | Application Pages (4K): Total | None |
- +-------------------------------------+--------------------------------------+
- | Application Huge Pages (2M): Total | 0 |
- +-------------------------------------+--------------------------------------+
- | Available | 0 |
- +-------------------------------------+--------------------------------------+
- | Application Huge Pages (1G): Total | 0 |
- +-------------------------------------+--------------------------------------+
- | Total Pending | 7 |
- +-------------------------------------+--------------------------------------+
- | Available | 0 |
- +-------------------------------------+--------------------------------------+
- | uuid | 544580b9-0f2a-41a1-b4e3-90fdf5c8f7e0 |
- +-------------------------------------+--------------------------------------+
- | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-------------------------------------+--------------------------------------+
- | inode_uuid | e02c3c41-03b7-4f49-8244-bca851fa2d2f |
- +-------------------------------------+--------------------------------------+
- | created_at | 2020-06-03T08:21:27.566279+00:00 |
- +-------------------------------------+--------------------------------------+
- | updated_at | 2020-06-03T10:10:49.220907+00:00 |
- +-------------------------------------+--------------------------------------+
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET0='physnet0'
-
- [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET1='physnet1'
-
- [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
-
- [sysadmin@controller-0 ~(keystone_admin)]$
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET0} flat
-
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | id | 1 |
- +--------------+--------------------------------------+
- | uuid | 0ffe0b0a-35ff-45b2-9a62-63a494d2d18b |
- +--------------+--------------------------------------+
- | name | physnet0 |
- +--------------+--------------------------------------+
- | network_type | flat |
- +--------------+--------------------------------------+
- | mtu | 1500 |
- +--------------+--------------------------------------+
- | description | None |
- +--------------+--------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET1} flat
-
- +--------------+--------------------------------------+
- | Property | Value |
- +--------------+--------------------------------------+
- | id | 2 |
- +--------------+--------------------------------------+
- | uuid | 899ca8ba-6800-4d62-aed4-3770878da738 |
- +--------------+--------------------------------------+
- | name | physnet1 |
- +--------------+--------------------------------------+
- | network_type | flat |
- +--------------+--------------------------------------+
- | mtu | 1500 |
- +--------------+--------------------------------------+
- | description | None |
- +--------------+--------------------------------------+
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-port-list controller-0
-
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | uuid | name | type | pci address | device | processor | accelerated | device type |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | 7007dfb1-ea55-4355-9d27-3ed4c6696283 | eno1 | ethernet | 0000:02:00.0 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | 51f62ecf-7897-43e3-beb8-46ae5550ef05 | eno2 | ethernet | 0000:02:00.1 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | e987725e-ab45-4151-b53d-94ba21706eb4 | eno3 | ethernet | 0000:02:00.2 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | f4a8115a-ca0b-4336-a634-2cad77c5a88c | eno4 | ethernet | 0000:02:00.3 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | 88965882-c816-42e8-ba0e-d3ff6f9f553e | eno49 | ethernet | 0000:04:00.0 | 0 | 0 | False | OneConnect NIC (Skyhawk) [0720] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | 98e2cedc-9ef8-4ca0-b3e0-db3a2bd0273a | eno50 | ethernet | 0000:04:00.1 | 0 | 0 | False | OneConnect NIC (Skyhawk) [0720] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | e75d7aa6-7832-4ec4-9dd4-ea8300773fd8 | ens2f0 | ethernet | 0000:05:00.0 | 0 | 0 | True | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
- | 8ac4ffca-4a61-4b47-8e17-171f2e6cea66 | ens2f1 | ethernet | 0000:05:00.1 | 0 | 0 | True | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
- +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f0
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov0 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
-
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | sriov0 |
- +-----------------+--------------------------------------+
- | iftype | ethernet |
- +-----------------+--------------------------------------+
- | ports | [u'ens2f0'] |
- +-----------------+--------------------------------------+
- | imac | 00:1b:21:b3:46:2c |
- +-----------------+--------------------------------------+
- | imtu | 1500 |
- +-----------------+--------------------------------------+
- | ifclass | pci-sriov |
- +-----------------+--------------------------------------+
- | aemode | None |
- +-----------------+--------------------------------------+
- | schedpolicy | None |
- +-----------------+--------------------------------------+
- | txhashpolicy | None |
- +-----------------+--------------------------------------+
- | uuid | 9450b1ae-d1a1-4ecb-811a-fa3cb1cdf362 |
- +-----------------+--------------------------------------+
- | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-----------------+--------------------------------------+
- | vlan_id | None |
- +-----------------+--------------------------------------+
- | uses | [] |
- +-----------------+--------------------------------------+
- | used_by | [] |
- +-----------------+--------------------------------------+
- | created_at | 2020-06-03T12:34:11.633254+00:00 |
- +-----------------+--------------------------------------+
- | updated_at | 2020-06-04T02:01:28.168127+00:00 |
- +-----------------+--------------------------------------+
- | sriov_numvfs | 6 |
- +-----------------+--------------------------------------+
- | sriov_vf_driver | vfio |
- +-----------------+--------------------------------------+
- | accelerated | [True] |
- +-----------------+--------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov0 ${PHYSNET0}
-
- +------------------+--------------------------------------+
- | Property | Value |
- +------------------+--------------------------------------+
- | hostname | controller-0 |
- +------------------+--------------------------------------+
- | uuid | 1d35219d-3258-42e7-8f0e-41d865710d58 |
- +------------------+--------------------------------------+
- | ifname | sriov0 |
- +------------------+--------------------------------------+
- | datanetwork_name | physnet0 |
- +------------------+--------------------------------------+
-
-
-
- [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f1
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov1 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
-
- +-----------------+--------------------------------------+
- | Property | Value |
- +-----------------+--------------------------------------+
- | ifname | sriov1 |
- +-----------------+--------------------------------------+
- | iftype | ethernet |
- +-----------------+--------------------------------------+
- | ports | [u'ens2f1'] |
- +-----------------+--------------------------------------+
- | imac | 00:1b:21:b3:46:2d |
- +-----------------+--------------------------------------+
- | imtu | 1500 |
- +-----------------+--------------------------------------+
- | ifclass | pci-sriov |
- +-----------------+--------------------------------------+
- | aemode | None |
- +-----------------+--------------------------------------+
- | schedpolicy | None |
- +-----------------+--------------------------------------+
- | txhashpolicy | None |
- +-----------------+--------------------------------------+
- | uuid | 13d5ee99-0399-4b6b-a359-0f96660afdc4 |
- | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
- | vlan_id | None |
- | uses | [] |
- | used_by | [] |
- | created_at | 2020-06-03T12:34:12.072624+00:00 |
- | updated_at | 2020-06-04T02:03:51.405322+00:00 |
- | sriov_numvfs | 6 |
- | sriov_vf_driver | vfio |
- | accelerated | [True] |
- +-----------------+--------------------------------------+
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov1 ${PHYSNET1}
-
- +------------------+--------------------------------------+
- | Property | Value |
- +------------------+--------------------------------------+
- | hostname | controller-0 |
- +------------------+--------------------------------------+
- | uuid | a87c6a5f-0c53-4ebf-9a32-38366700426f |
- +------------------+--------------------------------------+
- | ifname | sriov1 |
- +------------------+--------------------------------------+
- | datanetwork_name | physnet1 |
- +------------------+--------------------------------------+
-
-2.10 Unlock the server
-``````````````````````
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | id | hostname | personality | administrative | operational | availability |
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | 1 | controller-0 | controller | unlocked | enabled | degraded |
+ | 2 | None | None | locked | disabled | offline |
+ +----+--------------+-------------+----------------+-------------+--------------+
-Example:
+2.4 Using the host id, set the personality of this host to 'controller’:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
::
-
- [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
-
- +-----------------------+-------------------------------------------------+
- | Property | Value |
- +-----------------------+-------------------------------------------------+
- | action | none |
- +-----------------------+-------------------------------------------------+
- | administrative | locked |
- +-----------------------+-------------------------------------------------+
- | availability | online |
- +-----------------------+-------------------------------------------------+
- | bm_ip | None |
- +-----------------------+-------------------------------------------------+
- | bm_type | none |
- +-----------------------+-------------------------------------------------+
- | bm_username | None |
- +-----------------------+-------------------------------------------------+
- | boot_device | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
- +-----------------------+-------------------------------------------------+
- | capabilities | {u'stor_function': u'monitor'} |
- +-----------------------+-------------------------------------------------+
- | clock_synchronization | ntp |
- +-----------------------+-------------------------------------------------+
- | config_applied | 3daf20c7-c032-4aa2-839d-93da92051bea |
- +-----------------------+-------------------------------------------------+
- | config_status | Config out-of-date |
- +-----------------------+-------------------------------------------------+
- | config_target | bdaf20c7-c032-4aa2-839d-93da92051bea |
- +-----------------------+-------------------------------------------------+
- | console | tty0 |
- +-----------------------+-------------------------------------------------+
- | created_at | 2020-06-03T08:20:59.372602+00:00 |
- +-----------------------+-------------------------------------------------+
- | hostname | controller-0 |
- +-----------------------+-------------------------------------------------+
- | id | 1 |
- +-----------------------+-------------------------------------------------+
- | install_output | graphical |
- +-----------------------+-------------------------------------------------+
- | install_state | None |
- +-----------------------+-------------------------------------------------+
- | install_state_info | None |
- +-----------------------+-------------------------------------------------+
- | inv_state | inventoried |
- +-----------------------+-------------------------------------------------+
- | invprovision | provisioning |
- +-----------------------+-------------------------------------------------+
- | location | {} |
- +-----------------------+-------------------------------------------------+
- | mgmt_ip | 192.168.204.2 |
- +-----------------------+-------------------------------------------------+
- | mgmt_mac | 00:00:00:00:00:00 |
- +-----------------------+-------------------------------------------------+
- | operational | disabled |
- +-----------------------+-------------------------------------------------+
- | personality | controller |
- +-----------------------+-------------------------------------------------+
- | reserved | False |
- +-----------------------+-------------------------------------------------+
- | rootfs_device | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
- +-----------------------+-------------------------------------------------+
- | serialid | None |
- +-----------------------+-------------------------------------------------+
- | software_load | 19.12 |
- +-----------------------+-------------------------------------------------+
- | subfunction_avail | online |
- +-----------------------+-------------------------------------------------+
- | subfunction_oper | disabled |
- +-----------------------+-------------------------------------------------+
- | subfunctions | controller,worker |
- +-----------------------+-------------------------------------------------+
- | task | Unlocking |
- +-----------------------+-------------------------------------------------+
- | tboot | false |
- +-----------------------+-------------------------------------------------+
- | ttys_dcd | None |
- +-----------------------+-------------------------------------------------+
- | updated_at | 2020-06-04T02:04:59.514854+00:00 |
- +-----------------------+-------------------------------------------------+
- | uptime | 49475 |
- +-----------------------+-------------------------------------------------+
- | uuid | 08c23843-afa4-4b80-a285-be126365276e |
- +-----------------------+-------------------------------------------------+
- | vim_progress_status | None |
- +-----------------------+-------------------------------------------------+
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller
+ +-----------------------+--------------------------------------+
+ | Property | Value |
+ +-----------------------+--------------------------------------+
+ | action | none |
+ | administrative | locked |
+ | availability | offline |
+ | bm_ip | None |
+ | bm_type | None |
+ | bm_username | None |
+ | boot_device | /dev/sda |
+ | capabilities | {} |
+ | clock_synchronization | ntp |
+ | config_applied | None |
+ | config_status | None |
+ | config_target | None |
+ | console | ttyS0,115200 |
+ | created_at | 2021-11-17T10:17:44.387813+00:00 |
+ | hostname | controller-1 |
+ | id | 2 |
+ | install_output | text |
+ | install_state | None |
+ | install_state_info | None |
+ | inv_state | None |
+ | invprovision | None |
+ | location | {} |
+ | mgmt_ip | 192.168.18.4 |
+ | mgmt_mac | 24:6e:96:5d:38:ee |
+ | operational | disabled |
+ | personality | controller |
+ | reserved | False |
+ | rootfs_device | /dev/sda |
+ | serialid | None |
+ | software_load | 21.05 |
+ | subfunction_avail | not-installed |
+ | subfunction_oper | disabled |
+ | subfunctions | controller,worker,lowlatency |
+ | task | None |
+ | tboot | false |
+ | ttys_dcd | None |
+ | updated_at | None |
+ | uptime | 0 |
+ | uuid | f069381d-9743-49cc-bf8b-eb4bd3972203 |
+ | vim_progress_status | None |
+ +-----------------------+--------------------------------------+
+
+2.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This can take 5-10 minutes, depending on the performance of the host
+machine.
+
+::
+
+ [root@controller-0 hieradata(keystone_admin)]$ system host-list
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | id | hostname | personality | administrative | operational | availability |
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | 1 | controller-0 | controller | unlocked | enabled | available |
+ | 2 | controller-1 | controller | locked | disabled | online |
+ +----+--------------+-------------+----------------+-------------+--------------+
+
+2.6 Configure controller-1
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ OAM_IF=eno3
+ MGMT_IF=eno1
+ system host-if-modify controller-1 $OAM_IF -n oam0
+ system host-if-modify controller-1 oam0 -c platform
+ system interface-network-assign controller-1 oam0 oam
+
+ system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
+ system interface-network-assign controller-1 cluster0 cluster-host
+
+ system host-if-list controller-1
+
+ system host-disk-list controller-1
+ system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
+ system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
+ system host-stor-list controller-1
+
+Output:
+
+::
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
+ [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 $OAM_IF -n oam0
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | oam0 |
+ | iftype | ethernet |
+ | ports | [u'eno3'] |
+ | imac | 24:6e:96:5d:39:0e |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | c2473511-d0d6-445d-9739-4d43dc029de9 |
+ | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-22T14:01:32.365863+00:00 |
+ | updated_at | 2021-11-22T15:04:45.090904+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 oam0 -c platform
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | oam0 |
+ | iftype | ethernet |
+ | ports | [u'eno3'] |
+ | imac | 24:6e:96:5d:39:0e |
+ | imtu | 1500 |
+ | ifclass | platform |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | c2473511-d0d6-445d-9739-4d43dc029de9 |
+ | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | vlan_id | None |
+ | uses | [] |
+ | used_by | [] |
+ | created_at | 2021-11-22T14:05:16.052229+00:00 |
+ | updated_at | 2021-11-22T15:08:35.324634+00:00 |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 oam0 oam
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | hostname | controller-1 |
+ | uuid | f2e7f088-0dd0-4adc-8348-4e3cef23bc47 |
+ | ifname | oam0 |
+ | network_name | oam |
+ +--------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
+ +-----------------+--------------------------------------+
+ | Property | Value |
+ +-----------------+--------------------------------------+
+ | ifname | cluster0 |
+ | iftype | vlan |
+ | ports | [] |
+ | imac | 24:6e:96:5d:38:ee |
+ | imtu | 1500 |
+ | ifclass | None |
+ | aemode | None |
+ | schedpolicy | None |
+ | txhashpolicy | None |
+ | uuid | b6783682-b2aa-4135-90d2-676e1db41ae8 |
+ | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | vlan_id | 19 |
+ | uses | [u'pxeboot0'] |
+ | used_by | [] |
+ | created_at | 2021-11-22T15:08:43.932209+00:00 |
+ | updated_at | None |
+ | sriov_numvfs | 0 |
+ | sriov_vf_driver | None |
+ | accelerated | [True] |
+ +-----------------+--------------------------------------+
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 cluster0 cluster-host
+ +--------------+--------------------------------------+
+ | Property | Value |
+ +--------------+--------------------------------------+
+ | hostname | controller-1 |
+ | uuid | 8fc64805-b54b-45a4-b88a-e13b236abfe8 |
+ | ifname | cluster0 |
+ | network_name | cluster-host |
+ +--------------+--------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-1
+ +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
+ | uuid | name | class | type | vlan id | ports | uses i/f | used by i/f | attributes |
+ +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
+ | b6783682-b2aa-4135-90d2-676e1db41ae8 | cluster0 | platform | vlan | 19 | [] | [u'pxeboot0'] | [] | MTU=1500 |
+ | b8921960-fde5-44c3-960d-2aebf42ea400 | pxeboot0 | platform | ethernet | None | [u'eno1'] | [] | [u'mgmt0', u'cluster0'] | MTU=1500 |
+ | c103275b-2b75-4568-865f-ac6be32ecb2d | mgmt0 | platform | vlan | 18 | [] | [u'pxeboot0'] | [] | MTU=1500 |
+ | c2473511-d0d6-445d-9739-4d43dc029de9 | oam0 | platform | ethernet | None | [u'eno3'] | [] | [] | MTU=1500 |
+ +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
+ +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
+ | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path |
+ | | de | num | type | gib | gib | | | |
+ +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
+ | 5b8fade4-b048-48fa-b906-9dcbdbed8e96 | /dev/sda | 2048 | HDD | 893. | 644.726 | Undetermined | 00cbd97f3e36ccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
+ | | | | | 75 | | | | |
+ | | | | | | | | | |
+ | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/sdb | 2064 | HDD | 1117. | 1117.247 | Undetermined | 006d0e977b5fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
+ | | | | | 25 | | | | |
+ | | | | | | | | | |
+ | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/sdc | 2080 | HDD | 1117. | 1117.247 | Undetermined | 003a2377ac7fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
+ | | | | | 25 | | | | |
+ | | | | | | | | | |
+ | 774c3cd0-1178-4145-9573-f0d6dee2ba06 | /dev/sdd | 2096 | HDD | 1117. | 1117.247 | Undetermined | 00d7093ef0adccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
+ | | | | | 25 | | | | |
+ | | | | | | | | | |
+ | 00361302-8d55-4730-855c-b0098c73ab7e | /dev/sde | 2112 | SSD | 223. | 223.568 | N/A | PHDW730104QM240E | /dev/disk/by-path/pci-0000:d8:00.0-ata-1 |
+ | | | | | 57 | | | | |
+ | | | | | | | | | |
+ | 7ce735e6-920f-4424-a890-a7a7f48d7632 | /dev/sdf | 2128 | SSD | 223. | 223.568 | N/A | PHDW730104LL240E | /dev/disk/by-path/pci-0000:d8:00.0-ata-2 |
+ | | | | | 57 | | | | |
+ | | | | | | | | | |
+ +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
+ +------------------+-------------------------------------------------------+
+ | Property | Value |
+ +------------------+-------------------------------------------------------+
+ | osdid | 2 |
+ | function | osd |
+ | state | configuring-on-unlock |
+ | journal_location | 54a218d8-0466-4366-9ef0-3ec5a952fde7 |
+ | journal_size_gib | 1024 |
+ | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
+ | journal_node | /dev/sdb2 |
+ | uuid | 54a218d8-0466-4366-9ef0-3ec5a952fde7 |
+ | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | idisk_uuid | 1a3f0a36-5961-42e5-a271-e71db1c25d42 |
+ | tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
+ | tier_name | storage |
+ | created_at | 2021-11-22T15:11:55.641193+00:00 |
+ | updated_at | None |
+ +------------------+-------------------------------------------------------+
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
+ +------------------+-------------------------------------------------------+
+ | Property | Value |
+ +------------------+-------------------------------------------------------+
+ | osdid | 3 |
+ | function | osd |
+ | state | configuring-on-unlock |
+ | journal_location | 5be88c7a-3a94-4b97-9da5-b247bb89406c |
+ | journal_size_gib | 1024 |
+ | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
+ | journal_node | /dev/sdc2 |
+ | uuid | 5be88c7a-3a94-4b97-9da5-b247bb89406c |
+ | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | idisk_uuid | eddd732f-2cea-49b3-86db-b722c0b1a1ae |
+ | tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
+ | tier_name | storage |
+ | created_at | 2021-11-22T15:12:04.274839+00:00 |
+ | updated_at | None |
+ +------------------+-------------------------------------------------------+
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-1
+ +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
+ | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
+ +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
+ | 54a218d8-0466-4366-9ef0-3ec5a952fde7 | osd | 2 | configuring-on-unlock | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2 | 1 | storage |
+ | 5be88c7a-3a94-4b97-9da5-b247bb89406c | osd | 3 | configuring-on-unlock | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
+ +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
+
+2.7 Unlock controller-1
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Unlock controller-1 in order to bring it into service:
+
+::
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-1
+ +-----------------------+--------------------------------------+
+ | Property | Value |
+ +-----------------------+--------------------------------------+
+ | action | none |
+ | administrative | locked |
+ | availability | online |
+ | bm_ip | None |
+ | bm_type | None |
+ | bm_username | None |
+ | boot_device | /dev/sda |
+ | capabilities | {u'stor_function': u'monitor'} |
+ | clock_synchronization | ntp |
+ | config_applied | None |
+ | config_status | Config out-of-date |
+ | config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
+ | console | ttyS0,115200 |
+ | created_at | 2021-11-22T12:58:11.630526+00:00 |
+ | hostname | controller-1 |
+ | id | 3 |
+ | install_output | text |
+ | install_state | None |
+ | install_state_info | None |
+ | inv_state | inventoried |
+ | invprovision | unprovisioned |
+ | location | {} |
+ | mgmt_ip | 192.168.18.4 |
+ | mgmt_mac | 24:6e:96:5d:38:ee |
+ | operational | disabled |
+ | personality | controller |
+ | reserved | False |
+ | rootfs_device | /dev/sda |
+ | serialid | None |
+ | software_load | 21.05 |
+ | subfunction_avail | online |
+ | subfunction_oper | disabled |
+ | subfunctions | controller,worker,lowlatency |
+ | task | Unlocking |
+ | tboot | false |
+ | ttys_dcd | None |
+ | updated_at | 2021-11-22T15:13:09.716324+00:00 |
+ | uptime | 752 |
+ | uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | vim_progress_status | None |
+ +-----------------------+--------------------------------------+
+
+Controller-1 will reboot in order to apply configuration changes and
+come into service. This can take 5-10 minutes, depending on the
+performance of the host machine.
+
+::
+
+ [root@controller-0 hieradata(keystone_admin)]$ system host-list
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | id | hostname | personality | administrative | operational | availability |
+ +----+--------------+-------------+----------------+-------------+--------------+
+ | 1 | controller-0 | controller | unlocked | enabled | available |
+ | 2 | controller-1 | controller | unlocked | enabled | available |
+ +----+--------------+-------------+----------------+-------------+--------------+
+
+ [sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-1
+ +-----------------------+-----------------------------------------------------------------------+
+ | Property | Value |
+ +-----------------------+-----------------------------------------------------------------------+
+ | action | none |
+ | administrative | unlocked |
+ | availability | available |
+ | bm_ip | None |
+ | bm_type | None |
+ | bm_username | None |
+ | boot_device | /dev/sda |
+ | capabilities | {u'stor_function': u'monitor', u'Personality': u'Controller-Standby'} |
+ | clock_synchronization | ntp |
+ | config_applied | 9747e0ce-2319-409d-b75c-2475bc5065ac |
+ | config_status | None |
+ | config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
+ | console | ttyS0,115200 |
+ | created_at | 2021-11-22T12:58:11.630526+00:00 |
+ | hostname | controller-1 |
+ | id | 2 |
+ | install_output | text |
+ | install_state | None |
+ | install_state_info | None |
+ | inv_state | inventoried |
+ | invprovision | provisioned |
+ | location | {} |
+ | mgmt_ip | 192.168.18.4 |
+ | mgmt_mac | 24:6e:96:5d:38:ee |
+ | operational | enabled |
+ | personality | controller |
+ | reserved | False |
+ | rootfs_device | /dev/sda |
+ | serialid | None |
+ | software_load | 21.05 |
+ | subfunction_avail | available |
+ | subfunction_oper | enabled |
+ | subfunctions | controller,worker,lowlatency |
+ | task | |
+ | tboot | false |
+ | ttys_dcd | None |
+ | updated_at | 2021-11-22T23:59:07.787759+00:00 |
+ | uptime | 31008 |
+ | uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
+ | vim_progress_status | services-enabled |
+ +-----------------------+-----------------------------------------------------------------------+
+
3. Simple use case for sriov
````````````````````````````
3.1 After controller-0 is rebooted and up running, download the DPDK
-````````````````````````````````````````````````````````````````````
+''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
::
[sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
[sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
Password:
- --2020-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
+ --2021-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
[========================================================================================>] 9.78M
1.48MB/s in 6.8s
- 2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
+ 2021-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
sudo tar xvf dpdk-17.11.10.tar.xz
sudo ln -s dpdk-stable-17.11.10 dpdk-stable
-3.1 Prepare the yaml file for the network assignment container
-``````````````````````````````````````````````````````````````
+3.2 Prepare the yaml file for the network assignment container
+''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
The following the exmaple of the yaml file:
> }'
> EOF
-3.2 Run the network assignent container for the 2 VFs
-`````````````````````````````````````````````````````
+3.3 Run the network assignent container for the 2 VFs
+'''''''''''''''''''''''''''''''''''''''''''''''''''''
::
networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
-3.3 Prepare the VF container yaml file
-``````````````````````````````````````
+3.4 Prepare the VF container yaml file
+''''''''''''''''''''''''''''''''''''''
::
> path: /usr/src/
> EOF
-3.4 Run the VF container
-````````````````````````
+3.5 Run the VF container
+''''''''''''''''''''''''
Start the VF container.
Press enter to continue ...
Check the VF PCI information:
+
::
root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
+
::
[root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
- `StarlingX`_
.. _`StarlingX`: https://docs.starlingx.io/
-
\ No newline at end of file
+