docs/installation-guide.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3 .. Copyright (C) 2019 Wind River Systems, Inc.
   4
   5
   6 Installation Guide
   7 ==================
   8
   9 .. contents::
  10    :depth: 3
  11    :local:
  12
  13 Abstract
  14 --------
  15
  16 This document describes how to install O-RAN INF image, example configuration for better
  17 real time performance, and example deployment of Kubernetes cluster and plugins.
  18
  19 The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
  20 and container technology.
  21
  22 Version history
  23
  24 +--------------------+--------------------+--------------------+--------------------+
  25 | **Date**           | **Ver.**           | **Author**         | **Comment**        |
  26 |                    |                    |                    |                    |
  27 +--------------------+--------------------+--------------------+--------------------+
  28 | 2019-11-02         | 1.0.0              | Jackie Huang       | Initail version    |
  29 |                    |                    |                    |                    |
  30 +--------------------+--------------------+--------------------+--------------------+
  31 | 2020-06-03         | 2.0.0 (Bronze)     | Xiaohua Zhang      | Bronze version     |
  32 |                    |                    |                    |                    |
  33 +--------------------+--------------------+--------------------+--------------------+
  34 | 2020-11-23         | 3.0.0 (Cherry)     | Xiaohua Zhang      | Cherry version     |
  35 |                    |                    |                    |                    |
  36 +--------------------+--------------------+--------------------+--------------------+
  37
  38
  39 Preface
  40 -------
  41
  42 Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
  43
  44
  45 Hardware Requirements
  46 ---------------------
  47
  48 Following minimum hardware requirements must be met for installation of O-RAN INF image:
  49
  50 +--------------------+----------------------------------------------------+
  51 | **HW Aspect**      | **Requirement**                                    |
  52 |                    |                                                    |
  53 +--------------------+----------------------------------------------------+
  54 | **# of servers**   | 1                                                  |
  55 +--------------------+----------------------------------------------------+
  56 | **CPU**            | 4                                                  |
  57 |                    |                                                    |
  58 +--------------------+----------------------------------------------------+
  59 | **RAM**            | 16G                                                |
  60 |                    |                                                    |
  61 +--------------------+----------------------------------------------------+
  62 | **Disk**           | 500G * 2                                           |
  63 |                    |                                                    |
  64 +--------------------+----------------------------------------------------+
  65 | **NICs**           | 2                                                  |
  66 |                    |                                                    |
  67 +--------------------+----------------------------------------------------+
  68
  69 If want to enable the HA mode which is available in Cherry release, 2 servers are needed.
  70
  71
  72 ORAN INF Cherry test on Dell R740
  73 =================================
  74
  75 1. Installation for the first server from the O-RAN INF ISO image
  76 -----------------------------------------------------------------
  77
  78 -  Please see the README.md file for how to build the image.
  79 -  The Image is a live ISO image with CLI installer:
  80    inf-image-aio-installer-intel-corei7-64.iso
  81
  82 1.1 Burn the image to the USB device
  83 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  84
  85 -  Assume the the usb device is /dev/sdX here
  86
  87 ::
  88
  89     $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
  90
  91 1.2 Install the first Dell-R740 server (controller-0)
  92 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  93
  94 -  Reboot the target from the USB device.
  95
  96 -  Select “All-in-one Graphics console” or “All-in-one Serial console
  97    install” and press ENTER
  98
  99 -  Start the auto installation
 100
 101 -  It will reboot aotumatically after installation
 102
 103 2. Configuration and initialize the bootstrap
 104 ---------------------------------------------
 105
 106 2.1 First Login with sysadmin/sysadmin and change password
 107 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 108
 109 2.2 Set OAM network before bootstrap
 110 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 111
 112 ::
 113
 114     export OAM_DEV=eno3
 115     export CONTROLLER0_OAM_CIDR=128.224.210.110/24
 116     export DEFAULT_OAM_GATEWAY=128.224.210.1
 117     sudo ip address add $CONTROLLER0_OAM_CIDR dev $OAM_DEV
 118     sudo ip link set up dev $OAM_DEV
 119     sudo ip route add default via $DEFAULT_OAM_GATEWAY dev $OAM_DEV
 120
 121 2.3 Login the server through SSH with sysadmin
 122 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 123
 124 2.4 Prepare the localhost.yml for bootstrap
 125 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 126
 127 ::
 128
 129     cat << EOF > localhost.yml
 130     system_mode: duplex
 131     management_subnet: 192.168.18.0/24
 132     management_start_address: 192.168.18.2
 133     management_end_address: 192.168.18.50
 134     management_gateway_address: 192.168.18.1
 135     external_oam_subnet: 128.224.210.0/24
 136     external_oam_gateway_address: 128.224.210.1
 137     external_oam_floating_address: 128.224.210.110
 138     external_oam_node_0_address: 128.224.210.111
 139     external_oam_node_1_address: 128.224.210.112
 140     EOF
 141
 142 2.5 Run the ansible bootstrap
 143 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 144
 145 ::
 146
 147     ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
 148
 149 After the bootstrap successfully finish, it will show as following:
 150
 151 ::
 152
 153     PLAY RECAP *************************************************************************************************************
 154     localhost                  : ok=257  changed=151  unreachable=0    failed=0    skipped=214  rescued=0    ignored=0
 155
 156 2.6 Congiure controller-0
 157 ~~~~~~~~~~~~~~~~~~~~~~~~~
 158
 159 Acquire admin credentials:
 160
 161 ::
 162
 163     controller-0:~$ source /etc/platform/openrc
 164     [sysadmin@controller-0 ~(keystone_admin)]$
 165
 166 Configure the OAM and MGMT interfaces of controller-0 and specify the
 167 attached networks:
 168
 169 ::
 170
 171     OAM_IF=eno3
 172     MGMT_IF=eno1
 173     system host-if-modify controller-0 lo -c none
 174     IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
 175     for UUID in $IFNET_UUIDS; do
 176         system interface-network-remove ${UUID}
 177     done
 178
 179     system host-if-modify controller-0 $OAM_IF -n oam0
 180     system host-if-modify controller-0 $MGMT_IF -n pxeboot0
 181
 182     system host-if-modify controller-0 oam0 -c platform
 183     system interface-network-assign controller-0 oam0 oam
 184
 185     system host-if-modify controller-0 pxeboot0 -c platform
 186     system interface-network-assign controller-0  pxeboot0 pxeboot
 187
 188     system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
 189     system interface-network-assign controller-0 mgmt0 mgmt
 190
 191     system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
 192     system interface-network-assign controller-0 cluster0 cluster-host
 193
 194 Example output:
 195
 196 ::
 197
 198     [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
 199     [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
 200     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 lo -c none
 201     +-----------------+--------------------------------------+
 202     | Property        | Value                                |
 203     +-----------------+--------------------------------------+
 204     | ifname          | lo                                   |
 205     | iftype          | virtual                              |
 206     | ports           | []                                   |
 207     | imac            | 00:00:00:00:00:00                    |
 208     | imtu            | 1500                                 |
 209     | ifclass         | None                                 |
 210     | aemode          | None                                 |
 211     | schedpolicy     | None                                 |
 212     | txhashpolicy    | None                                 |
 213     | uuid            | 08c95952-892b-40b5-b17a-7d2ad46e725c |
 214     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 215     | vlan_id         | None                                 |
 216     | uses            | []                                   |
 217     | used_by         | []                                   |
 218     | created_at      | 2020-11-17T00:30:45.265032+00:00     |
 219     | updated_at      | 2020-11-17T01:03:39.031612+00:00     |
 220     | sriov_numvfs    | 0                                    |
 221     | sriov_vf_driver | None                                 |
 222     +-----------------+--------------------------------------+
 223     [sysadmin@controller-0 ~(keystone_admin)]$ IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
 224     [sysadmin@controller-0 ~(keystone_admin)]$ for UUID in $IFNET_UUIDS; do
 225     >     system interface-network-remove ${UUID}
 226     > done
 227     Deleted Interface Network: 0bf11f1b-4fc6-4e97-b896-3d6393a3744e
 228     Deleted Interface Network: a62d95f6-ad4e-4779-bfc0-6a885067f8d8
 229
 230     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -n oam0
 231     +-----------------+--------------------------------------+
 232     | Property        | Value                                |
 233     +-----------------+--------------------------------------+
 234     | ifname          | oam0                                 |
 235     | iftype          | ethernet                             |
 236     | ports           | [u'eno3']                            |
 237     | imac            | 24:6e:96:5d:0c:b2                    |
 238     | imtu            | 1500                                 |
 239     | ifclass         | None                                 |
 240     | aemode          | None                                 |
 241     | schedpolicy     | None                                 |
 242     | txhashpolicy    | None                                 |
 243     | uuid            | d8a048fa-67ef-43ac-8166-671be93caa30 |
 244     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 245     | vlan_id         | None                                 |
 246     | uses            | []                                   |
 247     | used_by         | []                                   |
 248     | created_at      | 2020-11-17T00:28:32.365863+00:00     |
 249     | updated_at      | 2020-11-17T01:03:45.090904+00:00     |
 250     | sriov_numvfs    | 0                                    |
 251     | sriov_vf_driver | None                                 |
 252     | accelerated     | [True]                               |
 253     +-----------------+--------------------------------------+
 254     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $MGMT_IF -n pxeboot0
 255     +-----------------+--------------------------------------+
 256     | Property        | Value                                |
 257     +-----------------+--------------------------------------+
 258     | ifname          | pxeboot0                             |
 259     | iftype          | ethernet                             |
 260     | ports           | [u'eno1']                            |
 261     | imac            | 24:6e:96:5d:0c:92                    |
 262     | imtu            | 1500                                 |
 263     | ifclass         | None                                 |
 264     | aemode          | None                                 |
 265     | schedpolicy     | None                                 |
 266     | txhashpolicy    | None                                 |
 267     | uuid            | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
 268     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 269     | vlan_id         | None                                 |
 270     | uses            | []                                   |
 271     | used_by         | []                                   |
 272     | created_at      | 2020-11-17T00:28:32.612230+00:00     |
 273     | updated_at      | 2020-11-17T01:03:47.341003+00:00     |
 274     | sriov_numvfs    | 0                                    |
 275     | sriov_vf_driver | None                                 |
 276     | accelerated     | [True]                               |
 277     +-----------------+--------------------------------------+
 278     [sysadmin@controller-0 ~(keystone_admin)]$
 279     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 oam0 -c platform
 280     +-----------------+--------------------------------------+
 281     | Property        | Value                                |
 282     +-----------------+--------------------------------------+
 283     | ifname          | oam0                                 |
 284     | iftype          | ethernet                             |
 285     | ports           | [u'eno3']                            |
 286     | imac            | 24:6e:96:5d:0c:b2                    |
 287     | imtu            | 1500                                 |
 288     | ifclass         | platform                             |
 289     | aemode          | None                                 |
 290     | schedpolicy     | None                                 |
 291     | txhashpolicy    | None                                 |
 292     | uuid            | d8a048fa-67ef-43ac-8166-671be93caa30 |
 293     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 294     | vlan_id         | None                                 |
 295     | uses            | []                                   |
 296     | used_by         | []                                   |
 297     | created_at      | 2020-11-17T00:28:32.365863+00:00     |
 298     | updated_at      | 2020-11-17T01:03:49.368879+00:00     |
 299     | sriov_numvfs    | 0                                    |
 300     | sriov_vf_driver | None                                 |
 301     | accelerated     | [True]                               |
 302     +-----------------+--------------------------------------+
 303     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 oam
 304     +--------------+--------------------------------------+
 305     | Property     | Value                                |
 306     +--------------+--------------------------------------+
 307     | hostname     | controller-0                         |
 308     | uuid         | 3c8bd181-d3f3-4e14-8e89-75a3432db1e4 |
 309     | ifname       | oam0                                 |
 310     | network_name | oam                                  |
 311     +--------------+--------------------------------------+
 312     [sysadmin@controller-0 ~(keystone_admin)]$
 313     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 pxeboot0 -c platform
 314     +-----------------+--------------------------------------+
 315     | Property        | Value                                |
 316     +-----------------+--------------------------------------+
 317     | ifname          | pxeboot0                             |
 318     | iftype          | ethernet                             |
 319     | ports           | [u'eno1']                            |
 320     | imac            | 24:6e:96:5d:0c:92                    |
 321     | imtu            | 1500                                 |
 322     | ifclass         | platform                             |
 323     | aemode          | None                                 |
 324     | schedpolicy     | None                                 |
 325     | txhashpolicy    | None                                 |
 326     | uuid            | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
 327     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 328     | vlan_id         | None                                 |
 329     | uses            | []                                   |
 330     | used_by         | []                                   |
 331     | created_at      | 2020-11-17T00:28:32.612230+00:00     |
 332     | updated_at      | 2020-11-17T01:03:53.143795+00:00     |
 333     | sriov_numvfs    | 0                                    |
 334     | sriov_vf_driver | None                                 |
 335     | accelerated     | [True]                               |
 336     +-----------------+--------------------------------------+
 337     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0  pxeboot0 pxeboot
 338     +--------------+--------------------------------------+
 339     | Property     | Value                                |
 340     +--------------+--------------------------------------+
 341     | hostname     | controller-0                         |
 342     | uuid         | 6c55622d-2da4-4f4e-ab5e-f8e06e03af7c |
 343     | ifname       | pxeboot0                             |
 344     | network_name | pxeboot                              |
 345     +--------------+--------------------------------------+
 346     [sysadmin@controller-0 ~(keystone_admin)]$
 347     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
 348     +-----------------+--------------------------------------+
 349     | Property        | Value                                |
 350     +-----------------+--------------------------------------+
 351     | ifname          | mgmt0                                |
 352     | iftype          | vlan                                 |
 353     | ports           | []                                   |
 354     | imac            | 24:6e:96:5d:0c:92                    |
 355     | imtu            | 1500                                 |
 356     | ifclass         | None                                 |
 357     | aemode          | None                                 |
 358     | schedpolicy     | None                                 |
 359     | txhashpolicy    | None                                 |
 360     | uuid            | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 |
 361     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 362     | vlan_id         | 18                                   |
 363     | uses            | [u'pxeboot0']                        |
 364     | used_by         | []                                   |
 365     | created_at      | 2020-11-17T01:03:57.303000+00:00     |
 366     | updated_at      | None                                 |
 367     | sriov_numvfs    | 0                                    |
 368     | sriov_vf_driver | None                                 |
 369     | accelerated     | [True]                               |
 370     +-----------------+--------------------------------------+
 371     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 mgmt0 mgmt
 372     +--------------+--------------------------------------+
 373     | Property     | Value                                |
 374     +--------------+--------------------------------------+
 375     | hostname     | controller-0                         |
 376     | uuid         | 2e93ef03-e9ee-457a-8667-05b52b7109a5 |
 377     | ifname       | mgmt0                                |
 378     | network_name | mgmt                                 |
 379     +--------------+--------------------------------------+
 380     [sysadmin@controller-0 ~(keystone_admin)]$
 381     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
 382     +-----------------+--------------------------------------+
 383     | Property        | Value                                |
 384     +-----------------+--------------------------------------+
 385     | ifname          | cluster0                             |
 386     | iftype          | vlan                                 |
 387     | ports           | []                                   |
 388     | imac            | 24:6e:96:5d:0c:92                    |
 389     | imtu            | 1500                                 |
 390     | ifclass         | None                                 |
 391     | aemode          | None                                 |
 392     | schedpolicy     | None                                 |
 393     | txhashpolicy    | None                                 |
 394     | uuid            | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 |
 395     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 396     | vlan_id         | 19                                   |
 397     | uses            | [u'pxeboot0']                        |
 398     | used_by         | []                                   |
 399     | created_at      | 2020-11-17T01:04:02.613518+00:00     |
 400     | updated_at      | None                                 |
 401     | sriov_numvfs    | 0                                    |
 402     | sriov_vf_driver | None                                 |
 403     | accelerated     | [True]                               |
 404     +-----------------+--------------------------------------+
 405     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 cluster0 cluster-host
 406     +--------------+--------------------------------------+
 407     | Property     | Value                                |
 408     +--------------+--------------------------------------+
 409     | hostname     | controller-0                         |
 410     | uuid         | fb8b6be6-1618-4662-b063-b1e8d340aa48 |
 411     | ifname       | cluster0                             |
 412     | network_name | cluster-host                         |
 413     +--------------+--------------------------------------+
 414     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-0
 415     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 416     | uuid                                 | name     | class    | type     | vlan id | ports     | uses i/f      | used by i/f             | attributes |
 417     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 418     | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 | mgmt0    | platform | vlan     | 18      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 419     | 23b5e923-1e53-4e70-a975-542d8380b7f2 | pxeboot0 | platform | ethernet | None    | [u'eno1'] | []            | [u'mgmt0', u'cluster0'] | MTU=1500   |
 420     | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 | cluster0 | platform | vlan     | 19      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 421     | d8a048fa-67ef-43ac-8166-671be93caa30 | oam0     | platform | ethernet | None    | [u'eno3'] | []            | []                      | MTU=1500   |
 422     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 423
 424 Configure NTP Servers for network time synchronization:
 425
 426 ::
 427
 428     system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
 429
 430 Output
 431
 432 ::
 433
 434     [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
 435     +--------------+--------------------------------------+
 436     | Property     | Value                                |
 437     +--------------+--------------------------------------+
 438     | uuid         | 3206cf01-c64a-457e-ac66-b8224c9684c3 |
 439     | ntpservers   | 0.pool.ntp.org,1.pool.ntp.org        |
 440     | isystem_uuid | cc79b616-d24e-4432-a953-85c9b242cb3a |
 441     | created_at   | 2020-11-17T00:27:23.529571+00:00     |
 442     | updated_at   | None                                 |
 443     +--------------+--------------------------------------+
 444
 445 Add an OSD on controller-0 for Ceph:
 446
 447 ::
 448
 449     system host-disk-list controller-0
 450     system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
 451     system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
 452     system host-stor-list controller-0
 453
 454 Output
 455
 456 ::
 457
 458     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
 459     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 460     | uuid                                 | device_no | device_ | device_ | size_ | available_ | rpm          | serial_id                        | device_path                                     |
 461     |                                      | de        | num     | type    | gib   | gib        |              |                                  |                                                 |
 462     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 463     | 8e2a719a-fa5a-4c25-89af-70a23fb7b238 | /dev/sda  | 2048    | HDD     | 893.  | 644.726    | Undetermined | 00c66a07604fa8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 464     |                                      |           |         |         | 75    |            |              |                                  |                                                 |
 465     |                                      |           |         |         |       |            |              |                                  |                                                 |
 466     | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/sdb  | 2064    | HDD     | 1117. | 1117.247   | Undetermined | 00c6b9139b76a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
 467     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 468     |                                      |           |         |         |       |            |              |                                  |                                                 |
 469     | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/sdc  | 2080    | HDD     | 1117. | 1117.247   | Undetermined | 0053be63c794a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
 470     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 471     |                                      |           |         |         |       |            |              |                                  |                                                 |
 472     | 4879b381-8e9f-48f3-84e2-f9c6a94bbfe0 | /dev/sdd  | 2096    | HDD     | 1117. | 0.0        | Undetermined | 0065482503bca8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
 473     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 474     |                                      |           |         |         |       |            |              |                                  |                                                 |
 475     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 476     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
 477     +------------------+-------------------------------------------------------+
 478     | Property         | Value                                                 |
 479     +------------------+-------------------------------------------------------+
 480     | osdid            | 0                                                     |
 481     | function         | osd                                                   |
 482     | state            | configuring-on-unlock                                 |
 483     | journal_location | 0816c72f-a4f0-49ea-9a95-0f02c880717c                  |
 484     | journal_size_gib | 1024                                                  |
 485     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
 486     | journal_node     | /dev/sdb2                                             |
 487     | uuid             | 0816c72f-a4f0-49ea-9a95-0f02c880717c                  |
 488     | ihost_uuid       | 16afe3a2-ba50-46b8-9fd7-09010059e8b9                  |
 489     | idisk_uuid       | 61b6f262-a51f-4310-aeac-373b1c1bbbc2                  |
 490     | tier_uuid        | 3af8c893-9dd4-40af-afc6-30bb79048448                  |
 491     | tier_name        | storage                                               |
 492     | created_at       | 2020-11-17T01:05:04.063823+00:00                      |
 493     | updated_at       | None                                                  |
 494     +------------------+-------------------------------------------------------+
 495     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
 496     +------------------+-------------------------------------------------------+
 497     | Property         | Value                                                 |
 498     +------------------+-------------------------------------------------------+
 499     | osdid            | 1                                                     |
 500     | function         | osd                                                   |
 501     | state            | configuring-on-unlock                                 |
 502     | journal_location | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5                  |
 503     | journal_size_gib | 1024                                                  |
 504     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
 505     | journal_node     | /dev/sdc2                                             |
 506     | uuid             | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5                  |
 507     | ihost_uuid       | 16afe3a2-ba50-46b8-9fd7-09010059e8b9                  |
 508     | idisk_uuid       | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79                  |
 509     | tier_uuid        | 3af8c893-9dd4-40af-afc6-30bb79048448                  |
 510     | tier_name        | storage                                               |
 511     | created_at       | 2020-11-17T01:05:06.939798+00:00                      |
 512     | updated_at       | None                                                  |
 513     +------------------+-------------------------------------------------------+
 514     [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
 515     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 516     | uuid                                 | function | osdid | state                 | idisk_uuid                           | journal_path                                          | journal_node | journal_size_gib | tier_name |
 517     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 518     | 0816c72f-a4f0-49ea-9a95-0f02c880717c | osd      | 0     | configuring-on-unlock | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2    | 1                | storage   |
 519     | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 | osd      | 1     | configuring-on-unlock | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2    | 1                | storage   |
 520     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 521
 522 2.7 Unlock controller-0
 523 ~~~~~~~~~~~~~~~~~~~~~~~
 524
 525 ::
 526
 527     system host-unlock controller-0
 528
 529 Output:
 530
 531 ::
 532
 533     [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
 534     +-----------------------+-------------------------------------------------+
 535     | Property              | Value                                           |
 536     +-----------------------+-------------------------------------------------+
 537     | action                | none                                            |
 538     | administrative        | locked                                          |
 539     | availability          | online                                          |
 540     | bm_ip                 | None                                            |
 541     | bm_type               | none                                            |
 542     | bm_username           | None                                            |
 543     | boot_device           | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 544     | capabilities          | {u'stor_function': u'monitor'}                  |
 545     | clock_synchronization | ntp                                             |
 546     | config_applied        | 6aa15fb4-8cb3-494e-b94e-95f85b560f22            |
 547     | config_status         | None                                            |
 548     | config_target         | c6ae9b2d-a3c4-4751-a79e-5487ba81ed82            |
 549     | console               | ttyS0,115200                                    |
 550     | created_at            | 2020-11-17T00:28:01.983673+00:00                |
 551     | hostname              | controller-0                                    |
 552     | id                    | 1                                               |
 553     | install_output        | graphical                                       |
 554     | install_state         | None                                            |
 555     | install_state_info    | None                                            |
 556     | inv_state             | inventoried                                     |
 557     | invprovision          | provisioning                                    |
 558     | location              | {}                                              |
 559     | mgmt_ip               | 192.168.18.3                                    |
 560     | mgmt_mac              | 24:6e:96:5d:0c:92                               |
 561     | operational           | disabled                                        |
 562     | personality           | controller                                      |
 563     | reserved              | False                                           |
 564     | rootfs_device         | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 565     | serialid              | None                                            |
 566     | software_load         | 19.12                                           |
 567     | subfunction_avail     | online                                          |
 568     | subfunction_oper      | disabled                                        |
 569     | subfunctions          | controller,worker,lowlatency                    |
 570     | task                  | Unlocking                                       |
 571     | tboot                 | false                                           |
 572     | ttys_dcd              | None                                            |
 573     | updated_at            | 2020-11-17T01:05:07.015414+00:00                |
 574     | uptime                | 3496                                            |
 575     | uuid                  | 16afe3a2-ba50-46b8-9fd7-09010059e8b9            |
 576     | vim_progress_status   | None                                            |
 577     +-----------------------+-------------------------------------------------+
 578
 579 Controller-0 will reboot to apply configuration changes and come into
 580 service. This can take 5-10 minutes, depending on the performance of the
 581 host machine.
 582
 583 Once the controller comes back up, check the status of controller-0
 584
 585 ::
 586
 587     controller-0:~$ source /etc/platform/openrc
 588     [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
 589     +----+--------------+-------------+----------------+-------------+--------------+
 590     | id | hostname     | personality | administrative | operational | availability |
 591     +----+--------------+-------------+----------------+-------------+--------------+
 592     | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
 593     +----+--------------+-------------+----------------+-------------+--------------+
 594
 595 2. Installation for the second server (controller-1)
 596 ----------------------------------------------------
 597
 598 2.1 Power on the controller-1 server and force it to network boot
 599 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 600
 601 2.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
 602 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 603
 604 2.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
 605 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 606
 607 ::
 608
 609     [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
 610     +----+--------------+-------------+----------------+-------------+--------------+
 611     | id | hostname     | personality | administrative | operational | availability |
 612     +----+--------------+-------------+----------------+-------------+--------------+
 613     | 1  | controller-0 | controller  | unlocked       | enabled     | degraded     |
 614     | 2  | None         | None        | locked         | disabled    | offline      |
 615     +----+--------------+-------------+----------------+-------------+--------------+
 616
 617 2.4 Using the host id, set the personality of this host to 'controller’:
 618 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 619
 620 ::
 621
 622     [sysadmin@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller
 623     +-----------------------+--------------------------------------+
 624     | Property              | Value                                |
 625     +-----------------------+--------------------------------------+
 626     | action                | none                                 |
 627     | administrative        | locked                               |
 628     | availability          | offline                              |
 629     | bm_ip                 | None                                 |
 630     | bm_type               | None                                 |
 631     | bm_username           | None                                 |
 632     | boot_device           | /dev/sda                             |
 633     | capabilities          | {}                                   |
 634     | clock_synchronization | ntp                                  |
 635     | config_applied        | None                                 |
 636     | config_status         | None                                 |
 637     | config_target         | None                                 |
 638     | console               | ttyS0,115200                         |
 639     | created_at            | 2020-11-17T10:17:44.387813+00:00     |
 640     | hostname              | controller-1                         |
 641     | id                    | 2                                    |
 642     | install_output        | text                                 |
 643     | install_state         | None                                 |
 644     | install_state_info    | None                                 |
 645     | inv_state             | None                                 |
 646     | invprovision          | None                                 |
 647     | location              | {}                                   |
 648     | mgmt_ip               | 192.168.18.4                         |
 649     | mgmt_mac              | 24:6e:96:5d:38:ee                    |
 650     | operational           | disabled                             |
 651     | personality           | controller                           |
 652     | reserved              | False                                |
 653     | rootfs_device         | /dev/sda                             |
 654     | serialid              | None                                 |
 655     | software_load         | 19.12                                |
 656     | subfunction_avail     | not-installed                        |
 657     | subfunction_oper      | disabled                             |
 658     | subfunctions          | controller,worker,lowlatency         |
 659     | task                  | None                                 |
 660     | tboot                 | false                                |
 661     | ttys_dcd              | None                                 |
 662     | updated_at            | None                                 |
 663     | uptime                | 0                                    |
 664     | uuid                  | f069381d-9743-49cc-bf8b-eb4bd3972203 |
 665     | vim_progress_status   | None                                 |
 666     +-----------------------+--------------------------------------+
 667
 668 2.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
 669 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 670
 671 This can take 5-10 minutes, depending on the performance of the host
 672 machine.
 673
 674 ::
 675
 676     [root@controller-0 hieradata(keystone_admin)]$ system host-list
 677     +----+--------------+-------------+----------------+-------------+--------------+
 678     | id | hostname     | personality | administrative | operational | availability |
 679     +----+--------------+-------------+----------------+-------------+--------------+
 680     | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
 681     | 2  | controller-1 | controller  | locked         | disabled    | online       |
 682     +----+--------------+-------------+----------------+-------------+--------------+
 683
 684 2.6 Configure controller-1
 685 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 686
 687 ::
 688
 689     OAM_IF=eno3
 690     MGMT_IF=eno1
 691     system host-if-modify controller-1 $OAM_IF -n oam0
 692     system host-if-modify controller-1 oam0 -c platform
 693     system interface-network-assign controller-1 oam0 oam
 694
 695     system host-if-add         -V 19 controller-1 cluster0 vlan pxeboot0
 696     system interface-network-assign controller-1 cluster0 cluster-host
 697
 698     system host-if-list controller-1
 699
 700     system host-disk-list controller-1
 701     system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
 702     system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
 703     system host-stor-list controller-1
 704
 705 Output:
 706
 707 ::
 708
 709     [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
 710     [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
 711     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 $OAM_IF -n oam0
 712     +-----------------+--------------------------------------+
 713     | Property        | Value                                |
 714     +-----------------+--------------------------------------+
 715     | ifname          | oam0                                 |
 716     | iftype          | ethernet                             |
 717     | ports           | [u'eno3']                            |
 718     | imac            | 24:6e:96:5d:39:0e                    |
 719     | imtu            | 1500                                 |
 720     | ifclass         | None                                 |
 721     | aemode          | None                                 |
 722     | schedpolicy     | None                                 |
 723     | txhashpolicy    | None                                 |
 724     | uuid            | c2473511-d0d6-445d-9739-4d43dc029de9 |
 725     | ihost_uuid      | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 726     | vlan_id         | None                                 |
 727     | uses            | []                                   |
 728     | used_by         | []                                   |
 729     | created_at      | 2020-11-22T14:01:32.365863+00:00     |
 730     | updated_at      | 2020-11-22T15:04:45.090904+00:00     |
 731     | sriov_numvfs    | 0                                    |
 732     | sriov_vf_driver | None                                 |
 733     | accelerated     | [True]                               |
 734     +-----------------+--------------------------------------+
 735     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 oam0 -c platform
 736     +-----------------+--------------------------------------+
 737     | Property        | Value                                |
 738     +-----------------+--------------------------------------+
 739     | ifname          | oam0                                 |
 740     | iftype          | ethernet                             |
 741     | ports           | [u'eno3']                            |
 742     | imac            | 24:6e:96:5d:39:0e                    |
 743     | imtu            | 1500                                 |
 744     | ifclass         | platform                             |
 745     | aemode          | None                                 |
 746     | schedpolicy     | None                                 |
 747     | txhashpolicy    | None                                 |
 748     | uuid            | c2473511-d0d6-445d-9739-4d43dc029de9 |
 749     | ihost_uuid      | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 750     | vlan_id         | None                                 |
 751     | uses            | []                                   |
 752     | used_by         | []                                   |
 753     | created_at      | 2020-11-22T14:05:16.052229+00:00     |
 754     | updated_at      | 2020-11-22T15:08:35.324634+00:00     |
 755     | sriov_numvfs    | 0                                    |
 756     | sriov_vf_driver | None                                 |
 757     | accelerated     | [True]                               |
 758     +-----------------+--------------------------------------+
 759     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 oam0 oam
 760     +--------------+--------------------------------------+
 761     | Property     | Value                                |
 762     +--------------+--------------------------------------+
 763     | hostname     | controller-1                         |
 764     | uuid         | f2e7f088-0dd0-4adc-8348-4e3cef23bc47 |
 765     | ifname       | oam0                                 |
 766     | network_name | oam                                  |
 767     +--------------+--------------------------------------+
 768     [sysadmin@controller-0 ~(keystone_admin)]$
 769
 770     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
 771     +-----------------+--------------------------------------+
 772     | Property        | Value                                |
 773     +-----------------+--------------------------------------+
 774     | ifname          | cluster0                             |
 775     | iftype          | vlan                                 |
 776     | ports           | []                                   |
 777     | imac            | 24:6e:96:5d:38:ee                    |
 778     | imtu            | 1500                                 |
 779     | ifclass         | None                                 |
 780     | aemode          | None                                 |
 781     | schedpolicy     | None                                 |
 782     | txhashpolicy    | None                                 |
 783     | uuid            | b6783682-b2aa-4135-90d2-676e1db41ae8 |
 784     | ihost_uuid      | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 785     | vlan_id         | 19                                   |
 786     | uses            | [u'pxeboot0']                        |
 787     | used_by         | []                                   |
 788     | created_at      | 2020-11-22T15:08:43.932209+00:00     |
 789     | updated_at      | None                                 |
 790     | sriov_numvfs    | 0                                    |
 791     | sriov_vf_driver | None                                 |
 792     | accelerated     | [True]                               |
 793     +-----------------+--------------------------------------+
 794
 795     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 cluster0 cluster-host
 796     +--------------+--------------------------------------+
 797     | Property     | Value                                |
 798     +--------------+--------------------------------------+
 799     | hostname     | controller-1                         |
 800     | uuid         | 8fc64805-b54b-45a4-b88a-e13b236abfe8 |
 801     | ifname       | cluster0                             |
 802     | network_name | cluster-host                         |
 803     +--------------+--------------------------------------+
 804     [sysadmin@controller-0 ~(keystone_admin)]$
 805     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-1
 806     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 807     | uuid                                 | name     | class    | type     | vlan id | ports     | uses i/f      | used by i/f             | attributes |
 808     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 809     | b6783682-b2aa-4135-90d2-676e1db41ae8 | cluster0 | platform | vlan     | 19      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 810     | b8921960-fde5-44c3-960d-2aebf42ea400 | pxeboot0 | platform | ethernet | None    | [u'eno1'] | []            | [u'mgmt0', u'cluster0'] | MTU=1500   |
 811     | c103275b-2b75-4568-865f-ac6be32ecb2d | mgmt0    | platform | vlan     | 18      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 812     | c2473511-d0d6-445d-9739-4d43dc029de9 | oam0     | platform | ethernet | None    | [u'eno3'] | []            | []                      | MTU=1500   |
 813     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 814     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
 815     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 816     | uuid                                 | device_no | device_ | device_ | size_ | available_ | rpm          | serial_id                        | device_path                                     |
 817     |                                      | de        | num     | type    | gib   | gib        |              |                                  |                                                 |
 818     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 819     | 5b8fade4-b048-48fa-b906-9dcbdbed8e96 | /dev/sda  | 2048    | HDD     | 893.  | 644.726    | Undetermined | 00cbd97f3e36ccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 820     |                                      |           |         |         | 75    |            |              |                                  |                                                 |
 821     |                                      |           |         |         |       |            |              |                                  |                                                 |
 822     | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/sdb  | 2064    | HDD     | 1117. | 1117.247   | Undetermined | 006d0e977b5fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
 823     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 824     |                                      |           |         |         |       |            |              |                                  |                                                 |
 825     | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/sdc  | 2080    | HDD     | 1117. | 1117.247   | Undetermined | 003a2377ac7fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
 826     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 827     |                                      |           |         |         |       |            |              |                                  |                                                 |
 828     | 774c3cd0-1178-4145-9573-f0d6dee2ba06 | /dev/sdd  | 2096    | HDD     | 1117. | 1117.247   | Undetermined | 00d7093ef0adccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
 829     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 830     |                                      |           |         |         |       |            |              |                                  |                                                 |
 831     | 00361302-8d55-4730-855c-b0098c73ab7e | /dev/sde  | 2112    | SSD     | 223.  | 223.568    | N/A          | PHDW730104QM240E                 | /dev/disk/by-path/pci-0000:d8:00.0-ata-1        |
 832     |                                      |           |         |         | 57    |            |              |                                  |                                                 |
 833     |                                      |           |         |         |       |            |              |                                  |                                                 |
 834     | 7ce735e6-920f-4424-a890-a7a7f48d7632 | /dev/sdf  | 2128    | SSD     | 223.  | 223.568    | N/A          | PHDW730104LL240E                 | /dev/disk/by-path/pci-0000:d8:00.0-ata-2        |
 835     |                                      |           |         |         | 57    |            |              |                                  |                                                 |
 836     |                                      |           |         |         |       |            |              |                                  |                                                 |
 837     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 838     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
 839     +------------------+-------------------------------------------------------+
 840     | Property         | Value                                                 |
 841     +------------------+-------------------------------------------------------+
 842     | osdid            | 2                                                     |
 843     | function         | osd                                                   |
 844     | state            | configuring-on-unlock                                 |
 845     | journal_location | 54a218d8-0466-4366-9ef0-3ec5a952fde7                  |
 846     | journal_size_gib | 1024                                                  |
 847     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
 848     | journal_node     | /dev/sdb2                                             |
 849     | uuid             | 54a218d8-0466-4366-9ef0-3ec5a952fde7                  |
 850     | ihost_uuid       | 63c930c7-2195-4d5a-870c-be610fd6b4fc                  |
 851     | idisk_uuid       | 1a3f0a36-5961-42e5-a271-e71db1c25d42                  |
 852     | tier_uuid        | 06b4740e-29db-4896-9600-03ee40fe0d6c                  |
 853     | tier_name        | storage                                               |
 854     | created_at       | 2020-11-22T15:11:55.641193+00:00                      |
 855     | updated_at       | None                                                  |
 856     +------------------+-------------------------------------------------------+
 857
 858     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
 859     +------------------+-------------------------------------------------------+
 860     | Property         | Value                                                 |
 861     +------------------+-------------------------------------------------------+
 862     | osdid            | 3                                                     |
 863     | function         | osd                                                   |
 864     | state            | configuring-on-unlock                                 |
 865     | journal_location | 5be88c7a-3a94-4b97-9da5-b247bb89406c                  |
 866     | journal_size_gib | 1024                                                  |
 867     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
 868     | journal_node     | /dev/sdc2                                             |
 869     | uuid             | 5be88c7a-3a94-4b97-9da5-b247bb89406c                  |
 870     | ihost_uuid       | 63c930c7-2195-4d5a-870c-be610fd6b4fc                  |
 871     | idisk_uuid       | eddd732f-2cea-49b3-86db-b722c0b1a1ae                  |
 872     | tier_uuid        | 06b4740e-29db-4896-9600-03ee40fe0d6c                  |
 873     | tier_name        | storage                                               |
 874     | created_at       | 2020-11-22T15:12:04.274839+00:00                      |
 875     | updated_at       | None                                                  |
 876     +------------------+-------------------------------------------------------+
 877
 878     [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-1
 879     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 880     | uuid                                 | function | osdid | state                 | idisk_uuid                           | journal_path                                          | journal_node | journal_size_gib | tier_name |
 881     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 882     | 54a218d8-0466-4366-9ef0-3ec5a952fde7 | osd      | 2     | configuring-on-unlock | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2    | 1                | storage   |
 883     | 5be88c7a-3a94-4b97-9da5-b247bb89406c | osd      | 3     | configuring-on-unlock | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2    | 1                | storage   |
 884     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 885
 886 2.7 Unlock controller-1
 887 ~~~~~~~~~~~~~~~~~~~~~~~
 888
 889 Unlock controller-1 in order to bring it into service:
 890
 891 ::
 892
 893     [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-1
 894     +-----------------------+--------------------------------------+
 895     | Property              | Value                                |
 896     +-----------------------+--------------------------------------+
 897     | action                | none                                 |
 898     | administrative        | locked                               |
 899     | availability          | online                               |
 900     | bm_ip                 | None                                 |
 901     | bm_type               | None                                 |
 902     | bm_username           | None                                 |
 903     | boot_device           | /dev/sda                             |
 904     | capabilities          | {u'stor_function': u'monitor'}       |
 905     | clock_synchronization | ntp                                  |
 906     | config_applied        | None                                 |
 907     | config_status         | Config out-of-date                   |
 908     | config_target         | 9747e0ce-2319-409d-b75c-2475bc5065ac |
 909     | console               | ttyS0,115200                         |
 910     | created_at            | 2020-11-22T12:58:11.630526+00:00     |
 911     | hostname              | controller-1                         |
 912     | id                    | 3                                    |
 913     | install_output        | text                                 |
 914     | install_state         | None                                 |
 915     | install_state_info    | None                                 |
 916     | inv_state             | inventoried                          |
 917     | invprovision          | unprovisioned                        |
 918     | location              | {}                                   |
 919     | mgmt_ip               | 192.168.18.4                         |
 920     | mgmt_mac              | 24:6e:96:5d:38:ee                    |
 921     | operational           | disabled                             |
 922     | personality           | controller                           |
 923     | reserved              | False                                |
 924     | rootfs_device         | /dev/sda                             |
 925     | serialid              | None                                 |
 926     | software_load         | 19.12                                |
 927     | subfunction_avail     | online                               |
 928     | subfunction_oper      | disabled                             |
 929     | subfunctions          | controller,worker,lowlatency         |
 930     | task                  | Unlocking                            |
 931     | tboot                 | false                                |
 932     | ttys_dcd              | None                                 |
 933     | updated_at            | 2020-11-22T15:13:09.716324+00:00     |
 934     | uptime                | 752                                  |
 935     | uuid                  | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 936     | vim_progress_status   | None                                 |
 937     +-----------------------+--------------------------------------+
 938
 939 Controller-1 will reboot in order to apply configuration changes and
 940 come into service. This can take 5-10 minutes, depending on the
 941 performance of the host machine.
 942
 943 ::
 944
 945     [root@controller-0 hieradata(keystone_admin)]$ system host-list
 946     +----+--------------+-------------+----------------+-------------+--------------+
 947     | id | hostname     | personality | administrative | operational | availability |
 948     +----+--------------+-------------+----------------+-------------+--------------+
 949     | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
 950     | 2  | controller-1 | controller  | unlocked       | enabled     | available    |
 951     +----+--------------+-------------+----------------+-------------+--------------+
 952
 953     [sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-1
 954     +-----------------------+-----------------------------------------------------------------------+
 955     | Property              | Value                                                                 |
 956     +-----------------------+-----------------------------------------------------------------------+
 957     | action                | none                                                                  |
 958     | administrative        | unlocked                                                              |
 959     | availability          | available                                                             |
 960     | bm_ip                 | None                                                                  |
 961     | bm_type               | None                                                                  |
 962     | bm_username           | None                                                                  |
 963     | boot_device           | /dev/sda                                                              |
 964     | capabilities          | {u'stor_function': u'monitor', u'Personality': u'Controller-Standby'} |
 965     | clock_synchronization | ntp                                                                   |
 966     | config_applied        | 9747e0ce-2319-409d-b75c-2475bc5065ac                                  |
 967     | config_status         | None                                                                  |
 968     | config_target         | 9747e0ce-2319-409d-b75c-2475bc5065ac                                  |
 969     | console               | ttyS0,115200                                                          |
 970     | created_at            | 2020-11-22T12:58:11.630526+00:00                                      |
 971     | hostname              | controller-1                                                          |
 972     | id                    | 2                                                                     |
 973     | install_output        | text                                                                  |
 974     | install_state         | None                                                                  |
 975     | install_state_info    | None                                                                  |
 976     | inv_state             | inventoried                                                           |
 977     | invprovision          | provisioned                                                           |
 978     | location              | {}                                                                    |
 979     | mgmt_ip               | 192.168.18.4                                                          |
 980     | mgmt_mac              | 24:6e:96:5d:38:ee                                                     |
 981     | operational           | enabled                                                               |
 982     | personality           | controller                                                            |
 983     | reserved              | False                                                                 |
 984     | rootfs_device         | /dev/sda                                                              |
 985     | serialid              | None                                                                  |
 986     | software_load         | 19.12                                                                 |
 987     | subfunction_avail     | available                                                             |
 988     | subfunction_oper      | enabled                                                               |
 989     | subfunctions          | controller,worker,lowlatency                                          |
 990     | task                  |                                                                       |
 991     | tboot                 | false                                                                 |
 992     | ttys_dcd              | None                                                                  |
 993     | updated_at            | 2020-11-22T23:59:07.787759+00:00                                      |
 994     | uptime                | 31008                                                                 |
 995     | uuid                  | 63c930c7-2195-4d5a-870c-be610fd6b4fc                                  |
 996     | vim_progress_status   | services-enabled                                                      |
 997     +-----------------------+-----------------------------------------------------------------------+
 998
 999
1000 3. Simple use case for sriov
1001 ````````````````````````````
1002
1003 3.1 After controller-0 is rebooted and up running, download the DPDK
1004 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
1005
1006 ::
1007
1008   [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
1009   [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
1010   Password:
1011   --2020-06-04 02:35:30--  https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
1012   Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
1013   Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
1014
1015   HTTP request sent, awaiting response... 200 OK
1016   Length: 10251680 (9.8M) [application/octet-stream]
1017   Saving to: ‘dpdk-17.11.10.tar.xz’
1018
1019   dpdk-17.11.10.tar.xz                        100%
1020   [========================================================================================>]   9.78M
1021   1.48MB/s    in 6.8s
1022
1023   2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
1024
1025   sudo tar xvf dpdk-17.11.10.tar.xz
1026
1027   sudo ln -s dpdk-stable-17.11.10 dpdk-stable
1028
1029 3.2 Prepare the yaml file for the network assignment container
1030 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
1031
1032 The following the exmaple of the yaml file:
1033
1034 ::
1035
1036   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
1037   > apiVersion: "k8s.cni.cncf.io/v1"
1038   > kind: NetworkAttachmentDefinition
1039   > metadata:
1040   >   name: sriov-data-dpdk-0
1041   >   annotations:
1042   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
1043   > spec:
1044   >   config: '{
1045   >   "type": "sriov",
1046   >   "name": "sriov-data-dpdk-0"
1047   > }'
1048   >
1049   > ---
1050   > apiVersion: "k8s.cni.cncf.io/v1"
1051   > kind: NetworkAttachmentDefinition
1052   > metadata:
1053   >   name: sriov-data-dpdk-1
1054   >   annotations:
1055   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
1056   > spec:
1057   >   config: '{
1058   >   "type": "sriov",
1059   >   "name": "sriov-data-dpdk-1"
1060   > }'
1061   > EOF
1062
1063 3.3 Run the network assignent container for the 2 VFs
1064 '''''''''''''''''''''''''''''''''''''''''''''''''''''
1065
1066 ::
1067
1068   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
1069   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
1070   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
1071
1072 3.4 Prepare the VF container yaml file
1073 ''''''''''''''''''''''''''''''''''''''
1074
1075 ::
1076
1077   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
1078   > apiVersion: v1
1079   > kind: Pod
1080   metadata:
1081   > metadata:
1082   >   name: pod-with-dpdk-vfs-0
1083   >   annotations:
1084   >     k8s.v1.cni.cncf.io/networks: '[
1085   >             { "name": "sriov-data-dpdk-0" },
1086               { "name": "sriov-data-dpdk-1" }
1087   >             { "name": "sriov-data-dpdk-1" }
1088   >     ]'
1089   > spec:
1090   >   restartPolicy: Never
1091   >   containers:
1092   >   - name: pod-with-dpdk-vfs-0
1093   >     image: wrsnfv/ubuntu-dpdk-build:v0.3
1094   >     env:
1095   >     - name: RTE_SDK
1096   >       value: "/usr/src/dpdk"
1097   >     command:
1098   >     - sleep
1099   >     - infinity
1100   >     stdin: true
1101   >     tty: true
1102   >     securityContext:
1103   >       privileged: true
1104   >       capabilities:
1105   >         add:
1106   >         - ALL
1107   >     resources:
1108   >       requests:
1109   >         cpu: 4
1110   >         memory: 4Gi
1111   >         intel.com/pci_sriov_net_physnet0: '1'
1112   >         intel.com/pci_sriov_net_physnet1: '1'
1113   >       limits:
1114   >         cpu: 4
1115   >         hugepages-1Gi: 2Gi
1116   >         memory: 4Gi
1117   >         intel.com/pci_sriov_net_physnet0: '1'
1118   >         intel.com/pci_sriov_net_physnet1: '1'
1119   >     volumeMounts:
1120   >     - mountPath: /mnt/huge-1048576kB
1121   >       name: hugepage
1122   >     - name: dpdk-volume
1123   >       mountPath: /usr/src/dpdk
1124   >     - name: lib-volume
1125   >       mountPath: /lib/modules
1126   >     - name: src-volume
1127   >       mountPath: /usr/src/
1128   >   volumes:
1129   >   - name: hugepage
1130   >     emptyDir:
1131   >       medium: HugePages
1132   >   - name: dpdk-volume
1133   >     hostPath:
1134   >       path: /opt/dpdk-stable/
1135   >   - name: lib-volume
1136   >     hostPath:
1137   >       path: /lib/modules
1138   >   - name: src-volume
1139   >     hostPath:
1140   >       path: /usr/src/
1141   > EOF
1142
1143 3.5 Run the VF container
1144 ''''''''''''''''''''''''
1145
1146 Start the VF container.
1147
1148 ::
1149
1150   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
1151   pod/pod-with-dpdk-vfs-0 created
1152
1153   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
1154   NAME                  READY   STATUS    RESTARTS   AGE
1155   pod-with-dpdk-vfs-0   1/1     Running   0          6m40s
1156
1157 Login the VF container
1158
1159 ::
1160
1161   kubectl exec -it pod-with-dpdk-vfs-0 -- bash
1162
1163 Build the DPDK
1164
1165 ::
1166
1167   cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
1168
1169   root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
1170     HOSTCC  scripts/basic/fixdep
1171     HOSTCC  scripts/kconfig/conf.o
1172     HOSTCC  scripts/kconfig/confdata.o
1173     HOSTCC  scripts/kconfig/expr.o
1174     HOSTCC  scripts/kconfig/symbol.o
1175     HOSTCC  scripts/kconfig/preprocess.o
1176     HOSTCC  scripts/kconfig/zconf.lex.o
1177     HOSTCC  scripts/kconfig/zconf.tab.o
1178     HOSTLD  scripts/kconfig/conf
1179   scripts/kconfig/conf  --syncconfig Kconfig
1180     HOSTCC  arch/x86/tools/relocs_32.o
1181     HOSTCC  arch/x86/tools/relocs_64.o
1182     HOSTCC  arch/x86/tools/relocs_common.o
1183     HOSTLD  arch/x86/tools/relocs
1184     HOSTCC  scripts/genksyms/genksyms.o
1185     YACC    scripts/genksyms/parse.tab.c
1186     HOSTCC  scripts/genksyms/parse.tab.o
1187     LEX     scripts/genksyms/lex.lex.c
1188     YACC    scripts/genksyms/parse.tab.h
1189     HOSTCC  scripts/genksyms/lex.lex.o
1190     HOSTLD  scripts/genksyms/genksyms
1191     HOSTCC  scripts/bin2c
1192     HOSTCC  scripts/kallsyms
1193     HOSTCC  scripts/conmakehash
1194     HOSTCC  scripts/recordmcount
1195     HOSTCC  scripts/sortextable
1196     HOSTCC  scripts/asn1_compiler
1197     HOSTCC  scripts/sign-file
1198     HOSTCC  scripts/extract-cert
1199     CC      scripts/mod/empty.o
1200     HOSTCC  scripts/mod/mk_elfconfig
1201     MKELF   scripts/mod/elfconfig.h
1202     HOSTCC  scripts/mod/modpost.o
1203     CC      scripts/mod/devicetable-offsets.s
1204     UPD     scripts/mod/devicetable-offsets.h
1205     HOSTCC  scripts/mod/file2alias.o
1206     HOSTCC  scripts/mod/sumversion.o
1207     HOSTLD  scripts/mod/modpost
1208     CC      kernel/bounds.s
1209     CC      arch/x86/kernel/asm-offsets.s
1210     CALL    scripts/checksyscalls.sh
1211
1212 Build the test_pmd application
1213
1214 ::
1215
1216   cd $RTE_SDK
1217   ./usertools/dpdk-setup.sh
1218   Option: 14
1219     CC config.o
1220     CC iofwd.o
1221     CC macfwd.o
1222     CC macswap.o
1223     CC flowgen.o
1224     CC rxonly.o
1225     CC txonly.o
1226     CC csumonly.o
1227     CC icmpecho.o
1228     CC tm.o
1229     LD testpmd
1230     INSTALL-APP testpmd
1231     INSTALL-MAP testpmd.map
1232   == Build app/proc_info
1233     CC main.o
1234     LD dpdk-procinfo
1235     INSTALL-APP dpdk-procinfo
1236     INSTALL-MAP dpdk-procinfo.map
1237   == Build app/pdump
1238     CC main.o
1239     LD dpdk-pdump
1240     INSTALL-APP dpdk-pdump
1241     INSTALL-MAP dpdk-pdump.map
1242   == Build app/test-crypto-perf
1243     CC main.o
1244     CC cperf_ops.o
1245     CC cperf_options_parsing.o
1246     CC cperf_test_vectors.o
1247     CC cperf_test_throughput.o
1248     CC cperf_test_latency.o
1249     CC cperf_test_pmd_cyclecount.o
1250     CC cperf_test_verify.o
1251     CC cperf_test_vector_parsing.o
1252     CC cperf_test_common.o
1253     LD dpdk-test-crypto-perf
1254     INSTALL-APP dpdk-test-crypto-perf
1255     INSTALL-MAP dpdk-test-crypto-perf.map
1256   == Build app/test-eventdev
1257     CC evt_main.o
1258     CC evt_options.o
1259     CC evt_test.o
1260     CC parser.o
1261     CC test_order_common.o
1262     CC test_order_queue.o
1263     CC test_order_atq.o
1264     CC test_perf_common.o
1265     CC test_perf_queue.o
1266     CC test_perf_atq.o
1267     LD dpdk-test-eventdev
1268     INSTALL-APP dpdk-test-eventdev
1269     INSTALL-MAP dpdk-test-eventdev.map
1270   Build complete [x86_64-native-linuxapp-gcc]
1271   Installation cannot run with T defined and DESTDIR undefined
1272   ------------------------------------------------------------------------------
1273   RTE_TARGET exported as x86_64-native-linuxapp-gcc
1274   ------------------------------------------------------------------------------
1275
1276   Press enter to continue ...
1277
1278 Check the VF PCI information:
1279
1280 ::
1281
1282   root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
1283   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
1284   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
1285
1286 Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
1287
1288 ::
1289
1290   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
1291   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
1292   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
1293   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
1294   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
1295   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
1296   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
1297
1298   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
1299   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
1300   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
1301   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
1302   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
1303   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
1304   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
1305
1306   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
1307   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
1308
1309
1310 Get back to pod and run the test_pmd
1311
1312 ::
1313
1314   cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
1315
1316   taskset -p --cpu-list 1
1317   pid 1's current affinity list: 2,3,18,19
1318
1319   ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
1320   start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
1321   peer=1,"9e:fd:e6:dd:c1:01"
1322   EAL: Detected 32 lcore(s)
1323   EAL: No free hugepages reported in hugepages-2048kB
1324   EAL: Probing VFIO support...
1325   EAL: VFIO support initialized
1326   EAL: PCI device 0000:05:11.0 on NUMA socket 0
1327   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1328   EAL:   using IOMMU type 1 (Type 1)
1329   EAL: PCI device 0000:05:11.1 on NUMA socket 0
1330   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1331   Auto-start selected
1332   Ports to start sending a burst of packets first
1333   Warning: lsc_interrupt needs to be off when  using tx_first. Disabling.
1334   USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
1335   Configuring Port 0 (socket 0)
1336   Port 0: 9E:FD:E6:DD:C1:01
1337   Configuring Port 1 (socket 0)
1338   Port 1: 9E:FD:E6:DD:C1:02
1339   Checking link statuses...
1340   Port0 Link Up. speed 10000 Mbps- full-duplex
1341   Port1 Link Up. speed 10000 Mbps- full-duplex
1342   Done
1343   No commandline core given, start packet forwarding
1344   io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
1345   disabled
1346   Logical Core 3 (socket 0) forwards packets on 2 streams:
1347     RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
1348     RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
1349
1350     io packet forwarding packets/burst=32
1351     nb forwarding cores=1 - nb forwarding ports=2
1352     port 0:
1353     CRC stripping enabled
1354     RX queues=1 - RX desc=128 - RX free threshold=32
1355     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1356     TX queues=1 - TX desc=512 - TX free threshold=32
1357     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1358     TX RS bit threshold=32 - TXQ flags=0xf01
1359     port 1:
1360     CRC stripping enabled
1361     RX queues=1 - RX desc=128 - RX free threshold=32
1362     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1363     TX queues=1 - TX desc=512 - TX free threshold=32
1364     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1365     TX RS bit threshold=32 - TXQ flags=0xf01
1366
1367   Port statistics ====================================
1368     ######################## NIC statistics for port 0  ########################
1369     RX-packets: 56         RX-missed: 0          RX-bytes:  4096
1370     RX-errors: 0
1371     RX-nombuf:  0
1372     TX-packets: 64         TX-errors: 0          TX-bytes:  4096
1373
1374     Throughput (since last show)
1375     Rx-pps:            0
1376     Tx-pps:            0
1377     ############################################################################
1378
1379     ######################## NIC statistics for port 1  ########################
1380     RX-packets: 432        RX-missed: 0          RX-bytes:  27712
1381     RX-errors: 0
1382     RX-nombuf:  0
1383     TX-packets: 461        TX-errors: 0          TX-bytes:  30080
1384
1385     Throughput (since last show)
1386     Rx-pps:            0
1387     Tx-pps:            0
1388     ############################################################################
1389
1390   Port statistics ====================================
1391     ######################## NIC statistics for port 0  ########################
1392     RX-packets: 14124641   RX-missed: 0          RX-bytes:  903977344
1393     RX-errors: 0
1394     RX-nombuf:  0
1395     TX-packets: 14170205   TX-errors: 0          TX-bytes:  906893376
1396
1397     Throughput (since last show)
1398     Rx-pps:      7068409
1399     Tx-pps:      7091206
1400     ############################################################################
1401
1402
1403
1404
1405
1406
1407 References
1408 ----------
1409
1410 - `StarlingX`_
1411
1412 .. _`StarlingX`: https://docs.starlingx.io/
1413