docs/installation-guide.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3 .. Copyright (C) 2019 Wind River Systems, Inc.
   4
   5
   6 Installation Guide
   7 ==================
   8
   9 .. contents::
  10    :depth: 3
  11    :local:
  12
  13 Abstract
  14 --------
  15
  16 This document describes how to install O-RAN INF image, example configuration for better
  17 real time performance, and example deployment of Kubernetes cluster and plugins.
  18
  19 The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
  20 and container technology.
  21
  22
  23 Preface
  24 -------
  25
  26 Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
  27
  28
  29 Hardware Requirements
  30 ---------------------
  31
  32 Following minimum hardware requirements must be met for installation of O-RAN INF image:
  33
  34 +--------------------+----------------------------------------------------+
  35 | **HW Aspect**      | **Requirement**                                    |
  36 |                    |                                                    |
  37 +--------------------+----------------------------------------------------+
  38 | **# of servers**   | 1                                                  |
  39 +--------------------+----------------------------------------------------+
  40 | **CPU**            | 4                                                  |
  41 |                    |                                                    |
  42 +--------------------+----------------------------------------------------+
  43 | **RAM**            | 16G                                                |
  44 |                    |                                                    |
  45 +--------------------+----------------------------------------------------+
  46 | **Disk**           | 500G * 2                                           |
  47 |                    |                                                    |
  48 +--------------------+----------------------------------------------------+
  49 | **NICs**           | 2                                                  |
  50 |                    |                                                    |
  51 +--------------------+----------------------------------------------------+
  52
  53 If want to enable the HA mode which is available in Cherry release, 2 servers are needed.
  54
  55
  56 ORAN INF Cherry test on Dell R740
  57 =================================
  58
  59 1. Installation for the first server from the O-RAN INF ISO image
  60 -----------------------------------------------------------------
  61
  62 -  Please see the README.md file for how to build the image.
  63 -  The Image is a live ISO image with CLI installer:
  64    inf-image-aio-installer-intel-corei7-64.iso
  65
  66 1.1 Burn the image to the USB device
  67 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  68
  69 -  Assume the the usb device is /dev/sdX here
  70
  71 ::
  72
  73     $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
  74
  75 1.2 Install the first Dell-R740 server (controller-0)
  76 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  77
  78 -  Reboot the target from the USB device.
  79
  80 -  Select “All-in-one Graphics console” or “All-in-one Serial console
  81    install” and press ENTER
  82
  83 -  Start the auto installation
  84
  85 -  It will reboot aotumatically after installation
  86
  87 2. Configuration and initialize the bootstrap
  88 ---------------------------------------------
  89
  90 2.1 First Login with sysadmin/sysadmin and change password
  91 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  92
  93 2.2 Set OAM network before bootstrap
  94 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  95
  96 ::
  97
  98     export OAM_DEV=eno3
  99     export CONTROLLER0_OAM_CIDR=128.224.210.110/24
 100     export DEFAULT_OAM_GATEWAY=128.224.210.1
 101     sudo ip address add $CONTROLLER0_OAM_CIDR dev $OAM_DEV
 102     sudo ip link set up dev $OAM_DEV
 103     sudo ip route add default via $DEFAULT_OAM_GATEWAY dev $OAM_DEV
 104
 105 2.3 Login the server through SSH with sysadmin
 106 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 107
 108 2.4 Prepare the localhost.yml for bootstrap
 109 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 110
 111 ::
 112
 113     cat << EOF > localhost.yml
 114     system_mode: duplex
 115     management_subnet: 192.168.18.0/24
 116     management_start_address: 192.168.18.2
 117     management_end_address: 192.168.18.50
 118     management_gateway_address: 192.168.18.1
 119     external_oam_subnet: 128.224.210.0/24
 120     external_oam_gateway_address: 128.224.210.1
 121     external_oam_floating_address: 128.224.210.110
 122     external_oam_node_0_address: 128.224.210.111
 123     external_oam_node_1_address: 128.224.210.112
 124     EOF
 125
 126 2.5 Run the ansible bootstrap
 127 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 128
 129 ::
 130
 131     ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
 132
 133 After the bootstrap successfully finish, it will show as following:
 134
 135 ::
 136
 137     PLAY RECAP *************************************************************************************************************
 138     localhost                  : ok=257  changed=151  unreachable=0    failed=0    skipped=214  rescued=0    ignored=0
 139
 140 2.6 Congiure controller-0
 141 ~~~~~~~~~~~~~~~~~~~~~~~~~
 142
 143 Acquire admin credentials:
 144
 145 ::
 146
 147     controller-0:~$ source /etc/platform/openrc
 148     [sysadmin@controller-0 ~(keystone_admin)]$
 149
 150 Configure the OAM and MGMT interfaces of controller-0 and specify the
 151 attached networks:
 152
 153 ::
 154
 155     OAM_IF=eno3
 156     MGMT_IF=eno1
 157     system host-if-modify controller-0 lo -c none
 158     IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
 159     for UUID in $IFNET_UUIDS; do
 160         system interface-network-remove ${UUID}
 161     done
 162
 163     system host-if-modify controller-0 $OAM_IF -n oam0
 164     system host-if-modify controller-0 $MGMT_IF -n pxeboot0
 165
 166     system host-if-modify controller-0 oam0 -c platform
 167     system interface-network-assign controller-0 oam0 oam
 168
 169     system host-if-modify controller-0 pxeboot0 -c platform
 170     system interface-network-assign controller-0  pxeboot0 pxeboot
 171
 172     system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
 173     system interface-network-assign controller-0 mgmt0 mgmt
 174
 175     system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
 176     system interface-network-assign controller-0 cluster0 cluster-host
 177
 178 Example output:
 179
 180 ::
 181
 182     [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
 183     [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
 184     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 lo -c none
 185     +-----------------+--------------------------------------+
 186     | Property        | Value                                |
 187     +-----------------+--------------------------------------+
 188     | ifname          | lo                                   |
 189     | iftype          | virtual                              |
 190     | ports           | []                                   |
 191     | imac            | 00:00:00:00:00:00                    |
 192     | imtu            | 1500                                 |
 193     | ifclass         | None                                 |
 194     | aemode          | None                                 |
 195     | schedpolicy     | None                                 |
 196     | txhashpolicy    | None                                 |
 197     | uuid            | 08c95952-892b-40b5-b17a-7d2ad46e725c |
 198     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 199     | vlan_id         | None                                 |
 200     | uses            | []                                   |
 201     | used_by         | []                                   |
 202     | created_at      | 2020-11-17T00:30:45.265032+00:00     |
 203     | updated_at      | 2020-11-17T01:03:39.031612+00:00     |
 204     | sriov_numvfs    | 0                                    |
 205     | sriov_vf_driver | None                                 |
 206     +-----------------+--------------------------------------+
 207     [sysadmin@controller-0 ~(keystone_admin)]$ IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
 208     [sysadmin@controller-0 ~(keystone_admin)]$ for UUID in $IFNET_UUIDS; do
 209     >     system interface-network-remove ${UUID}
 210     > done
 211     Deleted Interface Network: 0bf11f1b-4fc6-4e97-b896-3d6393a3744e
 212     Deleted Interface Network: a62d95f6-ad4e-4779-bfc0-6a885067f8d8
 213
 214     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -n oam0
 215     +-----------------+--------------------------------------+
 216     | Property        | Value                                |
 217     +-----------------+--------------------------------------+
 218     | ifname          | oam0                                 |
 219     | iftype          | ethernet                             |
 220     | ports           | [u'eno3']                            |
 221     | imac            | 24:6e:96:5d:0c:b2                    |
 222     | imtu            | 1500                                 |
 223     | ifclass         | None                                 |
 224     | aemode          | None                                 |
 225     | schedpolicy     | None                                 |
 226     | txhashpolicy    | None                                 |
 227     | uuid            | d8a048fa-67ef-43ac-8166-671be93caa30 |
 228     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 229     | vlan_id         | None                                 |
 230     | uses            | []                                   |
 231     | used_by         | []                                   |
 232     | created_at      | 2020-11-17T00:28:32.365863+00:00     |
 233     | updated_at      | 2020-11-17T01:03:45.090904+00:00     |
 234     | sriov_numvfs    | 0                                    |
 235     | sriov_vf_driver | None                                 |
 236     | accelerated     | [True]                               |
 237     +-----------------+--------------------------------------+
 238     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $MGMT_IF -n pxeboot0
 239     +-----------------+--------------------------------------+
 240     | Property        | Value                                |
 241     +-----------------+--------------------------------------+
 242     | ifname          | pxeboot0                             |
 243     | iftype          | ethernet                             |
 244     | ports           | [u'eno1']                            |
 245     | imac            | 24:6e:96:5d:0c:92                    |
 246     | imtu            | 1500                                 |
 247     | ifclass         | None                                 |
 248     | aemode          | None                                 |
 249     | schedpolicy     | None                                 |
 250     | txhashpolicy    | None                                 |
 251     | uuid            | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
 252     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 253     | vlan_id         | None                                 |
 254     | uses            | []                                   |
 255     | used_by         | []                                   |
 256     | created_at      | 2020-11-17T00:28:32.612230+00:00     |
 257     | updated_at      | 2020-11-17T01:03:47.341003+00:00     |
 258     | sriov_numvfs    | 0                                    |
 259     | sriov_vf_driver | None                                 |
 260     | accelerated     | [True]                               |
 261     +-----------------+--------------------------------------+
 262     [sysadmin@controller-0 ~(keystone_admin)]$
 263     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 oam0 -c platform
 264     +-----------------+--------------------------------------+
 265     | Property        | Value                                |
 266     +-----------------+--------------------------------------+
 267     | ifname          | oam0                                 |
 268     | iftype          | ethernet                             |
 269     | ports           | [u'eno3']                            |
 270     | imac            | 24:6e:96:5d:0c:b2                    |
 271     | imtu            | 1500                                 |
 272     | ifclass         | platform                             |
 273     | aemode          | None                                 |
 274     | schedpolicy     | None                                 |
 275     | txhashpolicy    | None                                 |
 276     | uuid            | d8a048fa-67ef-43ac-8166-671be93caa30 |
 277     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 278     | vlan_id         | None                                 |
 279     | uses            | []                                   |
 280     | used_by         | []                                   |
 281     | created_at      | 2020-11-17T00:28:32.365863+00:00     |
 282     | updated_at      | 2020-11-17T01:03:49.368879+00:00     |
 283     | sriov_numvfs    | 0                                    |
 284     | sriov_vf_driver | None                                 |
 285     | accelerated     | [True]                               |
 286     +-----------------+--------------------------------------+
 287     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 oam
 288     +--------------+--------------------------------------+
 289     | Property     | Value                                |
 290     +--------------+--------------------------------------+
 291     | hostname     | controller-0                         |
 292     | uuid         | 3c8bd181-d3f3-4e14-8e89-75a3432db1e4 |
 293     | ifname       | oam0                                 |
 294     | network_name | oam                                  |
 295     +--------------+--------------------------------------+
 296     [sysadmin@controller-0 ~(keystone_admin)]$
 297     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 pxeboot0 -c platform
 298     +-----------------+--------------------------------------+
 299     | Property        | Value                                |
 300     +-----------------+--------------------------------------+
 301     | ifname          | pxeboot0                             |
 302     | iftype          | ethernet                             |
 303     | ports           | [u'eno1']                            |
 304     | imac            | 24:6e:96:5d:0c:92                    |
 305     | imtu            | 1500                                 |
 306     | ifclass         | platform                             |
 307     | aemode          | None                                 |
 308     | schedpolicy     | None                                 |
 309     | txhashpolicy    | None                                 |
 310     | uuid            | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
 311     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 312     | vlan_id         | None                                 |
 313     | uses            | []                                   |
 314     | used_by         | []                                   |
 315     | created_at      | 2020-11-17T00:28:32.612230+00:00     |
 316     | updated_at      | 2020-11-17T01:03:53.143795+00:00     |
 317     | sriov_numvfs    | 0                                    |
 318     | sriov_vf_driver | None                                 |
 319     | accelerated     | [True]                               |
 320     +-----------------+--------------------------------------+
 321     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0  pxeboot0 pxeboot
 322     +--------------+--------------------------------------+
 323     | Property     | Value                                |
 324     +--------------+--------------------------------------+
 325     | hostname     | controller-0                         |
 326     | uuid         | 6c55622d-2da4-4f4e-ab5e-f8e06e03af7c |
 327     | ifname       | pxeboot0                             |
 328     | network_name | pxeboot                              |
 329     +--------------+--------------------------------------+
 330     [sysadmin@controller-0 ~(keystone_admin)]$
 331     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
 332     +-----------------+--------------------------------------+
 333     | Property        | Value                                |
 334     +-----------------+--------------------------------------+
 335     | ifname          | mgmt0                                |
 336     | iftype          | vlan                                 |
 337     | ports           | []                                   |
 338     | imac            | 24:6e:96:5d:0c:92                    |
 339     | imtu            | 1500                                 |
 340     | ifclass         | None                                 |
 341     | aemode          | None                                 |
 342     | schedpolicy     | None                                 |
 343     | txhashpolicy    | None                                 |
 344     | uuid            | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 |
 345     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 346     | vlan_id         | 18                                   |
 347     | uses            | [u'pxeboot0']                        |
 348     | used_by         | []                                   |
 349     | created_at      | 2020-11-17T01:03:57.303000+00:00     |
 350     | updated_at      | None                                 |
 351     | sriov_numvfs    | 0                                    |
 352     | sriov_vf_driver | None                                 |
 353     | accelerated     | [True]                               |
 354     +-----------------+--------------------------------------+
 355     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 mgmt0 mgmt
 356     +--------------+--------------------------------------+
 357     | Property     | Value                                |
 358     +--------------+--------------------------------------+
 359     | hostname     | controller-0                         |
 360     | uuid         | 2e93ef03-e9ee-457a-8667-05b52b7109a5 |
 361     | ifname       | mgmt0                                |
 362     | network_name | mgmt                                 |
 363     +--------------+--------------------------------------+
 364     [sysadmin@controller-0 ~(keystone_admin)]$
 365     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
 366     +-----------------+--------------------------------------+
 367     | Property        | Value                                |
 368     +-----------------+--------------------------------------+
 369     | ifname          | cluster0                             |
 370     | iftype          | vlan                                 |
 371     | ports           | []                                   |
 372     | imac            | 24:6e:96:5d:0c:92                    |
 373     | imtu            | 1500                                 |
 374     | ifclass         | None                                 |
 375     | aemode          | None                                 |
 376     | schedpolicy     | None                                 |
 377     | txhashpolicy    | None                                 |
 378     | uuid            | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 |
 379     | ihost_uuid      | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
 380     | vlan_id         | 19                                   |
 381     | uses            | [u'pxeboot0']                        |
 382     | used_by         | []                                   |
 383     | created_at      | 2020-11-17T01:04:02.613518+00:00     |
 384     | updated_at      | None                                 |
 385     | sriov_numvfs    | 0                                    |
 386     | sriov_vf_driver | None                                 |
 387     | accelerated     | [True]                               |
 388     +-----------------+--------------------------------------+
 389     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 cluster0 cluster-host
 390     +--------------+--------------------------------------+
 391     | Property     | Value                                |
 392     +--------------+--------------------------------------+
 393     | hostname     | controller-0                         |
 394     | uuid         | fb8b6be6-1618-4662-b063-b1e8d340aa48 |
 395     | ifname       | cluster0                             |
 396     | network_name | cluster-host                         |
 397     +--------------+--------------------------------------+
 398     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-0
 399     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 400     | uuid                                 | name     | class    | type     | vlan id | ports     | uses i/f      | used by i/f             | attributes |
 401     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 402     | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 | mgmt0    | platform | vlan     | 18      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 403     | 23b5e923-1e53-4e70-a975-542d8380b7f2 | pxeboot0 | platform | ethernet | None    | [u'eno1'] | []            | [u'mgmt0', u'cluster0'] | MTU=1500   |
 404     | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 | cluster0 | platform | vlan     | 19      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 405     | d8a048fa-67ef-43ac-8166-671be93caa30 | oam0     | platform | ethernet | None    | [u'eno3'] | []            | []                      | MTU=1500   |
 406     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 407
 408 Configure NTP Servers for network time synchronization:
 409
 410 ::
 411
 412     system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
 413
 414 Output
 415
 416 ::
 417
 418     [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
 419     +--------------+--------------------------------------+
 420     | Property     | Value                                |
 421     +--------------+--------------------------------------+
 422     | uuid         | 3206cf01-c64a-457e-ac66-b8224c9684c3 |
 423     | ntpservers   | 0.pool.ntp.org,1.pool.ntp.org        |
 424     | isystem_uuid | cc79b616-d24e-4432-a953-85c9b242cb3a |
 425     | created_at   | 2020-11-17T00:27:23.529571+00:00     |
 426     | updated_at   | None                                 |
 427     +--------------+--------------------------------------+
 428
 429 Add an OSD on controller-0 for Ceph:
 430
 431 ::
 432
 433     system host-disk-list controller-0
 434     system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
 435     system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
 436     system host-stor-list controller-0
 437
 438 Output
 439
 440 ::
 441
 442     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
 443     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 444     | uuid                                 | device_no | device_ | device_ | size_ | available_ | rpm          | serial_id                        | device_path                                     |
 445     |                                      | de        | num     | type    | gib   | gib        |              |                                  |                                                 |
 446     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 447     | 8e2a719a-fa5a-4c25-89af-70a23fb7b238 | /dev/sda  | 2048    | HDD     | 893.  | 644.726    | Undetermined | 00c66a07604fa8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 448     |                                      |           |         |         | 75    |            |              |                                  |                                                 |
 449     |                                      |           |         |         |       |            |              |                                  |                                                 |
 450     | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/sdb  | 2064    | HDD     | 1117. | 1117.247   | Undetermined | 00c6b9139b76a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
 451     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 452     |                                      |           |         |         |       |            |              |                                  |                                                 |
 453     | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/sdc  | 2080    | HDD     | 1117. | 1117.247   | Undetermined | 0053be63c794a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
 454     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 455     |                                      |           |         |         |       |            |              |                                  |                                                 |
 456     | 4879b381-8e9f-48f3-84e2-f9c6a94bbfe0 | /dev/sdd  | 2096    | HDD     | 1117. | 0.0        | Undetermined | 0065482503bca8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
 457     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 458     |                                      |           |         |         |       |            |              |                                  |                                                 |
 459     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 460     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
 461     +------------------+-------------------------------------------------------+
 462     | Property         | Value                                                 |
 463     +------------------+-------------------------------------------------------+
 464     | osdid            | 0                                                     |
 465     | function         | osd                                                   |
 466     | state            | configuring-on-unlock                                 |
 467     | journal_location | 0816c72f-a4f0-49ea-9a95-0f02c880717c                  |
 468     | journal_size_gib | 1024                                                  |
 469     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
 470     | journal_node     | /dev/sdb2                                             |
 471     | uuid             | 0816c72f-a4f0-49ea-9a95-0f02c880717c                  |
 472     | ihost_uuid       | 16afe3a2-ba50-46b8-9fd7-09010059e8b9                  |
 473     | idisk_uuid       | 61b6f262-a51f-4310-aeac-373b1c1bbbc2                  |
 474     | tier_uuid        | 3af8c893-9dd4-40af-afc6-30bb79048448                  |
 475     | tier_name        | storage                                               |
 476     | created_at       | 2020-11-17T01:05:04.063823+00:00                      |
 477     | updated_at       | None                                                  |
 478     +------------------+-------------------------------------------------------+
 479     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
 480     +------------------+-------------------------------------------------------+
 481     | Property         | Value                                                 |
 482     +------------------+-------------------------------------------------------+
 483     | osdid            | 1                                                     |
 484     | function         | osd                                                   |
 485     | state            | configuring-on-unlock                                 |
 486     | journal_location | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5                  |
 487     | journal_size_gib | 1024                                                  |
 488     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
 489     | journal_node     | /dev/sdc2                                             |
 490     | uuid             | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5                  |
 491     | ihost_uuid       | 16afe3a2-ba50-46b8-9fd7-09010059e8b9                  |
 492     | idisk_uuid       | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79                  |
 493     | tier_uuid        | 3af8c893-9dd4-40af-afc6-30bb79048448                  |
 494     | tier_name        | storage                                               |
 495     | created_at       | 2020-11-17T01:05:06.939798+00:00                      |
 496     | updated_at       | None                                                  |
 497     +------------------+-------------------------------------------------------+
 498     [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
 499     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 500     | uuid                                 | function | osdid | state                 | idisk_uuid                           | journal_path                                          | journal_node | journal_size_gib | tier_name |
 501     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 502     | 0816c72f-a4f0-49ea-9a95-0f02c880717c | osd      | 0     | configuring-on-unlock | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2    | 1                | storage   |
 503     | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 | osd      | 1     | configuring-on-unlock | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2    | 1                | storage   |
 504     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 505
 506 2.7 Unlock controller-0
 507 ~~~~~~~~~~~~~~~~~~~~~~~
 508
 509 ::
 510
 511     system host-unlock controller-0
 512
 513 Output:
 514
 515 ::
 516
 517     [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
 518     +-----------------------+-------------------------------------------------+
 519     | Property              | Value                                           |
 520     +-----------------------+-------------------------------------------------+
 521     | action                | none                                            |
 522     | administrative        | locked                                          |
 523     | availability          | online                                          |
 524     | bm_ip                 | None                                            |
 525     | bm_type               | none                                            |
 526     | bm_username           | None                                            |
 527     | boot_device           | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 528     | capabilities          | {u'stor_function': u'monitor'}                  |
 529     | clock_synchronization | ntp                                             |
 530     | config_applied        | 6aa15fb4-8cb3-494e-b94e-95f85b560f22            |
 531     | config_status         | None                                            |
 532     | config_target         | c6ae9b2d-a3c4-4751-a79e-5487ba81ed82            |
 533     | console               | ttyS0,115200                                    |
 534     | created_at            | 2020-11-17T00:28:01.983673+00:00                |
 535     | hostname              | controller-0                                    |
 536     | id                    | 1                                               |
 537     | install_output        | graphical                                       |
 538     | install_state         | None                                            |
 539     | install_state_info    | None                                            |
 540     | inv_state             | inventoried                                     |
 541     | invprovision          | provisioning                                    |
 542     | location              | {}                                              |
 543     | mgmt_ip               | 192.168.18.3                                    |
 544     | mgmt_mac              | 24:6e:96:5d:0c:92                               |
 545     | operational           | disabled                                        |
 546     | personality           | controller                                      |
 547     | reserved              | False                                           |
 548     | rootfs_device         | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 549     | serialid              | None                                            |
 550     | software_load         | 19.12                                           |
 551     | subfunction_avail     | online                                          |
 552     | subfunction_oper      | disabled                                        |
 553     | subfunctions          | controller,worker,lowlatency                    |
 554     | task                  | Unlocking                                       |
 555     | tboot                 | false                                           |
 556     | ttys_dcd              | None                                            |
 557     | updated_at            | 2020-11-17T01:05:07.015414+00:00                |
 558     | uptime                | 3496                                            |
 559     | uuid                  | 16afe3a2-ba50-46b8-9fd7-09010059e8b9            |
 560     | vim_progress_status   | None                                            |
 561     +-----------------------+-------------------------------------------------+
 562
 563 Controller-0 will reboot to apply configuration changes and come into
 564 service. This can take 5-10 minutes, depending on the performance of the
 565 host machine.
 566
 567 Once the controller comes back up, check the status of controller-0
 568
 569 ::
 570
 571     controller-0:~$ source /etc/platform/openrc
 572     [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
 573     +----+--------------+-------------+----------------+-------------+--------------+
 574     | id | hostname     | personality | administrative | operational | availability |
 575     +----+--------------+-------------+----------------+-------------+--------------+
 576     | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
 577     +----+--------------+-------------+----------------+-------------+--------------+
 578
 579 2. Installation for the second server (controller-1)
 580 ----------------------------------------------------
 581
 582 2.1 Power on the controller-1 server and force it to network boot
 583 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 584
 585 2.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
 586 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 587
 588 2.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
 589 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 590
 591 ::
 592
 593     [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
 594     +----+--------------+-------------+----------------+-------------+--------------+
 595     | id | hostname     | personality | administrative | operational | availability |
 596     +----+--------------+-------------+----------------+-------------+--------------+
 597     | 1  | controller-0 | controller  | unlocked       | enabled     | degraded     |
 598     | 2  | None         | None        | locked         | disabled    | offline      |
 599     +----+--------------+-------------+----------------+-------------+--------------+
 600
 601 2.4 Using the host id, set the personality of this host to 'controller’:
 602 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 603
 604 ::
 605
 606     [sysadmin@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller
 607     +-----------------------+--------------------------------------+
 608     | Property              | Value                                |
 609     +-----------------------+--------------------------------------+
 610     | action                | none                                 |
 611     | administrative        | locked                               |
 612     | availability          | offline                              |
 613     | bm_ip                 | None                                 |
 614     | bm_type               | None                                 |
 615     | bm_username           | None                                 |
 616     | boot_device           | /dev/sda                             |
 617     | capabilities          | {}                                   |
 618     | clock_synchronization | ntp                                  |
 619     | config_applied        | None                                 |
 620     | config_status         | None                                 |
 621     | config_target         | None                                 |
 622     | console               | ttyS0,115200                         |
 623     | created_at            | 2020-11-17T10:17:44.387813+00:00     |
 624     | hostname              | controller-1                         |
 625     | id                    | 2                                    |
 626     | install_output        | text                                 |
 627     | install_state         | None                                 |
 628     | install_state_info    | None                                 |
 629     | inv_state             | None                                 |
 630     | invprovision          | None                                 |
 631     | location              | {}                                   |
 632     | mgmt_ip               | 192.168.18.4                         |
 633     | mgmt_mac              | 24:6e:96:5d:38:ee                    |
 634     | operational           | disabled                             |
 635     | personality           | controller                           |
 636     | reserved              | False                                |
 637     | rootfs_device         | /dev/sda                             |
 638     | serialid              | None                                 |
 639     | software_load         | 19.12                                |
 640     | subfunction_avail     | not-installed                        |
 641     | subfunction_oper      | disabled                             |
 642     | subfunctions          | controller,worker,lowlatency         |
 643     | task                  | None                                 |
 644     | tboot                 | false                                |
 645     | ttys_dcd              | None                                 |
 646     | updated_at            | None                                 |
 647     | uptime                | 0                                    |
 648     | uuid                  | f069381d-9743-49cc-bf8b-eb4bd3972203 |
 649     | vim_progress_status   | None                                 |
 650     +-----------------------+--------------------------------------+
 651
 652 2.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
 653 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 654
 655 This can take 5-10 minutes, depending on the performance of the host
 656 machine.
 657
 658 ::
 659
 660     [root@controller-0 hieradata(keystone_admin)]$ system host-list
 661     +----+--------------+-------------+----------------+-------------+--------------+
 662     | id | hostname     | personality | administrative | operational | availability |
 663     +----+--------------+-------------+----------------+-------------+--------------+
 664     | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
 665     | 2  | controller-1 | controller  | locked         | disabled    | online       |
 666     +----+--------------+-------------+----------------+-------------+--------------+
 667
 668 2.6 Configure controller-1
 669 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 670
 671 ::
 672
 673     OAM_IF=eno3
 674     MGMT_IF=eno1
 675     system host-if-modify controller-1 $OAM_IF -n oam0
 676     system host-if-modify controller-1 oam0 -c platform
 677     system interface-network-assign controller-1 oam0 oam
 678
 679     system host-if-add         -V 19 controller-1 cluster0 vlan pxeboot0
 680     system interface-network-assign controller-1 cluster0 cluster-host
 681
 682     system host-if-list controller-1
 683
 684     system host-disk-list controller-1
 685     system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
 686     system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
 687     system host-stor-list controller-1
 688
 689 Output:
 690
 691 ::
 692
 693     [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
 694     [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
 695     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 $OAM_IF -n oam0
 696     +-----------------+--------------------------------------+
 697     | Property        | Value                                |
 698     +-----------------+--------------------------------------+
 699     | ifname          | oam0                                 |
 700     | iftype          | ethernet                             |
 701     | ports           | [u'eno3']                            |
 702     | imac            | 24:6e:96:5d:39:0e                    |
 703     | imtu            | 1500                                 |
 704     | ifclass         | None                                 |
 705     | aemode          | None                                 |
 706     | schedpolicy     | None                                 |
 707     | txhashpolicy    | None                                 |
 708     | uuid            | c2473511-d0d6-445d-9739-4d43dc029de9 |
 709     | ihost_uuid      | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 710     | vlan_id         | None                                 |
 711     | uses            | []                                   |
 712     | used_by         | []                                   |
 713     | created_at      | 2020-11-22T14:01:32.365863+00:00     |
 714     | updated_at      | 2020-11-22T15:04:45.090904+00:00     |
 715     | sriov_numvfs    | 0                                    |
 716     | sriov_vf_driver | None                                 |
 717     | accelerated     | [True]                               |
 718     +-----------------+--------------------------------------+
 719     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 oam0 -c platform
 720     +-----------------+--------------------------------------+
 721     | Property        | Value                                |
 722     +-----------------+--------------------------------------+
 723     | ifname          | oam0                                 |
 724     | iftype          | ethernet                             |
 725     | ports           | [u'eno3']                            |
 726     | imac            | 24:6e:96:5d:39:0e                    |
 727     | imtu            | 1500                                 |
 728     | ifclass         | platform                             |
 729     | aemode          | None                                 |
 730     | schedpolicy     | None                                 |
 731     | txhashpolicy    | None                                 |
 732     | uuid            | c2473511-d0d6-445d-9739-4d43dc029de9 |
 733     | ihost_uuid      | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 734     | vlan_id         | None                                 |
 735     | uses            | []                                   |
 736     | used_by         | []                                   |
 737     | created_at      | 2020-11-22T14:05:16.052229+00:00     |
 738     | updated_at      | 2020-11-22T15:08:35.324634+00:00     |
 739     | sriov_numvfs    | 0                                    |
 740     | sriov_vf_driver | None                                 |
 741     | accelerated     | [True]                               |
 742     +-----------------+--------------------------------------+
 743     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 oam0 oam
 744     +--------------+--------------------------------------+
 745     | Property     | Value                                |
 746     +--------------+--------------------------------------+
 747     | hostname     | controller-1                         |
 748     | uuid         | f2e7f088-0dd0-4adc-8348-4e3cef23bc47 |
 749     | ifname       | oam0                                 |
 750     | network_name | oam                                  |
 751     +--------------+--------------------------------------+
 752     [sysadmin@controller-0 ~(keystone_admin)]$
 753
 754     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
 755     +-----------------+--------------------------------------+
 756     | Property        | Value                                |
 757     +-----------------+--------------------------------------+
 758     | ifname          | cluster0                             |
 759     | iftype          | vlan                                 |
 760     | ports           | []                                   |
 761     | imac            | 24:6e:96:5d:38:ee                    |
 762     | imtu            | 1500                                 |
 763     | ifclass         | None                                 |
 764     | aemode          | None                                 |
 765     | schedpolicy     | None                                 |
 766     | txhashpolicy    | None                                 |
 767     | uuid            | b6783682-b2aa-4135-90d2-676e1db41ae8 |
 768     | ihost_uuid      | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 769     | vlan_id         | 19                                   |
 770     | uses            | [u'pxeboot0']                        |
 771     | used_by         | []                                   |
 772     | created_at      | 2020-11-22T15:08:43.932209+00:00     |
 773     | updated_at      | None                                 |
 774     | sriov_numvfs    | 0                                    |
 775     | sriov_vf_driver | None                                 |
 776     | accelerated     | [True]                               |
 777     +-----------------+--------------------------------------+
 778
 779     [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 cluster0 cluster-host
 780     +--------------+--------------------------------------+
 781     | Property     | Value                                |
 782     +--------------+--------------------------------------+
 783     | hostname     | controller-1                         |
 784     | uuid         | 8fc64805-b54b-45a4-b88a-e13b236abfe8 |
 785     | ifname       | cluster0                             |
 786     | network_name | cluster-host                         |
 787     +--------------+--------------------------------------+
 788     [sysadmin@controller-0 ~(keystone_admin)]$
 789     [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-1
 790     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 791     | uuid                                 | name     | class    | type     | vlan id | ports     | uses i/f      | used by i/f             | attributes |
 792     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 793     | b6783682-b2aa-4135-90d2-676e1db41ae8 | cluster0 | platform | vlan     | 19      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 794     | b8921960-fde5-44c3-960d-2aebf42ea400 | pxeboot0 | platform | ethernet | None    | [u'eno1'] | []            | [u'mgmt0', u'cluster0'] | MTU=1500   |
 795     | c103275b-2b75-4568-865f-ac6be32ecb2d | mgmt0    | platform | vlan     | 18      | []        | [u'pxeboot0'] | []                      | MTU=1500   |
 796     | c2473511-d0d6-445d-9739-4d43dc029de9 | oam0     | platform | ethernet | None    | [u'eno3'] | []            | []                      | MTU=1500   |
 797     +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
 798     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
 799     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 800     | uuid                                 | device_no | device_ | device_ | size_ | available_ | rpm          | serial_id                        | device_path                                     |
 801     |                                      | de        | num     | type    | gib   | gib        |              |                                  |                                                 |
 802     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 803     | 5b8fade4-b048-48fa-b906-9dcbdbed8e96 | /dev/sda  | 2048    | HDD     | 893.  | 644.726    | Undetermined | 00cbd97f3e36ccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
 804     |                                      |           |         |         | 75    |            |              |                                  |                                                 |
 805     |                                      |           |         |         |       |            |              |                                  |                                                 |
 806     | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/sdb  | 2064    | HDD     | 1117. | 1117.247   | Undetermined | 006d0e977b5fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
 807     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 808     |                                      |           |         |         |       |            |              |                                  |                                                 |
 809     | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/sdc  | 2080    | HDD     | 1117. | 1117.247   | Undetermined | 003a2377ac7fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
 810     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 811     |                                      |           |         |         |       |            |              |                                  |                                                 |
 812     | 774c3cd0-1178-4145-9573-f0d6dee2ba06 | /dev/sdd  | 2096    | HDD     | 1117. | 1117.247   | Undetermined | 00d7093ef0adccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
 813     |                                      |           |         |         | 25    |            |              |                                  |                                                 |
 814     |                                      |           |         |         |       |            |              |                                  |                                                 |
 815     | 00361302-8d55-4730-855c-b0098c73ab7e | /dev/sde  | 2112    | SSD     | 223.  | 223.568    | N/A          | PHDW730104QM240E                 | /dev/disk/by-path/pci-0000:d8:00.0-ata-1        |
 816     |                                      |           |         |         | 57    |            |              |                                  |                                                 |
 817     |                                      |           |         |         |       |            |              |                                  |                                                 |
 818     | 7ce735e6-920f-4424-a890-a7a7f48d7632 | /dev/sdf  | 2128    | SSD     | 223.  | 223.568    | N/A          | PHDW730104LL240E                 | /dev/disk/by-path/pci-0000:d8:00.0-ata-2        |
 819     |                                      |           |         |         | 57    |            |              |                                  |                                                 |
 820     |                                      |           |         |         |       |            |              |                                  |                                                 |
 821     +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
 822     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
 823     +------------------+-------------------------------------------------------+
 824     | Property         | Value                                                 |
 825     +------------------+-------------------------------------------------------+
 826     | osdid            | 2                                                     |
 827     | function         | osd                                                   |
 828     | state            | configuring-on-unlock                                 |
 829     | journal_location | 54a218d8-0466-4366-9ef0-3ec5a952fde7                  |
 830     | journal_size_gib | 1024                                                  |
 831     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
 832     | journal_node     | /dev/sdb2                                             |
 833     | uuid             | 54a218d8-0466-4366-9ef0-3ec5a952fde7                  |
 834     | ihost_uuid       | 63c930c7-2195-4d5a-870c-be610fd6b4fc                  |
 835     | idisk_uuid       | 1a3f0a36-5961-42e5-a271-e71db1c25d42                  |
 836     | tier_uuid        | 06b4740e-29db-4896-9600-03ee40fe0d6c                  |
 837     | tier_name        | storage                                               |
 838     | created_at       | 2020-11-22T15:11:55.641193+00:00                      |
 839     | updated_at       | None                                                  |
 840     +------------------+-------------------------------------------------------+
 841
 842     [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
 843     +------------------+-------------------------------------------------------+
 844     | Property         | Value                                                 |
 845     +------------------+-------------------------------------------------------+
 846     | osdid            | 3                                                     |
 847     | function         | osd                                                   |
 848     | state            | configuring-on-unlock                                 |
 849     | journal_location | 5be88c7a-3a94-4b97-9da5-b247bb89406c                  |
 850     | journal_size_gib | 1024                                                  |
 851     | journal_path     | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
 852     | journal_node     | /dev/sdc2                                             |
 853     | uuid             | 5be88c7a-3a94-4b97-9da5-b247bb89406c                  |
 854     | ihost_uuid       | 63c930c7-2195-4d5a-870c-be610fd6b4fc                  |
 855     | idisk_uuid       | eddd732f-2cea-49b3-86db-b722c0b1a1ae                  |
 856     | tier_uuid        | 06b4740e-29db-4896-9600-03ee40fe0d6c                  |
 857     | tier_name        | storage                                               |
 858     | created_at       | 2020-11-22T15:12:04.274839+00:00                      |
 859     | updated_at       | None                                                  |
 860     +------------------+-------------------------------------------------------+
 861
 862     [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-1
 863     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 864     | uuid                                 | function | osdid | state                 | idisk_uuid                           | journal_path                                          | journal_node | journal_size_gib | tier_name |
 865     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 866     | 54a218d8-0466-4366-9ef0-3ec5a952fde7 | osd      | 2     | configuring-on-unlock | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2    | 1                | storage   |
 867     | 5be88c7a-3a94-4b97-9da5-b247bb89406c | osd      | 3     | configuring-on-unlock | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2    | 1                | storage   |
 868     +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 869
 870 2.7 Unlock controller-1
 871 ~~~~~~~~~~~~~~~~~~~~~~~
 872
 873 Unlock controller-1 in order to bring it into service:
 874
 875 ::
 876
 877     [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-1
 878     +-----------------------+--------------------------------------+
 879     | Property              | Value                                |
 880     +-----------------------+--------------------------------------+
 881     | action                | none                                 |
 882     | administrative        | locked                               |
 883     | availability          | online                               |
 884     | bm_ip                 | None                                 |
 885     | bm_type               | None                                 |
 886     | bm_username           | None                                 |
 887     | boot_device           | /dev/sda                             |
 888     | capabilities          | {u'stor_function': u'monitor'}       |
 889     | clock_synchronization | ntp                                  |
 890     | config_applied        | None                                 |
 891     | config_status         | Config out-of-date                   |
 892     | config_target         | 9747e0ce-2319-409d-b75c-2475bc5065ac |
 893     | console               | ttyS0,115200                         |
 894     | created_at            | 2020-11-22T12:58:11.630526+00:00     |
 895     | hostname              | controller-1                         |
 896     | id                    | 3                                    |
 897     | install_output        | text                                 |
 898     | install_state         | None                                 |
 899     | install_state_info    | None                                 |
 900     | inv_state             | inventoried                          |
 901     | invprovision          | unprovisioned                        |
 902     | location              | {}                                   |
 903     | mgmt_ip               | 192.168.18.4                         |
 904     | mgmt_mac              | 24:6e:96:5d:38:ee                    |
 905     | operational           | disabled                             |
 906     | personality           | controller                           |
 907     | reserved              | False                                |
 908     | rootfs_device         | /dev/sda                             |
 909     | serialid              | None                                 |
 910     | software_load         | 19.12                                |
 911     | subfunction_avail     | online                               |
 912     | subfunction_oper      | disabled                             |
 913     | subfunctions          | controller,worker,lowlatency         |
 914     | task                  | Unlocking                            |
 915     | tboot                 | false                                |
 916     | ttys_dcd              | None                                 |
 917     | updated_at            | 2020-11-22T15:13:09.716324+00:00     |
 918     | uptime                | 752                                  |
 919     | uuid                  | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
 920     | vim_progress_status   | None                                 |
 921     +-----------------------+--------------------------------------+
 922
 923 Controller-1 will reboot in order to apply configuration changes and
 924 come into service. This can take 5-10 minutes, depending on the
 925 performance of the host machine.
 926
 927 ::
 928
 929     [root@controller-0 hieradata(keystone_admin)]$ system host-list
 930     +----+--------------+-------------+----------------+-------------+--------------+
 931     | id | hostname     | personality | administrative | operational | availability |
 932     +----+--------------+-------------+----------------+-------------+--------------+
 933     | 1  | controller-0 | controller  | unlocked       | enabled     | available    |
 934     | 2  | controller-1 | controller  | unlocked       | enabled     | available    |
 935     +----+--------------+-------------+----------------+-------------+--------------+
 936
 937     [sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-1
 938     +-----------------------+-----------------------------------------------------------------------+
 939     | Property              | Value                                                                 |
 940     +-----------------------+-----------------------------------------------------------------------+
 941     | action                | none                                                                  |
 942     | administrative        | unlocked                                                              |
 943     | availability          | available                                                             |
 944     | bm_ip                 | None                                                                  |
 945     | bm_type               | None                                                                  |
 946     | bm_username           | None                                                                  |
 947     | boot_device           | /dev/sda                                                              |
 948     | capabilities          | {u'stor_function': u'monitor', u'Personality': u'Controller-Standby'} |
 949     | clock_synchronization | ntp                                                                   |
 950     | config_applied        | 9747e0ce-2319-409d-b75c-2475bc5065ac                                  |
 951     | config_status         | None                                                                  |
 952     | config_target         | 9747e0ce-2319-409d-b75c-2475bc5065ac                                  |
 953     | console               | ttyS0,115200                                                          |
 954     | created_at            | 2020-11-22T12:58:11.630526+00:00                                      |
 955     | hostname              | controller-1                                                          |
 956     | id                    | 2                                                                     |
 957     | install_output        | text                                                                  |
 958     | install_state         | None                                                                  |
 959     | install_state_info    | None                                                                  |
 960     | inv_state             | inventoried                                                           |
 961     | invprovision          | provisioned                                                           |
 962     | location              | {}                                                                    |
 963     | mgmt_ip               | 192.168.18.4                                                          |
 964     | mgmt_mac              | 24:6e:96:5d:38:ee                                                     |
 965     | operational           | enabled                                                               |
 966     | personality           | controller                                                            |
 967     | reserved              | False                                                                 |
 968     | rootfs_device         | /dev/sda                                                              |
 969     | serialid              | None                                                                  |
 970     | software_load         | 19.12                                                                 |
 971     | subfunction_avail     | available                                                             |
 972     | subfunction_oper      | enabled                                                               |
 973     | subfunctions          | controller,worker,lowlatency                                          |
 974     | task                  |                                                                       |
 975     | tboot                 | false                                                                 |
 976     | ttys_dcd              | None                                                                  |
 977     | updated_at            | 2020-11-22T23:59:07.787759+00:00                                      |
 978     | uptime                | 31008                                                                 |
 979     | uuid                  | 63c930c7-2195-4d5a-870c-be610fd6b4fc                                  |
 980     | vim_progress_status   | services-enabled                                                      |
 981     +-----------------------+-----------------------------------------------------------------------+
 982
 983
 984 3. Simple use case for sriov
 985 ````````````````````````````
 986
 987 3.1 After controller-0 is rebooted and up running, download the DPDK
 988 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 989
 990 ::
 991
 992   [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
 993   [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
 994   Password:
 995   --2020-06-04 02:35:30--  https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
 996   Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
 997   Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
 998
 999   HTTP request sent, awaiting response... 200 OK
1000   Length: 10251680 (9.8M) [application/octet-stream]
1001   Saving to: ‘dpdk-17.11.10.tar.xz’
1002
1003   dpdk-17.11.10.tar.xz                        100%
1004   [========================================================================================>]   9.78M
1005   1.48MB/s    in 6.8s
1006
1007   2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
1008
1009   sudo tar xvf dpdk-17.11.10.tar.xz
1010
1011   sudo ln -s dpdk-stable-17.11.10 dpdk-stable
1012
1013 3.2 Prepare the yaml file for the network assignment container
1014 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
1015
1016 The following the exmaple of the yaml file:
1017
1018 ::
1019
1020   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
1021   > apiVersion: "k8s.cni.cncf.io/v1"
1022   > kind: NetworkAttachmentDefinition
1023   > metadata:
1024   >   name: sriov-data-dpdk-0
1025   >   annotations:
1026   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
1027   > spec:
1028   >   config: '{
1029   >   "type": "sriov",
1030   >   "name": "sriov-data-dpdk-0"
1031   > }'
1032   >
1033   > ---
1034   > apiVersion: "k8s.cni.cncf.io/v1"
1035   > kind: NetworkAttachmentDefinition
1036   > metadata:
1037   >   name: sriov-data-dpdk-1
1038   >   annotations:
1039   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
1040   > spec:
1041   >   config: '{
1042   >   "type": "sriov",
1043   >   "name": "sriov-data-dpdk-1"
1044   > }'
1045   > EOF
1046
1047 3.3 Run the network assignent container for the 2 VFs
1048 '''''''''''''''''''''''''''''''''''''''''''''''''''''
1049
1050 ::
1051
1052   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
1053   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
1054   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
1055
1056 3.4 Prepare the VF container yaml file
1057 ''''''''''''''''''''''''''''''''''''''
1058
1059 ::
1060
1061   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
1062   > apiVersion: v1
1063   > kind: Pod
1064   metadata:
1065   > metadata:
1066   >   name: pod-with-dpdk-vfs-0
1067   >   annotations:
1068   >     k8s.v1.cni.cncf.io/networks: '[
1069   >             { "name": "sriov-data-dpdk-0" },
1070               { "name": "sriov-data-dpdk-1" }
1071   >             { "name": "sriov-data-dpdk-1" }
1072   >     ]'
1073   > spec:
1074   >   restartPolicy: Never
1075   >   containers:
1076   >   - name: pod-with-dpdk-vfs-0
1077   >     image: wrsnfv/ubuntu-dpdk-build:v0.3
1078   >     env:
1079   >     - name: RTE_SDK
1080   >       value: "/usr/src/dpdk"
1081   >     command:
1082   >     - sleep
1083   >     - infinity
1084   >     stdin: true
1085   >     tty: true
1086   >     securityContext:
1087   >       privileged: true
1088   >       capabilities:
1089   >         add:
1090   >         - ALL
1091   >     resources:
1092   >       requests:
1093   >         cpu: 4
1094   >         memory: 4Gi
1095   >         intel.com/pci_sriov_net_physnet0: '1'
1096   >         intel.com/pci_sriov_net_physnet1: '1'
1097   >       limits:
1098   >         cpu: 4
1099   >         hugepages-1Gi: 2Gi
1100   >         memory: 4Gi
1101   >         intel.com/pci_sriov_net_physnet0: '1'
1102   >         intel.com/pci_sriov_net_physnet1: '1'
1103   >     volumeMounts:
1104   >     - mountPath: /mnt/huge-1048576kB
1105   >       name: hugepage
1106   >     - name: dpdk-volume
1107   >       mountPath: /usr/src/dpdk
1108   >     - name: lib-volume
1109   >       mountPath: /lib/modules
1110   >     - name: src-volume
1111   >       mountPath: /usr/src/
1112   >   volumes:
1113   >   - name: hugepage
1114   >     emptyDir:
1115   >       medium: HugePages
1116   >   - name: dpdk-volume
1117   >     hostPath:
1118   >       path: /opt/dpdk-stable/
1119   >   - name: lib-volume
1120   >     hostPath:
1121   >       path: /lib/modules
1122   >   - name: src-volume
1123   >     hostPath:
1124   >       path: /usr/src/
1125   > EOF
1126
1127 3.5 Run the VF container
1128 ''''''''''''''''''''''''
1129
1130 Start the VF container.
1131
1132 ::
1133
1134   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
1135   pod/pod-with-dpdk-vfs-0 created
1136
1137   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
1138   NAME                  READY   STATUS    RESTARTS   AGE
1139   pod-with-dpdk-vfs-0   1/1     Running   0          6m40s
1140
1141 Login the VF container
1142
1143 ::
1144
1145   kubectl exec -it pod-with-dpdk-vfs-0 -- bash
1146
1147 Build the DPDK
1148
1149 ::
1150
1151   cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
1152
1153   root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
1154     HOSTCC  scripts/basic/fixdep
1155     HOSTCC  scripts/kconfig/conf.o
1156     HOSTCC  scripts/kconfig/confdata.o
1157     HOSTCC  scripts/kconfig/expr.o
1158     HOSTCC  scripts/kconfig/symbol.o
1159     HOSTCC  scripts/kconfig/preprocess.o
1160     HOSTCC  scripts/kconfig/zconf.lex.o
1161     HOSTCC  scripts/kconfig/zconf.tab.o
1162     HOSTLD  scripts/kconfig/conf
1163   scripts/kconfig/conf  --syncconfig Kconfig
1164     HOSTCC  arch/x86/tools/relocs_32.o
1165     HOSTCC  arch/x86/tools/relocs_64.o
1166     HOSTCC  arch/x86/tools/relocs_common.o
1167     HOSTLD  arch/x86/tools/relocs
1168     HOSTCC  scripts/genksyms/genksyms.o
1169     YACC    scripts/genksyms/parse.tab.c
1170     HOSTCC  scripts/genksyms/parse.tab.o
1171     LEX     scripts/genksyms/lex.lex.c
1172     YACC    scripts/genksyms/parse.tab.h
1173     HOSTCC  scripts/genksyms/lex.lex.o
1174     HOSTLD  scripts/genksyms/genksyms
1175     HOSTCC  scripts/bin2c
1176     HOSTCC  scripts/kallsyms
1177     HOSTCC  scripts/conmakehash
1178     HOSTCC  scripts/recordmcount
1179     HOSTCC  scripts/sortextable
1180     HOSTCC  scripts/asn1_compiler
1181     HOSTCC  scripts/sign-file
1182     HOSTCC  scripts/extract-cert
1183     CC      scripts/mod/empty.o
1184     HOSTCC  scripts/mod/mk_elfconfig
1185     MKELF   scripts/mod/elfconfig.h
1186     HOSTCC  scripts/mod/modpost.o
1187     CC      scripts/mod/devicetable-offsets.s
1188     UPD     scripts/mod/devicetable-offsets.h
1189     HOSTCC  scripts/mod/file2alias.o
1190     HOSTCC  scripts/mod/sumversion.o
1191     HOSTLD  scripts/mod/modpost
1192     CC      kernel/bounds.s
1193     CC      arch/x86/kernel/asm-offsets.s
1194     CALL    scripts/checksyscalls.sh
1195
1196 Build the test_pmd application
1197
1198 ::
1199
1200   cd $RTE_SDK
1201   ./usertools/dpdk-setup.sh
1202   Option: 14
1203     CC config.o
1204     CC iofwd.o
1205     CC macfwd.o
1206     CC macswap.o
1207     CC flowgen.o
1208     CC rxonly.o
1209     CC txonly.o
1210     CC csumonly.o
1211     CC icmpecho.o
1212     CC tm.o
1213     LD testpmd
1214     INSTALL-APP testpmd
1215     INSTALL-MAP testpmd.map
1216   == Build app/proc_info
1217     CC main.o
1218     LD dpdk-procinfo
1219     INSTALL-APP dpdk-procinfo
1220     INSTALL-MAP dpdk-procinfo.map
1221   == Build app/pdump
1222     CC main.o
1223     LD dpdk-pdump
1224     INSTALL-APP dpdk-pdump
1225     INSTALL-MAP dpdk-pdump.map
1226   == Build app/test-crypto-perf
1227     CC main.o
1228     CC cperf_ops.o
1229     CC cperf_options_parsing.o
1230     CC cperf_test_vectors.o
1231     CC cperf_test_throughput.o
1232     CC cperf_test_latency.o
1233     CC cperf_test_pmd_cyclecount.o
1234     CC cperf_test_verify.o
1235     CC cperf_test_vector_parsing.o
1236     CC cperf_test_common.o
1237     LD dpdk-test-crypto-perf
1238     INSTALL-APP dpdk-test-crypto-perf
1239     INSTALL-MAP dpdk-test-crypto-perf.map
1240   == Build app/test-eventdev
1241     CC evt_main.o
1242     CC evt_options.o
1243     CC evt_test.o
1244     CC parser.o
1245     CC test_order_common.o
1246     CC test_order_queue.o
1247     CC test_order_atq.o
1248     CC test_perf_common.o
1249     CC test_perf_queue.o
1250     CC test_perf_atq.o
1251     LD dpdk-test-eventdev
1252     INSTALL-APP dpdk-test-eventdev
1253     INSTALL-MAP dpdk-test-eventdev.map
1254   Build complete [x86_64-native-linuxapp-gcc]
1255   Installation cannot run with T defined and DESTDIR undefined
1256   ------------------------------------------------------------------------------
1257   RTE_TARGET exported as x86_64-native-linuxapp-gcc
1258   ------------------------------------------------------------------------------
1259
1260   Press enter to continue ...
1261
1262 Check the VF PCI information:
1263
1264 ::
1265
1266   root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
1267   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
1268   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
1269
1270 Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
1271
1272 ::
1273
1274   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
1275   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
1276   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
1277   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
1278   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
1279   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
1280   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
1281
1282   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
1283   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
1284   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
1285   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
1286   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
1287   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
1288   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
1289
1290   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
1291   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
1292
1293
1294 Get back to pod and run the test_pmd
1295
1296 ::
1297
1298   cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
1299
1300   taskset -p --cpu-list 1
1301   pid 1's current affinity list: 2,3,18,19
1302
1303   ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
1304   start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
1305   peer=1,"9e:fd:e6:dd:c1:01"
1306   EAL: Detected 32 lcore(s)
1307   EAL: No free hugepages reported in hugepages-2048kB
1308   EAL: Probing VFIO support...
1309   EAL: VFIO support initialized
1310   EAL: PCI device 0000:05:11.0 on NUMA socket 0
1311   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1312   EAL:   using IOMMU type 1 (Type 1)
1313   EAL: PCI device 0000:05:11.1 on NUMA socket 0
1314   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1315   Auto-start selected
1316   Ports to start sending a burst of packets first
1317   Warning: lsc_interrupt needs to be off when  using tx_first. Disabling.
1318   USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
1319   Configuring Port 0 (socket 0)
1320   Port 0: 9E:FD:E6:DD:C1:01
1321   Configuring Port 1 (socket 0)
1322   Port 1: 9E:FD:E6:DD:C1:02
1323   Checking link statuses...
1324   Port0 Link Up. speed 10000 Mbps- full-duplex
1325   Port1 Link Up. speed 10000 Mbps- full-duplex
1326   Done
1327   No commandline core given, start packet forwarding
1328   io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
1329   disabled
1330   Logical Core 3 (socket 0) forwards packets on 2 streams:
1331     RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
1332     RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
1333
1334     io packet forwarding packets/burst=32
1335     nb forwarding cores=1 - nb forwarding ports=2
1336     port 0:
1337     CRC stripping enabled
1338     RX queues=1 - RX desc=128 - RX free threshold=32
1339     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1340     TX queues=1 - TX desc=512 - TX free threshold=32
1341     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1342     TX RS bit threshold=32 - TXQ flags=0xf01
1343     port 1:
1344     CRC stripping enabled
1345     RX queues=1 - RX desc=128 - RX free threshold=32
1346     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1347     TX queues=1 - TX desc=512 - TX free threshold=32
1348     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1349     TX RS bit threshold=32 - TXQ flags=0xf01
1350
1351   Port statistics ====================================
1352     ######################## NIC statistics for port 0  ########################
1353     RX-packets: 56         RX-missed: 0          RX-bytes:  4096
1354     RX-errors: 0
1355     RX-nombuf:  0
1356     TX-packets: 64         TX-errors: 0          TX-bytes:  4096
1357
1358     Throughput (since last show)
1359     Rx-pps:            0
1360     Tx-pps:            0
1361     ############################################################################
1362
1363     ######################## NIC statistics for port 1  ########################
1364     RX-packets: 432        RX-missed: 0          RX-bytes:  27712
1365     RX-errors: 0
1366     RX-nombuf:  0
1367     TX-packets: 461        TX-errors: 0          TX-bytes:  30080
1368
1369     Throughput (since last show)
1370     Rx-pps:            0
1371     Tx-pps:            0
1372     ############################################################################
1373
1374   Port statistics ====================================
1375     ######################## NIC statistics for port 0  ########################
1376     RX-packets: 14124641   RX-missed: 0          RX-bytes:  903977344
1377     RX-errors: 0
1378     RX-nombuf:  0
1379     TX-packets: 14170205   TX-errors: 0          TX-bytes:  906893376
1380
1381     Throughput (since last show)
1382     Rx-pps:      7068409
1383     Tx-pps:      7091206
1384     ############################################################################
1385
1386
1387
1388
1389
1390
1391 References
1392 ----------
1393
1394 - `StarlingX`_
1395
1396 .. _`StarlingX`: https://docs.starlingx.io/
1397