docs/installation-guide.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3 .. Copyright (C) 2019 Wind River Systems, Inc.
   4
   5
   6 Installation Guide
   7 ==================
   8
   9 .. contents::
  10    :depth: 3
  11    :local:
  12
  13 Abstract
  14 --------
  15
  16 This document describes how to install O-RAN INF image, example configuration for better
  17 real time performance, and example deployment of Kubernetes cluster and plugins.
  18
  19 The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
  20 and container technology.
  21
  22 Version history
  23
  24 +--------------------+--------------------+--------------------+--------------------+
  25 | **Date**           | **Ver.**           | **Author**         | **Comment**        |
  26 |                    |                    |                    |                    |
  27 +--------------------+--------------------+--------------------+--------------------+
  28 | 2019-11-02         | 1.0.0              | Jackie Huang       | Initail version    |
  29 |                    |                    |                    |                    |
  30 +--------------------+--------------------+--------------------+--------------------+
  31 | 2020-06-03         | 2.0.0 (Bronze)     | Xiaohua Zhang      | Bronze version     |
  32 |                    |                    |                    |                    |
  33 +--------------------+--------------------+--------------------+--------------------+
  34 |                    |                    |                    |                    |
  35 |                    |                    |                    |                    |
  36 +--------------------+--------------------+--------------------+--------------------+
  37
  38
  39 Preface
  40 -------
  41
  42 Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
  43
  44
  45 Hardware Requirements
  46 ---------------------
  47
  48 Following minimum hardware requirements must be met for installation of O-RAN INF image:
  49
  50 +--------------------+----------------------------------------------------+
  51 | **HW Aspect**      | **Requirement**                                    |
  52 |                    |                                                    |
  53 +--------------------+----------------------------------------------------+
  54 | **# of servers**   | 1                                                  |
  55 +--------------------+----------------------------------------------------+
  56 | **CPU**            | 4                                                  |
  57 |                    |                                                    |
  58 +--------------------+----------------------------------------------------+
  59 | **RAM**            | 16G                                                |
  60 |                    |                                                    |
  61 +--------------------+----------------------------------------------------+
  62 | **Disk**           | 500G * 2                                           |
  63 |                    |                                                    |
  64 +--------------------+----------------------------------------------------+
  65 | **NICs**           | 2                                                  |
  66 |                    |                                                    |
  67 +--------------------+----------------------------------------------------+
  68
  69
  70
  71 Software Installation and Deployment
  72 ------------------------------------
  73
  74 1. Installation from the O-RAN INF ISO image
  75 ````````````````````````````````````````````
  76
  77 - Please see the README.md file for how to build the image.
  78 - The Image is a live ISO image with CLI installer: inf-image-aio-installer-intel-corei7-64.iso
  79
  80 1.1 Burn the image to the USB device
  81 ''''''''''''''''''''''''''''''''''''
  82
  83 - Assume the the usb device is /dev/sdX here
  84
  85 ::
  86
  87   $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
  88
  89 1.2 Install the server
  90 ''''''''''''''''''''''
  91
  92 - Reboot the target from the USB device.
  93
  94 - Select "Graphics console install" or "Serial console install" and press ENTER
  95
  96 - Remove the USB device and press ENTER to reboot
  97
  98
  99 2. Configuration and initialize the bootstrap
 100 `````````````````````````````````````````````
 101
 102 2.1 First Login
 103 '''''''''''''''
 104 Example:
 105
 106 ::
 107
 108     ifconfig eno1 128.224.180.14/24 up
 109
 110     ip route add default via 128.224.180.1
 111
 112     ping 8.8.8.8
 113
 114 2.3 Login the server through SSH
 115 ''''''''''''''''''''''''''''''''
 116
 117 2.4 Prepare the localhost.yml for bootstrap
 118 '''''''''''''''''''''''''''''''''''''''''''
 119
 120 Example:
 121
 122 ::
 123
 124     system_mode: simplex
 125
 126     external_oam_subnet: 128.224.180.0/24
 127
 128     external_oam_gateway_address: 128.224.180.1
 129
 130     external_oam_floating_address: 128.224.180.14
 131
 132     admin_username: admin
 133
 134     admin_password: Wind123!
 135
 136     ansible_become_pass: Wind123!
 137
 138     docker_http_proxy: http://xxx.xxx.xxx.xxx:8080
 139
 140     docker_https_proxy: https://xxx.xxx.xxx.xxx:8080
 141
 142 2.5 Run the bootstrap
 143 '''''''''''''''''''''
 144
 145 Use the following command:
 146
 147 ::
 148
 149     ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
 150
 151 After the bootstrap successfully finish, it will show as following:
 152
 153 ::
 154
 155     PLAY RECAP ******************************************************************************************************************************************
 156     localhost                  : ok=256  changed=135  unreachable=0    failed=0    skipped=220  rescued=0    ignored=0
 157
 158 2.6 Configure the OAM network
 159 '''''''''''''''''''''''''''''
 160
 161 Example:
 162
 163 ::
 164
 165   controller-0:~$ source /etc/platform/openrc
 166
 167   [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno1
 168
 169   [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -c platform
 170
 171   +-----------------+--------------------------------------+
 172   | Property        | Value                                |
 173   +-----------------+--------------------------------------+
 174   | ifname          | eno1                                 |
 175   +-----------------+--------------------------------------+
 176   | iftype          | ethernet                             |
 177   +-----------------+--------------------------------------+
 178   | ports           | [u'eno1']                            |
 179   +-----------------+--------------------------------------+
 180   | imac            | 3c:a8:2a:1d:31:d0                    |
 181   +-----------------+--------------------------------------+
 182   | imtu            | 1500                                 |
 183   +-----------------+--------------------------------------+
 184   | ifclass         | platform                             |
 185   +-----------------+--------------------------------------+
 186   | aemode          | None                                 |
 187   +-----------------+--------------------------------------+
 188   | schedpolicy     | None                                 |
 189   +-----------------+--------------------------------------+
 190   | txhashpolicy    | None                                 |
 191   +-----------------+--------------------------------------+
 192   | uuid            | 95a32d13-01fc-43d2-90ab-801970fb3859 |
 193   +-----------------+--------------------------------------+
 194   | ihost_uuid      | 08c23843-afa4-4b80-a285-be126365276e |
 195   +-----------------+--------------------------------------+
 196   | vlan_id         | None                                 |
 197   +-----------------+--------------------------------------+
 198   | uses            | []                                   |
 199   +-----------------+--------------------------------------+
 200   | used_by         | []                                   |
 201   +-----------------+--------------------------------------+
 202   | created_at      | 2020-06-03T08:21:17.526303+00:00     |
 203   +-----------------+--------------------------------------+
 204   | updated_at      | 2020-06-03T10:02:54.682501+00:00     |
 205   +-----------------+--------------------------------------+
 206   | sriov_numvfs    | 0                                    |
 207   +-----------------+--------------------------------------+
 208   | sriov_vf_driver | None                                 |
 209   +-----------------+--------------------------------------+
 210   | accelerated     | [False]                              |
 211   +-----------------+--------------------------------------+
 212
 213   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 $OAM_IF oam
 214
 215   +--------------+--------------------------------------+
 216   | Property     | Value                                |
 217   +--------------+--------------------------------------+
 218   | hostname     | controller-0                         |
 219   +--------------+--------------------------------------+
 220   | uuid         | 32592cc3-9d66-4961-a51e-23686da00290 |
 221   +--------------+--------------------------------------+
 222   | ifname       | eno1                                 |
 223   +--------------+--------------------------------------+
 224   | network_name | oam                                  |
 225   +--------------+--------------------------------------+
 226
 227
 228   [sysadmin@controller-0 ~(keystone_admin)]$
 229   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-list controller-0
 230
 231   +--------------+--------------------------------------+--------+--------------+
 232   | hostname     | uuid                                 | ifname | network_name |
 233   +--------------+--------------------------------------+--------+--------------+
 234   | controller-0 | 2478f4c0-a7b4-4656-aa2a-97a0e8891955 | lo     | mgmt         |
 235   +--------------+--------------------------------------+--------+--------------+
 236   | controller-0 | 32592cc3-9d66-4961-a51e-23686da00290 | eno1   | oam          |
 237   +--------------+--------------------------------------+--------+--------------+
 238   | controller-0 | 43721e4d-4f52-4b4b-bd02-0efa00ef2124 | lo     | cluster-host |
 239   +--------------+--------------------------------------+--------+--------------+
 240
 241 2.7 Configure the NTP server
 242 ''''''''''''''''''''''''''''
 243
 244 Example:
 245
 246 ::
 247
 248   [sysadmin@controller-0 ~(keystone_admin)]$
 249   [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
 250
 251   +--------------+--------------------------------------+
 252   | Property     | Value                                |
 253   +--------------+--------------------------------------+
 254   | uuid         | 8fbe5712-6ba0-4e61-b174-ed60f152f616 |
 255   +--------------+--------------------------------------+
 256   | ntpservers   | 0.pool.ntp.org,1.pool.ntp.org        |
 257   +--------------+--------------------------------------+
 258   | isystem_uuid | 802ee0be-3728-4df4-b9f7-1533c67fd960 |
 259   +--------------+--------------------------------------+
 260   | created_at   | 2020-06-03T08:19:47.812249+00:00     |
 261   +--------------+--------------------------------------+
 262   | updated_at   | None                                 |
 263   +--------------+--------------------------------------+
 264
 265 2.8 Configure the Storage by using the second hard drive as ceph backend
 266 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 267
 268 Example:
 269
 270 ::
 271
 272   [sysadmin@controller-0 ~(keystone_admin)]$
 273
 274   [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
 275
 276   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 277   | uuid                                 | device_no | device_ | device_ | size_ | available_ | rpm          | serial_ | device_path                 |
 278   |                                      | de        | num     | type    | gib   | gib        |              | id      |                             |
 279   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 280   | 3884f906-cb9b-4052-bcb8-8be70e599b9f | /dev/sda  | 2048    | HDD     | 1117. | 868.759    | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
 281   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 282   |                                      |           |         |         | 782   |            |              | RH8W4QQ | 03:00.0-scsi-0:1:0:0        |
 283   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 284   |                                      |           |         |         |       |            |              |         |                             |
 285   | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/sdb  | 2064    | HDD     | 1117. | 0.0        | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
 286   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 287   |                                      |           |         |         | 782   |            |              | RH8W4QQ | 03:00.0-scsi-0:1:0:1        |
 288   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 289   |                                      |           |         |         |       |            |              |         |                             |
 290   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 291
 292   [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
 293
 294   +------------------+-------------------------------------------------------+
 295   | Property         | Value                                                 |
 296   +------------------+-------------------------------------------------------+
 297   | osdid            | 0                                                     |
 298   +------------------+-------------------------------------------------------+
 299   | function         | osd                                                   |
 300   +------------------+-------------------------------------------------------+
 301   | state            | configuring-on-unlock                                 |
 302   +------------------+-------------------------------------------------------+
 303   | journal_location | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf                  |
 304   +------------------+-------------------------------------------------------+
 305   | journal_size_gib | 1024                                                  |
 306   +------------------+-------------------------------------------------------+
 307   | journal_path     | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 |
 308   +------------------+-------------------------------------------------------+
 309   | journal_node     | /dev/sdb2                                             |
 310   +------------------+-------------------------------------------------------+
 311   | uuid             | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf                  |
 312   +------------------+-------------------------------------------------------+
 313   | ihost_uuid       | 08c23843-afa4-4b80-a285-be126365276e                  |
 314   +------------------+-------------------------------------------------------+
 315   | idisk_uuid       | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9                  |
 316   +------------------+-------------------------------------------------------+
 317   | tier_uuid        | db6eabf9-9922-4d8e-9740-c0f9878e692d                  |
 318   +------------------+-------------------------------------------------------+
 319   | tier_name        | storage                                               |
 320   +------------------+-------------------------------------------------------+
 321   | created_at       | 2020-06-03T10:03:55.645744+00:00                      |
 322   +------------------+-------------------------------------------------------+
 323   | updated_at       | None                                                  |
 324   +------------------+-------------------------------------------------------+
 325
 326   [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
 327
 328   +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 329   | uuid                                 | function | osdid | state                 | idisk_uuid                           | journal_path                                          | journal_node | journal_size_gib | tier_name |
 330   +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 331   | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf | osd      | 0     | configuring-on-unlock | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 | /dev/sdb2    | 1                | storage   |
 332   +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 333
 334 2.9 Configure the SR-IOV enabled network
 335 ''''''''''''''''''''''''''''''''''''''''
 336
 337 It needs :
 338   - assign the label of SR-IOV
 339   - change the CPU policy to static
 340   - change the hugepage setting
 341   - create the data network
 342   - assign the SR-IOV interface to data network
 343
 344 Example:
 345
 346 ::
 347
 348   [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
 349
 350   [sysadmin@controller-0 ~(keystone_admin)]$
 351
 352   [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER sriovdp=enabled
 353
 354   +-------------+--------------------------------------+
 355   | Property    | Value                                |
 356   +-------------+--------------------------------------+
 357   | uuid        | 7101010d-1897-4cc8-8c87-7dffaebf848b |
 358   +-------------+--------------------------------------+
 359   | host_uuid   | 08c23843-afa4-4b80-a285-be126365276e |
 360   +-------------+--------------------------------------+
 361   | label_key   | sriovdp                              |
 362   +-------------+--------------------------------------+
 363   | label_value | enabled                              |
 364   +-------------+--------------------------------------+
 365
 366   [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER kube-cpu-mgr-policy=static
 367
 368   +-------------+--------------------------------------+
 369   | Property    | Value                                |
 370   +-------------+--------------------------------------+
 371   | uuid        | 4936f32b-6939-4115-b643-846bd50e7738 |
 372   +-------------+--------------------------------------+
 373   | host_uuid   | 08c23843-afa4-4b80-a285-be126365276e |
 374   +-------------+--------------------------------------+
 375   | label_key   | kube-cpu-mgr-policy                  |
 376   +-------------+--------------------------------------+
 377   | label_value | static                               |
 378   +-------------+--------------------------------------+
 379
 380
 381   [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7  0
 382
 383   +-------------------------------------+--------------------------------------+
 384   | Property                            | Value                                |
 385   +-------------------------------------+--------------------------------------+
 386   | Memory: Usable Total (MiB)          | 0                                    |
 387   +-------------------------------------+--------------------------------------+
 388   |         Platform     (MiB)          | 7000                                 |
 389   +-------------------------------------+--------------------------------------+
 390   |         Available    (MiB)          | 0                                    |
 391   +-------------------------------------+--------------------------------------+
 392   | Huge Pages Configured               | True                                 |
 393   +-------------------------------------+--------------------------------------+
 394   | vSwitch Huge Pages: Size (MiB)      | 1024                                 |
 395   +-------------------------------------+--------------------------------------+
 396   |                     Total           | 0                                    |
 397   +-------------------------------------+--------------------------------------+
 398   |                     Available       | 0                                    |
 399   +-------------------------------------+--------------------------------------+
 400   |                     Required        | None                                 |
 401   +-------------------------------------+--------------------------------------+
 402   | Application  Pages (4K): Total      | None                                 |
 403   +-------------------------------------+--------------------------------------+
 404   | Application  Huge Pages (2M): Total | 0                                    |
 405   +-------------------------------------+--------------------------------------+
 406   |                 Available           | 0                                    |
 407   +-------------------------------------+--------------------------------------+
 408   | Application  Huge Pages (1G): Total | 0                                    |
 409   +-------------------------------------+--------------------------------------+
 410   |                 Total Pending       | 7                                    |
 411   +-------------------------------------+--------------------------------------+
 412   |                 Available           | 0                                    |
 413   +-------------------------------------+--------------------------------------+
 414   | uuid                                | 15702f44-2281-41f3-8c83-a1e9a300ad01 |
 415   +-------------------------------------+--------------------------------------+
 416   | ihost_uuid                          | 08c23843-afa4-4b80-a285-be126365276e |
 417   +-------------------------------------+--------------------------------------+
 418   | inode_uuid                          | d52648bc-dfa6-4214-afa3-5d2148ee4180 |
 419   +-------------------------------------+--------------------------------------+
 420   | created_at                          | 2020-06-03T08:21:27.468357+00:00     |
 421   +-------------------------------------+--------------------------------------+
 422   | updated_at                          | 2020-06-03T10:10:49.159080+00:00     |
 423   +-------------------------------------+--------------------------------------+
 424
 425
 426   [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7 1
 427
 428   +-------------------------------------+--------------------------------------+
 429   | Property                            | Value                                |
 430   +-------------------------------------+--------------------------------------+
 431   | Memory: Usable Total (MiB)          | 0                                    |
 432   +-------------------------------------+--------------------------------------+
 433   |         Platform     (MiB)          | 1000                                 |
 434   +-------------------------------------+--------------------------------------+
 435   |         Available    (MiB)          | 0                                    |
 436   +-------------------------------------+--------------------------------------+
 437   | Huge Pages Configured               | True                                 |
 438   +-------------------------------------+--------------------------------------+
 439   | vSwitch Huge Pages: Size (MiB)      | 1024                                 |
 440   +-------------------------------------+--------------------------------------+
 441   |                     Total           | 0                                    |
 442   +-------------------------------------+--------------------------------------+
 443   |                     Available       | 0                                    |
 444   +-------------------------------------+--------------------------------------+
 445   |                     Required        | None                                 |
 446   +-------------------------------------+--------------------------------------+
 447   | Application  Pages (4K): Total      | None                                 |
 448   +-------------------------------------+--------------------------------------+
 449   | Application  Huge Pages (2M): Total | 0                                    |
 450   +-------------------------------------+--------------------------------------+
 451   |                 Available           | 0                                    |
 452   +-------------------------------------+--------------------------------------+
 453   | Application  Huge Pages (1G): Total | 0                                    |
 454   +-------------------------------------+--------------------------------------+
 455   |                 Total Pending       | 7                                    |
 456   +-------------------------------------+--------------------------------------+
 457   |                 Available           | 0                                    |
 458   +-------------------------------------+--------------------------------------+
 459   | uuid                                | 544580b9-0f2a-41a1-b4e3-90fdf5c8f7e0 |
 460   +-------------------------------------+--------------------------------------+
 461   | ihost_uuid                          | 08c23843-afa4-4b80-a285-be126365276e |
 462   +-------------------------------------+--------------------------------------+
 463   | inode_uuid                          | e02c3c41-03b7-4f49-8244-bca851fa2d2f |
 464   +-------------------------------------+--------------------------------------+
 465   | created_at                          | 2020-06-03T08:21:27.566279+00:00     |
 466   +-------------------------------------+--------------------------------------+
 467   | updated_at                          | 2020-06-03T10:10:49.220907+00:00     |
 468   +-------------------------------------+--------------------------------------+
 469
 470
 471   [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET0='physnet0'
 472
 473   [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET1='physnet1'
 474
 475   [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
 476
 477   [sysadmin@controller-0 ~(keystone_admin)]$
 478
 479   [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET0} flat
 480
 481   +--------------+--------------------------------------+
 482   | Property     | Value                                |
 483   +--------------+--------------------------------------+
 484   | id           | 1                                    |
 485   +--------------+--------------------------------------+
 486   | uuid         | 0ffe0b0a-35ff-45b2-9a62-63a494d2d18b |
 487   +--------------+--------------------------------------+
 488   | name         | physnet0                             |
 489   +--------------+--------------------------------------+
 490   | network_type | flat                                 |
 491   +--------------+--------------------------------------+
 492   | mtu          | 1500                                 |
 493   +--------------+--------------------------------------+
 494   | description  | None                                 |
 495   +--------------+--------------------------------------+
 496
 497   [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET1} flat
 498
 499   +--------------+--------------------------------------+
 500   | Property     | Value                                |
 501   +--------------+--------------------------------------+
 502   | id           | 2                                    |
 503   +--------------+--------------------------------------+
 504   | uuid         | 899ca8ba-6800-4d62-aed4-3770878da738 |
 505   +--------------+--------------------------------------+
 506   | name         | physnet1                             |
 507   +--------------+--------------------------------------+
 508   | network_type | flat                                 |
 509   +--------------+--------------------------------------+
 510   | mtu          | 1500                                 |
 511   +--------------+--------------------------------------+
 512   | description  | None                                 |
 513   +--------------+--------------------------------------+
 514
 515
 516   [sysadmin@controller-0 ~(keystone_admin)]$ system host-port-list controller-0
 517
 518   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 519   | uuid                                 | name   | type     | pci address  | device | processor | accelerated | device type                                           |
 520   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 521   | 7007dfb1-ea55-4355-9d27-3ed4c6696283 | eno1   | ethernet | 0000:02:00.0 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 522   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 523   | 51f62ecf-7897-43e3-beb8-46ae5550ef05 | eno2   | ethernet | 0000:02:00.1 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 524   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 525   | e987725e-ab45-4151-b53d-94ba21706eb4 | eno3   | ethernet | 0000:02:00.2 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 526   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 527   | f4a8115a-ca0b-4336-a634-2cad77c5a88c | eno4   | ethernet | 0000:02:00.3 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 528   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 529   | 88965882-c816-42e8-ba0e-d3ff6f9f553e | eno49  | ethernet | 0000:04:00.0 | 0      | 0         | False       | OneConnect NIC (Skyhawk) [0720]                       |
 530   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 531   | 98e2cedc-9ef8-4ca0-b3e0-db3a2bd0273a | eno50  | ethernet | 0000:04:00.1 | 0      | 0         | False       | OneConnect NIC (Skyhawk) [0720]                       |
 532   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 533   | e75d7aa6-7832-4ec4-9dd4-ea8300773fd8 | ens2f0 | ethernet | 0000:05:00.0 | 0      | 0         | True        | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
 534   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 535   | 8ac4ffca-4a61-4b47-8e17-171f2e6cea66 | ens2f1 | ethernet | 0000:05:00.1 | 0      | 0         | True        | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
 536   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 537
 538
 539   [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f0
 540
 541   [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov0 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
 542
 543   +-----------------+--------------------------------------+
 544   | Property        | Value                                |
 545   +-----------------+--------------------------------------+
 546   | ifname          | sriov0                               |
 547   +-----------------+--------------------------------------+
 548   | iftype          | ethernet                             |
 549   +-----------------+--------------------------------------+
 550   | ports           | [u'ens2f0']                          |
 551   +-----------------+--------------------------------------+
 552   | imac            | 00:1b:21:b3:46:2c                    |
 553   +-----------------+--------------------------------------+
 554   | imtu            | 1500                                 |
 555   +-----------------+--------------------------------------+
 556   | ifclass         | pci-sriov                            |
 557   +-----------------+--------------------------------------+
 558   | aemode          | None                                 |
 559   +-----------------+--------------------------------------+
 560   | schedpolicy     | None                                 |
 561   +-----------------+--------------------------------------+
 562   | txhashpolicy    | None                                 |
 563   +-----------------+--------------------------------------+
 564   | uuid            | 9450b1ae-d1a1-4ecb-811a-fa3cb1cdf362 |
 565   +-----------------+--------------------------------------+
 566   | ihost_uuid      | 08c23843-afa4-4b80-a285-be126365276e |
 567   +-----------------+--------------------------------------+
 568   | vlan_id         | None                                 |
 569   +-----------------+--------------------------------------+
 570   | uses            | []                                   |
 571   +-----------------+--------------------------------------+
 572   | used_by         | []                                   |
 573   +-----------------+--------------------------------------+
 574   | created_at      | 2020-06-03T12:34:11.633254+00:00     |
 575   +-----------------+--------------------------------------+
 576   | updated_at      | 2020-06-04T02:01:28.168127+00:00     |
 577   +-----------------+--------------------------------------+
 578   | sriov_numvfs    | 6                                    |
 579   +-----------------+--------------------------------------+
 580   | sriov_vf_driver | vfio                                 |
 581   +-----------------+--------------------------------------+
 582   | accelerated     | [True]                               |
 583   +-----------------+--------------------------------------+
 584
 585   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov0 ${PHYSNET0}
 586
 587   +------------------+--------------------------------------+
 588   | Property         | Value                                |
 589   +------------------+--------------------------------------+
 590   | hostname         | controller-0                         |
 591   +------------------+--------------------------------------+
 592   | uuid             | 1d35219d-3258-42e7-8f0e-41d865710d58 |
 593   +------------------+--------------------------------------+
 594   | ifname           | sriov0                               |
 595   +------------------+--------------------------------------+
 596   | datanetwork_name | physnet0                             |
 597   +------------------+--------------------------------------+
 598
 599
 600
 601   [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f1
 602
 603   [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov1 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
 604
 605   +-----------------+--------------------------------------+
 606   | Property        | Value                                |
 607   +-----------------+--------------------------------------+
 608   | ifname          | sriov1                               |
 609   +-----------------+--------------------------------------+
 610   | iftype          | ethernet                             |
 611   +-----------------+--------------------------------------+
 612   | ports           | [u'ens2f1']                          |
 613   +-----------------+--------------------------------------+
 614   | imac            | 00:1b:21:b3:46:2d                    |
 615   +-----------------+--------------------------------------+
 616   | imtu            | 1500                                 |
 617   +-----------------+--------------------------------------+
 618   | ifclass         | pci-sriov                            |
 619   +-----------------+--------------------------------------+
 620   | aemode          | None                                 |
 621   +-----------------+--------------------------------------+
 622   | schedpolicy     | None                                 |
 623   +-----------------+--------------------------------------+
 624   | txhashpolicy    | None                                 |
 625   +-----------------+--------------------------------------+
 626   | uuid            | 13d5ee99-0399-4b6b-a359-0f96660afdc4 |
 627   | ihost_uuid      | 08c23843-afa4-4b80-a285-be126365276e |
 628   | vlan_id         | None                                 |
 629   | uses            | []                                   |
 630   | used_by         | []                                   |
 631   | created_at      | 2020-06-03T12:34:12.072624+00:00     |
 632   | updated_at      | 2020-06-04T02:03:51.405322+00:00     |
 633   | sriov_numvfs    | 6                                    |
 634   | sriov_vf_driver | vfio                                 |
 635   | accelerated     | [True]                               |
 636   +-----------------+--------------------------------------+
 637
 638   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov1 ${PHYSNET1}
 639
 640   +------------------+--------------------------------------+
 641   | Property         | Value                                |
 642   +------------------+--------------------------------------+
 643   | hostname         | controller-0                         |
 644   +------------------+--------------------------------------+
 645   | uuid             | a87c6a5f-0c53-4ebf-9a32-38366700426f |
 646   +------------------+--------------------------------------+
 647   | ifname           | sriov1                               |
 648   +------------------+--------------------------------------+
 649   | datanetwork_name | physnet1                             |
 650   +------------------+--------------------------------------+
 651
 652 2.10 Unlock the server
 653 ''''''''''''''''''''''
 654
 655 Example:
 656
 657 ::
 658
 659   [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
 660
 661   +-----------------------+-------------------------------------------------+
 662   | Property              | Value                                           |
 663   +-----------------------+-------------------------------------------------+
 664   | action                | none                                            |
 665   +-----------------------+-------------------------------------------------+
 666   | administrative        | locked                                          |
 667   +-----------------------+-------------------------------------------------+
 668   | availability          | online                                          |
 669   +-----------------------+-------------------------------------------------+
 670   | bm_ip                 | None                                            |
 671   +-----------------------+-------------------------------------------------+
 672   | bm_type               | none                                            |
 673   +-----------------------+-------------------------------------------------+
 674   | bm_username           | None                                            |
 675   +-----------------------+-------------------------------------------------+
 676   | boot_device           | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
 677   +-----------------------+-------------------------------------------------+
 678   | capabilities          | {u'stor_function': u'monitor'}                  |
 679   +-----------------------+-------------------------------------------------+
 680   | clock_synchronization | ntp                                             |
 681   +-----------------------+-------------------------------------------------+
 682   | config_applied        | 3daf20c7-c032-4aa2-839d-93da92051bea            |
 683   +-----------------------+-------------------------------------------------+
 684   | config_status         | Config out-of-date                              |
 685   +-----------------------+-------------------------------------------------+
 686   | config_target         | bdaf20c7-c032-4aa2-839d-93da92051bea            |
 687   +-----------------------+-------------------------------------------------+
 688   | console               | tty0                                            |
 689   +-----------------------+-------------------------------------------------+
 690   | created_at            | 2020-06-03T08:20:59.372602+00:00                |
 691   +-----------------------+-------------------------------------------------+
 692   | hostname              | controller-0                                    |
 693   +-----------------------+-------------------------------------------------+
 694   | id                    | 1                                               |
 695   +-----------------------+-------------------------------------------------+
 696   | install_output        | graphical                                       |
 697   +-----------------------+-------------------------------------------------+
 698   | install_state         | None                                            |
 699   +-----------------------+-------------------------------------------------+
 700   | install_state_info    | None                                            |
 701   +-----------------------+-------------------------------------------------+
 702   | inv_state             | inventoried                                     |
 703   +-----------------------+-------------------------------------------------+
 704   | invprovision          | provisioning                                    |
 705   +-----------------------+-------------------------------------------------+
 706   | location              | {}                                              |
 707   +-----------------------+-------------------------------------------------+
 708   | mgmt_ip               | 192.168.204.2                                   |
 709   +-----------------------+-------------------------------------------------+
 710   | mgmt_mac              | 00:00:00:00:00:00                               |
 711   +-----------------------+-------------------------------------------------+
 712   | operational           | disabled                                        |
 713   +-----------------------+-------------------------------------------------+
 714   | personality           | controller                                      |
 715   +-----------------------+-------------------------------------------------+
 716   | reserved              | False                                           |
 717   +-----------------------+-------------------------------------------------+
 718   | rootfs_device         | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
 719   +-----------------------+-------------------------------------------------+
 720   | serialid              | None                                            |
 721   +-----------------------+-------------------------------------------------+
 722   | software_load         | 19.12                                           |
 723   +-----------------------+-------------------------------------------------+
 724   | subfunction_avail     | online                                          |
 725   +-----------------------+-------------------------------------------------+
 726   | subfunction_oper      | disabled                                        |
 727   +-----------------------+-------------------------------------------------+
 728   | subfunctions          | controller,worker                               |
 729   +-----------------------+-------------------------------------------------+
 730   | task                  | Unlocking                                       |
 731   +-----------------------+-------------------------------------------------+
 732   | tboot                 | false                                           |
 733   +-----------------------+-------------------------------------------------+
 734   | ttys_dcd              | None                                            |
 735   +-----------------------+-------------------------------------------------+
 736   | updated_at            | 2020-06-04T02:04:59.514854+00:00                |
 737   +-----------------------+-------------------------------------------------+
 738   | uptime                | 49475                                           |
 739   +-----------------------+-------------------------------------------------+
 740   | uuid                  | 08c23843-afa4-4b80-a285-be126365276e            |
 741   +-----------------------+-------------------------------------------------+
 742   | vim_progress_status   | None                                            |
 743   +-----------------------+-------------------------------------------------+
 744
 745 3. Simple use case for sriov
 746 ````````````````````````````
 747
 748 3.1 After controller-0 is rebooted and up running, download the DPDK
 749 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 750
 751 ::
 752
 753   [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
 754   [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
 755   Password:
 756   --2020-06-04 02:35:30--  https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
 757   Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
 758   Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
 759
 760   HTTP request sent, awaiting response... 200 OK
 761   Length: 10251680 (9.8M) [application/octet-stream]
 762   Saving to: ‘dpdk-17.11.10.tar.xz’
 763
 764   dpdk-17.11.10.tar.xz                        100%
 765   [========================================================================================>]   9.78M
 766   1.48MB/s    in 6.8s
 767
 768   2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
 769
 770   sudo tar xvf dpdk-17.11.10.tar.xz
 771
 772   sudo ln -s dpdk-stable-17.11.10 dpdk-stable
 773
 774 3.2 Prepare the yaml file for the network assignment container
 775 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
 776
 777 The following the exmaple of the yaml file:
 778
 779 ::
 780
 781   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
 782   > apiVersion: "k8s.cni.cncf.io/v1"
 783   > kind: NetworkAttachmentDefinition
 784   > metadata:
 785   >   name: sriov-data-dpdk-0
 786   >   annotations:
 787   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
 788   > spec:
 789   >   config: '{
 790   >   "type": "sriov",
 791   >   "name": "sriov-data-dpdk-0"
 792   > }'
 793   >
 794   > ---
 795   > apiVersion: "k8s.cni.cncf.io/v1"
 796   > kind: NetworkAttachmentDefinition
 797   > metadata:
 798   >   name: sriov-data-dpdk-1
 799   >   annotations:
 800   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
 801   > spec:
 802   >   config: '{
 803   >   "type": "sriov",
 804   >   "name": "sriov-data-dpdk-1"
 805   > }'
 806   > EOF
 807
 808 3.3 Run the network assignent container for the 2 VFs
 809 '''''''''''''''''''''''''''''''''''''''''''''''''''''
 810
 811 ::
 812
 813   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
 814   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
 815   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
 816
 817 3.4 Prepare the VF container yaml file
 818 ''''''''''''''''''''''''''''''''''''''
 819
 820 ::
 821
 822   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
 823   > apiVersion: v1
 824   > kind: Pod
 825   metadata:
 826   > metadata:
 827   >   name: pod-with-dpdk-vfs-0
 828   >   annotations:
 829   >     k8s.v1.cni.cncf.io/networks: '[
 830   >             { "name": "sriov-data-dpdk-0" },
 831               { "name": "sriov-data-dpdk-1" }
 832   >             { "name": "sriov-data-dpdk-1" }
 833   >     ]'
 834   > spec:
 835   >   restartPolicy: Never
 836   >   containers:
 837   >   - name: pod-with-dpdk-vfs-0
 838   >     image: wrsnfv/ubuntu-dpdk-build:v0.3
 839   >     env:
 840   >     - name: RTE_SDK
 841   >       value: "/usr/src/dpdk"
 842   >     command:
 843   >     - sleep
 844   >     - infinity
 845   >     stdin: true
 846   >     tty: true
 847   >     securityContext:
 848   >       privileged: true
 849   >       capabilities:
 850   >         add:
 851   >         - ALL
 852   >     resources:
 853   >       requests:
 854   >         cpu: 4
 855   >         memory: 4Gi
 856   >         intel.com/pci_sriov_net_physnet0: '1'
 857   >         intel.com/pci_sriov_net_physnet1: '1'
 858   >       limits:
 859   >         cpu: 4
 860   >         hugepages-1Gi: 2Gi
 861   >         memory: 4Gi
 862   >         intel.com/pci_sriov_net_physnet0: '1'
 863   >         intel.com/pci_sriov_net_physnet1: '1'
 864   >     volumeMounts:
 865   >     - mountPath: /mnt/huge-1048576kB
 866   >       name: hugepage
 867   >     - name: dpdk-volume
 868   >       mountPath: /usr/src/dpdk
 869   >     - name: lib-volume
 870   >       mountPath: /lib/modules
 871   >     - name: src-volume
 872   >       mountPath: /usr/src/
 873   >   volumes:
 874   >   - name: hugepage
 875   >     emptyDir:
 876   >       medium: HugePages
 877   >   - name: dpdk-volume
 878   >     hostPath:
 879   >       path: /opt/dpdk-stable/
 880   >   - name: lib-volume
 881   >     hostPath:
 882   >       path: /lib/modules
 883   >   - name: src-volume
 884   >     hostPath:
 885   >       path: /usr/src/
 886   > EOF
 887
 888 3.5 Run the VF container
 889 ''''''''''''''''''''''''
 890
 891 Start the VF container.
 892
 893 ::
 894
 895   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
 896   pod/pod-with-dpdk-vfs-0 created
 897
 898   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
 899   NAME                  READY   STATUS    RESTARTS   AGE
 900   pod-with-dpdk-vfs-0   1/1     Running   0          6m40s
 901
 902 Login the VF container
 903
 904 ::
 905
 906   kubectl exec -it pod-with-dpdk-vfs-0 -- bash
 907
 908 Build the DPDK
 909
 910 ::
 911
 912   cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
 913
 914   root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
 915     HOSTCC  scripts/basic/fixdep
 916     HOSTCC  scripts/kconfig/conf.o
 917     HOSTCC  scripts/kconfig/confdata.o
 918     HOSTCC  scripts/kconfig/expr.o
 919     HOSTCC  scripts/kconfig/symbol.o
 920     HOSTCC  scripts/kconfig/preprocess.o
 921     HOSTCC  scripts/kconfig/zconf.lex.o
 922     HOSTCC  scripts/kconfig/zconf.tab.o
 923     HOSTLD  scripts/kconfig/conf
 924   scripts/kconfig/conf  --syncconfig Kconfig
 925     HOSTCC  arch/x86/tools/relocs_32.o
 926     HOSTCC  arch/x86/tools/relocs_64.o
 927     HOSTCC  arch/x86/tools/relocs_common.o
 928     HOSTLD  arch/x86/tools/relocs
 929     HOSTCC  scripts/genksyms/genksyms.o
 930     YACC    scripts/genksyms/parse.tab.c
 931     HOSTCC  scripts/genksyms/parse.tab.o
 932     LEX     scripts/genksyms/lex.lex.c
 933     YACC    scripts/genksyms/parse.tab.h
 934     HOSTCC  scripts/genksyms/lex.lex.o
 935     HOSTLD  scripts/genksyms/genksyms
 936     HOSTCC  scripts/bin2c
 937     HOSTCC  scripts/kallsyms
 938     HOSTCC  scripts/conmakehash
 939     HOSTCC  scripts/recordmcount
 940     HOSTCC  scripts/sortextable
 941     HOSTCC  scripts/asn1_compiler
 942     HOSTCC  scripts/sign-file
 943     HOSTCC  scripts/extract-cert
 944     CC      scripts/mod/empty.o
 945     HOSTCC  scripts/mod/mk_elfconfig
 946     MKELF   scripts/mod/elfconfig.h
 947     HOSTCC  scripts/mod/modpost.o
 948     CC      scripts/mod/devicetable-offsets.s
 949     UPD     scripts/mod/devicetable-offsets.h
 950     HOSTCC  scripts/mod/file2alias.o
 951     HOSTCC  scripts/mod/sumversion.o
 952     HOSTLD  scripts/mod/modpost
 953     CC      kernel/bounds.s
 954     CC      arch/x86/kernel/asm-offsets.s
 955     CALL    scripts/checksyscalls.sh
 956
 957 Build the test_pmd application
 958
 959 ::
 960
 961   cd $RTE_SDK
 962   ./usertools/dpdk-setup.sh
 963   Option: 14
 964     CC config.o
 965     CC iofwd.o
 966     CC macfwd.o
 967     CC macswap.o
 968     CC flowgen.o
 969     CC rxonly.o
 970     CC txonly.o
 971     CC csumonly.o
 972     CC icmpecho.o
 973     CC tm.o
 974     LD testpmd
 975     INSTALL-APP testpmd
 976     INSTALL-MAP testpmd.map
 977   == Build app/proc_info
 978     CC main.o
 979     LD dpdk-procinfo
 980     INSTALL-APP dpdk-procinfo
 981     INSTALL-MAP dpdk-procinfo.map
 982   == Build app/pdump
 983     CC main.o
 984     LD dpdk-pdump
 985     INSTALL-APP dpdk-pdump
 986     INSTALL-MAP dpdk-pdump.map
 987   == Build app/test-crypto-perf
 988     CC main.o
 989     CC cperf_ops.o
 990     CC cperf_options_parsing.o
 991     CC cperf_test_vectors.o
 992     CC cperf_test_throughput.o
 993     CC cperf_test_latency.o
 994     CC cperf_test_pmd_cyclecount.o
 995     CC cperf_test_verify.o
 996     CC cperf_test_vector_parsing.o
 997     CC cperf_test_common.o
 998     LD dpdk-test-crypto-perf
 999     INSTALL-APP dpdk-test-crypto-perf
1000     INSTALL-MAP dpdk-test-crypto-perf.map
1001   == Build app/test-eventdev
1002     CC evt_main.o
1003     CC evt_options.o
1004     CC evt_test.o
1005     CC parser.o
1006     CC test_order_common.o
1007     CC test_order_queue.o
1008     CC test_order_atq.o
1009     CC test_perf_common.o
1010     CC test_perf_queue.o
1011     CC test_perf_atq.o
1012     LD dpdk-test-eventdev
1013     INSTALL-APP dpdk-test-eventdev
1014     INSTALL-MAP dpdk-test-eventdev.map
1015   Build complete [x86_64-native-linuxapp-gcc]
1016   Installation cannot run with T defined and DESTDIR undefined
1017   ------------------------------------------------------------------------------
1018   RTE_TARGET exported as x86_64-native-linuxapp-gcc
1019   ------------------------------------------------------------------------------
1020
1021   Press enter to continue ...
1022
1023 Check the VF PCI information:
1024
1025 ::
1026
1027   root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
1028   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
1029   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
1030
1031 Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
1032
1033 ::
1034
1035   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
1036   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
1037   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
1038   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
1039   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
1040   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
1041   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
1042
1043   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
1044   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
1045   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
1046   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
1047   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
1048   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
1049   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
1050
1051   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
1052   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
1053
1054
1055 Get back to pod and run the test_pmd
1056
1057 ::
1058
1059   cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
1060
1061   taskset -p --cpu-list 1
1062   pid 1's current affinity list: 2,3,18,19
1063
1064   ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
1065   start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
1066   peer=1,"9e:fd:e6:dd:c1:01"
1067   EAL: Detected 32 lcore(s)
1068   EAL: No free hugepages reported in hugepages-2048kB
1069   EAL: Probing VFIO support...
1070   EAL: VFIO support initialized
1071   EAL: PCI device 0000:05:11.0 on NUMA socket 0
1072   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1073   EAL:   using IOMMU type 1 (Type 1)
1074   EAL: PCI device 0000:05:11.1 on NUMA socket 0
1075   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1076   Auto-start selected
1077   Ports to start sending a burst of packets first
1078   Warning: lsc_interrupt needs to be off when  using tx_first. Disabling.
1079   USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
1080   Configuring Port 0 (socket 0)
1081   Port 0: 9E:FD:E6:DD:C1:01
1082   Configuring Port 1 (socket 0)
1083   Port 1: 9E:FD:E6:DD:C1:02
1084   Checking link statuses...
1085   Port0 Link Up. speed 10000 Mbps- full-duplex
1086   Port1 Link Up. speed 10000 Mbps- full-duplex
1087   Done
1088   No commandline core given, start packet forwarding
1089   io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
1090   disabled
1091   Logical Core 3 (socket 0) forwards packets on 2 streams:
1092     RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
1093     RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
1094
1095     io packet forwarding packets/burst=32
1096     nb forwarding cores=1 - nb forwarding ports=2
1097     port 0:
1098     CRC stripping enabled
1099     RX queues=1 - RX desc=128 - RX free threshold=32
1100     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1101     TX queues=1 - TX desc=512 - TX free threshold=32
1102     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1103     TX RS bit threshold=32 - TXQ flags=0xf01
1104     port 1:
1105     CRC stripping enabled
1106     RX queues=1 - RX desc=128 - RX free threshold=32
1107     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1108     TX queues=1 - TX desc=512 - TX free threshold=32
1109     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1110     TX RS bit threshold=32 - TXQ flags=0xf01
1111
1112   Port statistics ====================================
1113     ######################## NIC statistics for port 0  ########################
1114     RX-packets: 56         RX-missed: 0          RX-bytes:  4096
1115     RX-errors: 0
1116     RX-nombuf:  0
1117     TX-packets: 64         TX-errors: 0          TX-bytes:  4096
1118
1119     Throughput (since last show)
1120     Rx-pps:            0
1121     Tx-pps:            0
1122     ############################################################################
1123
1124     ######################## NIC statistics for port 1  ########################
1125     RX-packets: 432        RX-missed: 0          RX-bytes:  27712
1126     RX-errors: 0
1127     RX-nombuf:  0
1128     TX-packets: 461        TX-errors: 0          TX-bytes:  30080
1129
1130     Throughput (since last show)
1131     Rx-pps:            0
1132     Tx-pps:            0
1133     ############################################################################
1134
1135   Port statistics ====================================
1136     ######################## NIC statistics for port 0  ########################
1137     RX-packets: 14124641   RX-missed: 0          RX-bytes:  903977344
1138     RX-errors: 0
1139     RX-nombuf:  0
1140     TX-packets: 14170205   TX-errors: 0          TX-bytes:  906893376
1141
1142     Throughput (since last show)
1143     Rx-pps:      7068409
1144     Tx-pps:      7091206
1145     ############################################################################
1146
1147
1148
1149
1150
1151
1152 References
1153 ----------
1154
1155 - `StarlingX`_
1156
1157 .. _`StarlingX`: https://docs.starlingx.io/
1158