docs/installation-guide.rst

   1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
   2 .. SPDX-License-Identifier: CC-BY-4.0
   3 .. Copyright (C) 2019 Wind River Systems, Inc.
   4
   5
   6 Installation Guide
   7 ==================
   8
   9 .. contents::
  10    :depth: 3
  11    :local:
  12
  13 Abstract
  14 --------
  15
  16 This document describes how to install O-RAN INF image, example configuration for better
  17 real time performance, and example deployment of Kubernetes cluster and plugins.
  18
  19 The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
  20 and container technology.
  21
  22 Version history
  23
  24 +--------------------+--------------------+--------------------+--------------------+
  25 | **Date**           | **Ver.**           | **Author**         | **Comment**        |
  26 |                    |                    |                    |                    |
  27 +--------------------+--------------------+--------------------+--------------------+
  28 | 2019-11-02         | 1.0.0              | Jackie Huang       | Initail version    |
  29 |                    |                    |                    |                    |
  30 +--------------------+--------------------+--------------------+--------------------+
  31 | 2020-06-03         | 2.0.0 (Bronze)     | Xiaohua Zhang      | Bronze version     |
  32 |                    |                    |                    |                    |
  33 +--------------------+--------------------+--------------------+--------------------+
  34 |                    |                    |                    |                    |
  35 |                    |                    |                    |                    |
  36 +--------------------+--------------------+--------------------+--------------------+
  37
  38
  39 Preface
  40 -------
  41
  42 Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
  43
  44
  45 Hardware Requirements
  46 ---------------------
  47
  48 Following minimum hardware requirements must be met for installation of O-RAN INF image:
  49
  50 +--------------------+----------------------------------------------------+
  51 | **HW Aspect**      | **Requirement**                                    |
  52 |                    |                                                    |
  53 +--------------------+----------------------------------------------------+
  54 | **# of servers**   | 1                                                  |
  55 +--------------------+----------------------------------------------------+
  56 | **CPU**            | 4                                                  |
  57 |                    |                                                    |
  58 +--------------------+----------------------------------------------------+
  59 | **RAM**            | 16G                                                |
  60 |                    |                                                    |
  61 +--------------------+----------------------------------------------------+
  62 | **Disk**           | 500G * 2                                           |
  63 |                    |                                                    |
  64 +--------------------+----------------------------------------------------+
  65 | **NICs**           | 2                                                  |
  66 |                    |                                                    |
  67 +--------------------+----------------------------------------------------+
  68
  69
  70
  71 Software Installation and Deployment
  72 ------------------------------------
  73
  74 1. Installation from the O-RAN INF ISO image
  75 ````````````````````````````````````````````
  76
  77 - Please see the README.md file for how to build the image.
  78 - The Image is a live ISO image with CLI installer: inf-image-aio-installer-intel-corei7-64.iso
  79
  80 1.1 Burn the image to the USB device
  81 ''''''''''''''''''''''''''''''''''''
  82
  83 - Assume the the usb device is /dev/sdX here
  84
  85 ::
  86
  87   $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
  88
  89 1.2 Install the server
  90 ''''''''''''''''''''''
  91
  92 - Reboot the target from the USB device.
  93
  94 - Select "Graphics console install" or "Serial console install" and press ENTER
  95
  96 - Remove the USB device and press ENTER to reboot
  97
  98
  99 2. Configuration and initialize the bootstrap
 100 `````````````````````````````````````````````
 101
 102 2.1 First Login
 103 ```````````````
 104 Example:
 105
 106 ::
 107
 108     ifconfig eno1 128.224.180.14/24 up
 109
 110     ip route add default via 128.224.180.1
 111
 112     ping 8.8.8.8
 113
 114 2.3 Login the server through SSH
 115 ````````````````````````````````
 116
 117 2.4 Prepare the localhost.yml for bootstrap
 118 ```````````````````````````````````````````
 119
 120 Example:
 121
 122 ::
 123
 124     system_mode: simplex
 125
 126     external_oam_subnet: 128.224.180.0/24
 127
 128     external_oam_gateway_address: 128.224.180.1
 129
 130     external_oam_floating_address: 128.224.180.14
 131
 132     admin_username: admin
 133
 134     admin_password: Wind123!
 135
 136     ansible_become_pass: Wind123!
 137
 138     docker_http_proxy: http://xxx.xxx.xxx.xxx:8080
 139
 140     docker_https_proxy: https://xxx.xxx.xxx.xxx:8080
 141
 142 2.5 Run the bootstrap
 143 `````````````````````
 144
 145 Use the following command:
 146
 147 ::
 148
 149     ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
 150
 151 After the bootstrap successfully finish, it will show as following:
 152
 153 ::
 154
 155     PLAY RECAP ******************************************************************************************************************************************
 156     localhost                  : ok=256  changed=135  unreachable=0    failed=0    skipped=220  rescued=0    ignored=0
 157
 158 2.6 Configure the OAM network
 159 `````````````````````````````
 160
 161 Example:
 162
 163 ::
 164
 165   controller-0:~$ source /etc/platform/openrc
 166
 167   [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno1
 168
 169   [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -c platform
 170
 171   +-----------------+--------------------------------------+
 172   | Property        | Value                                |
 173   +-----------------+--------------------------------------+
 174   | ifname          | eno1                                 |
 175   +-----------------+--------------------------------------+
 176   | iftype          | ethernet                             |
 177   +-----------------+--------------------------------------+
 178   | ports           | [u'eno1']                            |
 179   +-----------------+--------------------------------------+
 180   | imac            | 3c:a8:2a:1d:31:d0                    |
 181   +-----------------+--------------------------------------+
 182   | imtu            | 1500                                 |
 183   +-----------------+--------------------------------------+
 184   | ifclass         | platform                             |
 185   +-----------------+--------------------------------------+
 186   | aemode          | None                                 |
 187   +-----------------+--------------------------------------+
 188   | schedpolicy     | None                                 |
 189   +-----------------+--------------------------------------+
 190   | txhashpolicy    | None                                 |
 191   +-----------------+--------------------------------------+
 192   | uuid            | 95a32d13-01fc-43d2-90ab-801970fb3859 |
 193   +-----------------+--------------------------------------+
 194   | ihost_uuid      | 08c23843-afa4-4b80-a285-be126365276e |
 195   +-----------------+--------------------------------------+
 196   | vlan_id         | None                                 |
 197   +-----------------+--------------------------------------+
 198   | uses            | []                                   |
 199   +-----------------+--------------------------------------+
 200   | used_by         | []                                   |
 201   +-----------------+--------------------------------------+
 202   | created_at      | 2020-06-03T08:21:17.526303+00:00     |
 203   +-----------------+--------------------------------------+
 204   | updated_at      | 2020-06-03T10:02:54.682501+00:00     |
 205   +-----------------+--------------------------------------+
 206   | sriov_numvfs    | 0                                    |
 207   +-----------------+--------------------------------------+
 208   | sriov_vf_driver | None                                 |
 209   +-----------------+--------------------------------------+
 210   | accelerated     | [False]                              |
 211   +-----------------+--------------------------------------+
 212
 213   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 $OAM_IF oam
 214
 215   +--------------+--------------------------------------+
 216   | Property     | Value                                |
 217   +--------------+--------------------------------------+
 218   | hostname     | controller-0                         |
 219   +--------------+--------------------------------------+
 220   | uuid         | 32592cc3-9d66-4961-a51e-23686da00290 |
 221   +--------------+--------------------------------------+
 222   | ifname       | eno1                                 |
 223   +--------------+--------------------------------------+
 224   | network_name | oam                                  |
 225   +--------------+--------------------------------------+
 226
 227
 228   [sysadmin@controller-0 ~(keystone_admin)]$
 229   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-list controller-0
 230
 231   +--------------+--------------------------------------+--------+--------------+
 232   | hostname     | uuid                                 | ifname | network_name |
 233   +--------------+--------------------------------------+--------+--------------+
 234   | controller-0 | 2478f4c0-a7b4-4656-aa2a-97a0e8891955 | lo     | mgmt         |
 235   +--------------+--------------------------------------+--------+--------------+
 236   | controller-0 | 32592cc3-9d66-4961-a51e-23686da00290 | eno1   | oam          |
 237   +--------------+--------------------------------------+--------+--------------+
 238   | controller-0 | 43721e4d-4f52-4b4b-bd02-0efa00ef2124 | lo     | cluster-host |
 239   +--------------+--------------------------------------+--------+--------------+
 240
 241 2.7 Configure the NTP server
 242 ````````````````````````````
 243
 244 Example:
 245
 246 ::
 247
 248   [sysadmin@controller-0 ~(keystone_admin)]$
 249   [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
 250
 251   +--------------+--------------------------------------+
 252   | Property     | Value                                |
 253   +--------------+--------------------------------------+
 254   | uuid         | 8fbe5712-6ba0-4e61-b174-ed60f152f616 |
 255   +--------------+--------------------------------------+
 256   | ntpservers   | 0.pool.ntp.org,1.pool.ntp.org        |
 257   +--------------+--------------------------------------+
 258   | isystem_uuid | 802ee0be-3728-4df4-b9f7-1533c67fd960 |
 259   +--------------+--------------------------------------+
 260   | created_at   | 2020-06-03T08:19:47.812249+00:00     |
 261   +--------------+--------------------------------------+
 262   | updated_at   | None                                 |
 263   +--------------+--------------------------------------+
 264
 265 2.8 Configure the Storage by using the second hard drive as ceph backend
 266 ````````````````````````````````````````````````````````````````````````
 267
 268 Example:
 269
 270 ::
 271
 272   [sysadmin@controller-0 ~(keystone_admin)]$
 273
 274   [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
 275
 276   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 277   | uuid                                 | device_no | device_ | device_ | size_ | available_ | rpm          | serial_ | device_path                 |
 278   |                                      | de        | num     | type    | gib   | gib        |              | id      |                             |
 279   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 280   | 3884f906-cb9b-4052-bcb8-8be70e599b9f | /dev/sda  | 2048    | HDD     | 1117. | 868.759    | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
 281   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 282   |                                      |           |         |         | 782   |            |              | RH8W4QQ | 03:00.0-scsi-0:1:0:0        |
 283   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 284   |                                      |           |         |         |       |            |              |         |                             |
 285   | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/sdb  | 2064    | HDD     | 1117. | 0.0        | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
 286   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 287   |                                      |           |         |         | 782   |            |              | RH8W4QQ | 03:00.0-scsi-0:1:0:1        |
 288   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 289   |                                      |           |         |         |       |            |              |         |                             |
 290   +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
 291
 292   [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
 293
 294   +------------------+-------------------------------------------------------+
 295   | Property         | Value                                                 |
 296   +------------------+-------------------------------------------------------+
 297   | osdid            | 0                                                     |
 298   +------------------+-------------------------------------------------------+
 299   | function         | osd                                                   |
 300   +------------------+-------------------------------------------------------+
 301   | state            | configuring-on-unlock                                 |
 302   +------------------+-------------------------------------------------------+
 303   | journal_location | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf                  |
 304   +------------------+-------------------------------------------------------+
 305   | journal_size_gib | 1024                                                  |
 306   +------------------+-------------------------------------------------------+
 307   | journal_path     | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 |
 308   +------------------+-------------------------------------------------------+
 309   | journal_node     | /dev/sdb2                                             |
 310   +------------------+-------------------------------------------------------+
 311   | uuid             | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf                  |
 312   +------------------+-------------------------------------------------------+
 313   | ihost_uuid       | 08c23843-afa4-4b80-a285-be126365276e                  |
 314   +------------------+-------------------------------------------------------+
 315   | idisk_uuid       | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9                  |
 316   +------------------+-------------------------------------------------------+
 317   | tier_uuid        | db6eabf9-9922-4d8e-9740-c0f9878e692d                  |
 318   +------------------+-------------------------------------------------------+
 319   | tier_name        | storage                                               |
 320   +------------------+-------------------------------------------------------+
 321   | created_at       | 2020-06-03T10:03:55.645744+00:00                      |
 322   +------------------+-------------------------------------------------------+
 323   | updated_at       | None                                                  |
 324   +------------------+-------------------------------------------------------+
 325
 326   [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
 327
 328   +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 329   | uuid                                 | function | osdid | state                 | idisk_uuid                           | journal_path                                          | journal_node | journal_size_gib | tier_name |
 330   +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 331   | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf | osd      | 0     | configuring-on-unlock | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 | /dev/sdb2    | 1                | storage   |
 332   +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
 333
 334 2.9 Configure the SR-IOV enabled network
 335 ````````````````````````````````````````
 336 It need :
 337   - assign the label of SR-IOV
 338   - change the CPU policy to static
 339   - change the hugepage setting
 340   - create the data network
 341   - assign the SR-IOV interface to data network
 342
 343 Example:
 344
 345 ::
 346
 347   [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
 348
 349   [sysadmin@controller-0 ~(keystone_admin)]$
 350
 351   [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER sriovdp=enabled
 352
 353   +-------------+--------------------------------------+
 354   | Property    | Value                                |
 355   +-------------+--------------------------------------+
 356   | uuid        | 7101010d-1897-4cc8-8c87-7dffaebf848b |
 357   +-------------+--------------------------------------+
 358   | host_uuid   | 08c23843-afa4-4b80-a285-be126365276e |
 359   +-------------+--------------------------------------+
 360   | label_key   | sriovdp                              |
 361   +-------------+--------------------------------------+
 362   | label_value | enabled                              |
 363   +-------------+--------------------------------------+
 364
 365   [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER kube-cpu-mgr-policy=static
 366
 367   +-------------+--------------------------------------+
 368   | Property    | Value                                |
 369   +-------------+--------------------------------------+
 370   | uuid        | 4936f32b-6939-4115-b643-846bd50e7738 |
 371   +-------------+--------------------------------------+
 372   | host_uuid   | 08c23843-afa4-4b80-a285-be126365276e |
 373   +-------------+--------------------------------------+
 374   | label_key   | kube-cpu-mgr-policy                  |
 375   +-------------+--------------------------------------+
 376   | label_value | static                               |
 377   +-------------+--------------------------------------+
 378
 379
 380   [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7  0
 381
 382   +-------------------------------------+--------------------------------------+
 383   | Property                            | Value                                |
 384   +-------------------------------------+--------------------------------------+
 385   | Memory: Usable Total (MiB)          | 0                                    |
 386   +-------------------------------------+--------------------------------------+
 387   |         Platform     (MiB)          | 7000                                 |
 388   +-------------------------------------+--------------------------------------+
 389   |         Available    (MiB)          | 0                                    |
 390   +-------------------------------------+--------------------------------------+
 391   | Huge Pages Configured               | True                                 |
 392   +-------------------------------------+--------------------------------------+
 393   | vSwitch Huge Pages: Size (MiB)      | 1024                                 |
 394   +-------------------------------------+--------------------------------------+
 395   |                     Total           | 0                                    |
 396   +-------------------------------------+--------------------------------------+
 397   |                     Available       | 0                                    |
 398   +-------------------------------------+--------------------------------------+
 399   |                     Required        | None                                 |
 400   +-------------------------------------+--------------------------------------+
 401   | Application  Pages (4K): Total      | None                                 |
 402   +-------------------------------------+--------------------------------------+
 403   | Application  Huge Pages (2M): Total | 0                                    |
 404   +-------------------------------------+--------------------------------------+
 405   |                 Available           | 0                                    |
 406   +-------------------------------------+--------------------------------------+
 407   | Application  Huge Pages (1G): Total | 0                                    |
 408   +-------------------------------------+--------------------------------------+
 409   |                 Total Pending       | 7                                    |
 410   +-------------------------------------+--------------------------------------+
 411   |                 Available           | 0                                    |
 412   +-------------------------------------+--------------------------------------+
 413   | uuid                                | 15702f44-2281-41f3-8c83-a1e9a300ad01 |
 414   +-------------------------------------+--------------------------------------+
 415   | ihost_uuid                          | 08c23843-afa4-4b80-a285-be126365276e |
 416   +-------------------------------------+--------------------------------------+
 417   | inode_uuid                          | d52648bc-dfa6-4214-afa3-5d2148ee4180 |
 418   +-------------------------------------+--------------------------------------+
 419   | created_at                          | 2020-06-03T08:21:27.468357+00:00     |
 420   +-------------------------------------+--------------------------------------+
 421   | updated_at                          | 2020-06-03T10:10:49.159080+00:00     |
 422   +-------------------------------------+--------------------------------------+
 423
 424
 425   [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7 1
 426
 427   +-------------------------------------+--------------------------------------+
 428   | Property                            | Value                                |
 429   +-------------------------------------+--------------------------------------+
 430   | Memory: Usable Total (MiB)          | 0                                    |
 431   +-------------------------------------+--------------------------------------+
 432   |         Platform     (MiB)          | 1000                                 |
 433   +-------------------------------------+--------------------------------------+
 434   |         Available    (MiB)          | 0                                    |
 435   +-------------------------------------+--------------------------------------+
 436   | Huge Pages Configured               | True                                 |
 437   +-------------------------------------+--------------------------------------+
 438   | vSwitch Huge Pages: Size (MiB)      | 1024                                 |
 439   +-------------------------------------+--------------------------------------+
 440   |                     Total           | 0                                    |
 441   +-------------------------------------+--------------------------------------+
 442   |                     Available       | 0                                    |
 443   +-------------------------------------+--------------------------------------+
 444   |                     Required        | None                                 |
 445   +-------------------------------------+--------------------------------------+
 446   | Application  Pages (4K): Total      | None                                 |
 447   +-------------------------------------+--------------------------------------+
 448   | Application  Huge Pages (2M): Total | 0                                    |
 449   +-------------------------------------+--------------------------------------+
 450   |                 Available           | 0                                    |
 451   +-------------------------------------+--------------------------------------+
 452   | Application  Huge Pages (1G): Total | 0                                    |
 453   +-------------------------------------+--------------------------------------+
 454   |                 Total Pending       | 7                                    |
 455   +-------------------------------------+--------------------------------------+
 456   |                 Available           | 0                                    |
 457   +-------------------------------------+--------------------------------------+
 458   | uuid                                | 544580b9-0f2a-41a1-b4e3-90fdf5c8f7e0 |
 459   +-------------------------------------+--------------------------------------+
 460   | ihost_uuid                          | 08c23843-afa4-4b80-a285-be126365276e |
 461   +-------------------------------------+--------------------------------------+
 462   | inode_uuid                          | e02c3c41-03b7-4f49-8244-bca851fa2d2f |
 463   +-------------------------------------+--------------------------------------+
 464   | created_at                          | 2020-06-03T08:21:27.566279+00:00     |
 465   +-------------------------------------+--------------------------------------+
 466   | updated_at                          | 2020-06-03T10:10:49.220907+00:00     |
 467   +-------------------------------------+--------------------------------------+
 468
 469
 470   [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET0='physnet0'
 471
 472   [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET1='physnet1'
 473
 474   [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
 475
 476   [sysadmin@controller-0 ~(keystone_admin)]$
 477
 478   [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET0} flat
 479
 480   +--------------+--------------------------------------+
 481   | Property     | Value                                |
 482   +--------------+--------------------------------------+
 483   | id           | 1                                    |
 484   +--------------+--------------------------------------+
 485   | uuid         | 0ffe0b0a-35ff-45b2-9a62-63a494d2d18b |
 486   +--------------+--------------------------------------+
 487   | name         | physnet0                             |
 488   +--------------+--------------------------------------+
 489   | network_type | flat                                 |
 490   +--------------+--------------------------------------+
 491   | mtu          | 1500                                 |
 492   +--------------+--------------------------------------+
 493   | description  | None                                 |
 494   +--------------+--------------------------------------+
 495
 496   [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET1} flat
 497
 498   +--------------+--------------------------------------+
 499   | Property     | Value                                |
 500   +--------------+--------------------------------------+
 501   | id           | 2                                    |
 502   +--------------+--------------------------------------+
 503   | uuid         | 899ca8ba-6800-4d62-aed4-3770878da738 |
 504   +--------------+--------------------------------------+
 505   | name         | physnet1                             |
 506   +--------------+--------------------------------------+
 507   | network_type | flat                                 |
 508   +--------------+--------------------------------------+
 509   | mtu          | 1500                                 |
 510   +--------------+--------------------------------------+
 511   | description  | None                                 |
 512   +--------------+--------------------------------------+
 513
 514
 515   [sysadmin@controller-0 ~(keystone_admin)]$ system host-port-list controller-0
 516
 517   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 518   | uuid                                 | name   | type     | pci address  | device | processor | accelerated | device type                                           |
 519   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 520   | 7007dfb1-ea55-4355-9d27-3ed4c6696283 | eno1   | ethernet | 0000:02:00.0 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 521   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 522   | 51f62ecf-7897-43e3-beb8-46ae5550ef05 | eno2   | ethernet | 0000:02:00.1 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 523   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 524   | e987725e-ab45-4151-b53d-94ba21706eb4 | eno3   | ethernet | 0000:02:00.2 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 525   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 526   | f4a8115a-ca0b-4336-a634-2cad77c5a88c | eno4   | ethernet | 0000:02:00.3 | 0      | 0         | False       | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657]        |
 527   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 528   | 88965882-c816-42e8-ba0e-d3ff6f9f553e | eno49  | ethernet | 0000:04:00.0 | 0      | 0         | False       | OneConnect NIC (Skyhawk) [0720]                       |
 529   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 530   | 98e2cedc-9ef8-4ca0-b3e0-db3a2bd0273a | eno50  | ethernet | 0000:04:00.1 | 0      | 0         | False       | OneConnect NIC (Skyhawk) [0720]                       |
 531   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 532   | e75d7aa6-7832-4ec4-9dd4-ea8300773fd8 | ens2f0 | ethernet | 0000:05:00.0 | 0      | 0         | True        | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
 533   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 534   | 8ac4ffca-4a61-4b47-8e17-171f2e6cea66 | ens2f1 | ethernet | 0000:05:00.1 | 0      | 0         | True        | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
 535   +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
 536
 537
 538   [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f0
 539
 540   [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov0 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
 541
 542   +-----------------+--------------------------------------+
 543   | Property        | Value                                |
 544   +-----------------+--------------------------------------+
 545   | ifname          | sriov0                               |
 546   +-----------------+--------------------------------------+
 547   | iftype          | ethernet                             |
 548   +-----------------+--------------------------------------+
 549   | ports           | [u'ens2f0']                          |
 550   +-----------------+--------------------------------------+
 551   | imac            | 00:1b:21:b3:46:2c                    |
 552   +-----------------+--------------------------------------+
 553   | imtu            | 1500                                 |
 554   +-----------------+--------------------------------------+
 555   | ifclass         | pci-sriov                            |
 556   +-----------------+--------------------------------------+
 557   | aemode          | None                                 |
 558   +-----------------+--------------------------------------+
 559   | schedpolicy     | None                                 |
 560   +-----------------+--------------------------------------+
 561   | txhashpolicy    | None                                 |
 562   +-----------------+--------------------------------------+
 563   | uuid            | 9450b1ae-d1a1-4ecb-811a-fa3cb1cdf362 |
 564   +-----------------+--------------------------------------+
 565   | ihost_uuid      | 08c23843-afa4-4b80-a285-be126365276e |
 566   +-----------------+--------------------------------------+
 567   | vlan_id         | None                                 |
 568   +-----------------+--------------------------------------+
 569   | uses            | []                                   |
 570   +-----------------+--------------------------------------+
 571   | used_by         | []                                   |
 572   +-----------------+--------------------------------------+
 573   | created_at      | 2020-06-03T12:34:11.633254+00:00     |
 574   +-----------------+--------------------------------------+
 575   | updated_at      | 2020-06-04T02:01:28.168127+00:00     |
 576   +-----------------+--------------------------------------+
 577   | sriov_numvfs    | 6                                    |
 578   +-----------------+--------------------------------------+
 579   | sriov_vf_driver | vfio                                 |
 580   +-----------------+--------------------------------------+
 581   | accelerated     | [True]                               |
 582   +-----------------+--------------------------------------+
 583
 584   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov0 ${PHYSNET0}
 585
 586   +------------------+--------------------------------------+
 587   | Property         | Value                                |
 588   +------------------+--------------------------------------+
 589   | hostname         | controller-0                         |
 590   +------------------+--------------------------------------+
 591   | uuid             | 1d35219d-3258-42e7-8f0e-41d865710d58 |
 592   +------------------+--------------------------------------+
 593   | ifname           | sriov0                               |
 594   +------------------+--------------------------------------+
 595   | datanetwork_name | physnet0                             |
 596   +------------------+--------------------------------------+
 597
 598
 599
 600   [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f1
 601
 602   [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov1 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
 603
 604   +-----------------+--------------------------------------+
 605   | Property        | Value                                |
 606   +-----------------+--------------------------------------+
 607   | ifname          | sriov1                               |
 608   +-----------------+--------------------------------------+
 609   | iftype          | ethernet                             |
 610   +-----------------+--------------------------------------+
 611   | ports           | [u'ens2f1']                          |
 612   +-----------------+--------------------------------------+
 613   | imac            | 00:1b:21:b3:46:2d                    |
 614   +-----------------+--------------------------------------+
 615   | imtu            | 1500                                 |
 616   +-----------------+--------------------------------------+
 617   | ifclass         | pci-sriov                            |
 618   +-----------------+--------------------------------------+
 619   | aemode          | None                                 |
 620   +-----------------+--------------------------------------+
 621   | schedpolicy     | None                                 |
 622   +-----------------+--------------------------------------+
 623   | txhashpolicy    | None                                 |
 624   +-----------------+--------------------------------------+
 625   | uuid            | 13d5ee99-0399-4b6b-a359-0f96660afdc4 |
 626   | ihost_uuid      | 08c23843-afa4-4b80-a285-be126365276e |
 627   | vlan_id         | None                                 |
 628   | uses            | []                                   |
 629   | used_by         | []                                   |
 630   | created_at      | 2020-06-03T12:34:12.072624+00:00     |
 631   | updated_at      | 2020-06-04T02:03:51.405322+00:00     |
 632   | sriov_numvfs    | 6                                    |
 633   | sriov_vf_driver | vfio                                 |
 634   | accelerated     | [True]                               |
 635   +-----------------+--------------------------------------+
 636
 637   [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov1 ${PHYSNET1}
 638
 639   +------------------+--------------------------------------+
 640   | Property         | Value                                |
 641   +------------------+--------------------------------------+
 642   | hostname         | controller-0                         |
 643   +------------------+--------------------------------------+
 644   | uuid             | a87c6a5f-0c53-4ebf-9a32-38366700426f |
 645   +------------------+--------------------------------------+
 646   | ifname           | sriov1                               |
 647   +------------------+--------------------------------------+
 648   | datanetwork_name | physnet1                             |
 649   +------------------+--------------------------------------+
 650
 651 2.10 Unlock the server
 652 ``````````````````````
 653
 654 Example:
 655
 656 ::
 657
 658   [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
 659
 660   +-----------------------+-------------------------------------------------+
 661   | Property              | Value                                           |
 662   +-----------------------+-------------------------------------------------+
 663   | action                | none                                            |
 664   +-----------------------+-------------------------------------------------+
 665   | administrative        | locked                                          |
 666   +-----------------------+-------------------------------------------------+
 667   | availability          | online                                          |
 668   +-----------------------+-------------------------------------------------+
 669   | bm_ip                 | None                                            |
 670   +-----------------------+-------------------------------------------------+
 671   | bm_type               | none                                            |
 672   +-----------------------+-------------------------------------------------+
 673   | bm_username           | None                                            |
 674   +-----------------------+-------------------------------------------------+
 675   | boot_device           | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
 676   +-----------------------+-------------------------------------------------+
 677   | capabilities          | {u'stor_function': u'monitor'}                  |
 678   +-----------------------+-------------------------------------------------+
 679   | clock_synchronization | ntp                                             |
 680   +-----------------------+-------------------------------------------------+
 681   | config_applied        | 3daf20c7-c032-4aa2-839d-93da92051bea            |
 682   +-----------------------+-------------------------------------------------+
 683   | config_status         | Config out-of-date                              |
 684   +-----------------------+-------------------------------------------------+
 685   | config_target         | bdaf20c7-c032-4aa2-839d-93da92051bea            |
 686   +-----------------------+-------------------------------------------------+
 687   | console               | tty0                                            |
 688   +-----------------------+-------------------------------------------------+
 689   | created_at            | 2020-06-03T08:20:59.372602+00:00                |
 690   +-----------------------+-------------------------------------------------+
 691   | hostname              | controller-0                                    |
 692   +-----------------------+-------------------------------------------------+
 693   | id                    | 1                                               |
 694   +-----------------------+-------------------------------------------------+
 695   | install_output        | graphical                                       |
 696   +-----------------------+-------------------------------------------------+
 697   | install_state         | None                                            |
 698   +-----------------------+-------------------------------------------------+
 699   | install_state_info    | None                                            |
 700   +-----------------------+-------------------------------------------------+
 701   | inv_state             | inventoried                                     |
 702   +-----------------------+-------------------------------------------------+
 703   | invprovision          | provisioning                                    |
 704   +-----------------------+-------------------------------------------------+
 705   | location              | {}                                              |
 706   +-----------------------+-------------------------------------------------+
 707   | mgmt_ip               | 192.168.204.2                                   |
 708   +-----------------------+-------------------------------------------------+
 709   | mgmt_mac              | 00:00:00:00:00:00                               |
 710   +-----------------------+-------------------------------------------------+
 711   | operational           | disabled                                        |
 712   +-----------------------+-------------------------------------------------+
 713   | personality           | controller                                      |
 714   +-----------------------+-------------------------------------------------+
 715   | reserved              | False                                           |
 716   +-----------------------+-------------------------------------------------+
 717   | rootfs_device         | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
 718   +-----------------------+-------------------------------------------------+
 719   | serialid              | None                                            |
 720   +-----------------------+-------------------------------------------------+
 721   | software_load         | 19.12                                           |
 722   +-----------------------+-------------------------------------------------+
 723   | subfunction_avail     | online                                          |
 724   +-----------------------+-------------------------------------------------+
 725   | subfunction_oper      | disabled                                        |
 726   +-----------------------+-------------------------------------------------+
 727   | subfunctions          | controller,worker                               |
 728   +-----------------------+-------------------------------------------------+
 729   | task                  | Unlocking                                       |
 730   +-----------------------+-------------------------------------------------+
 731   | tboot                 | false                                           |
 732   +-----------------------+-------------------------------------------------+
 733   | ttys_dcd              | None                                            |
 734   +-----------------------+-------------------------------------------------+
 735   | updated_at            | 2020-06-04T02:04:59.514854+00:00                |
 736   +-----------------------+-------------------------------------------------+
 737   | uptime                | 49475                                           |
 738   +-----------------------+-------------------------------------------------+
 739   | uuid                  | 08c23843-afa4-4b80-a285-be126365276e            |
 740   +-----------------------+-------------------------------------------------+
 741   | vim_progress_status   | None                                            |
 742   +-----------------------+-------------------------------------------------+
 743
 744 3. Simple use case for sriov
 745 ````````````````````````````
 746
 747 3.1 After controller-0 is rebooted and up running, download the DPDK
 748 ````````````````````````````````````````````````````````````````````
 749
 750 ::
 751
 752   [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
 753   [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
 754   Password:
 755   --2020-06-04 02:35:30--  https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
 756   Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
 757   Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
 758
 759   HTTP request sent, awaiting response... 200 OK
 760   Length: 10251680 (9.8M) [application/octet-stream]
 761   Saving to: ‘dpdk-17.11.10.tar.xz’
 762
 763   dpdk-17.11.10.tar.xz                        100%
 764   [========================================================================================>]   9.78M
 765   1.48MB/s    in 6.8s
 766
 767   2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
 768
 769   sudo tar xvf dpdk-17.11.10.tar.xz
 770
 771   sudo ln -s dpdk-stable-17.11.10 dpdk-stable
 772
 773 3.1 Prepare the yaml file for the network assignment container
 774 ``````````````````````````````````````````````````````````````
 775
 776 The following the exmaple of the yaml file:
 777
 778 ::
 779
 780   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
 781   > apiVersion: "k8s.cni.cncf.io/v1"
 782   > kind: NetworkAttachmentDefinition
 783   > metadata:
 784   >   name: sriov-data-dpdk-0
 785   >   annotations:
 786   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
 787   > spec:
 788   >   config: '{
 789   >   "type": "sriov",
 790   >   "name": "sriov-data-dpdk-0"
 791   > }'
 792   >
 793   > ---
 794   > apiVersion: "k8s.cni.cncf.io/v1"
 795   > kind: NetworkAttachmentDefinition
 796   > metadata:
 797   >   name: sriov-data-dpdk-1
 798   >   annotations:
 799   >     k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
 800   > spec:
 801   >   config: '{
 802   >   "type": "sriov",
 803   >   "name": "sriov-data-dpdk-1"
 804   > }'
 805   > EOF
 806
 807 3.2 Run the network assignent container for the 2 VFs
 808 `````````````````````````````````````````````````````
 809
 810 ::
 811
 812   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
 813   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
 814   networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
 815
 816 3.3 Prepare the VF container yaml file
 817 ``````````````````````````````````````
 818
 819 ::
 820
 821   [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
 822   > apiVersion: v1
 823   > kind: Pod
 824   metadata:
 825   > metadata:
 826   >   name: pod-with-dpdk-vfs-0
 827   >   annotations:
 828   >     k8s.v1.cni.cncf.io/networks: '[
 829   >             { "name": "sriov-data-dpdk-0" },
 830               { "name": "sriov-data-dpdk-1" }
 831   >             { "name": "sriov-data-dpdk-1" }
 832   >     ]'
 833   > spec:
 834   >   restartPolicy: Never
 835   >   containers:
 836   >   - name: pod-with-dpdk-vfs-0
 837   >     image: wrsnfv/ubuntu-dpdk-build:v0.3
 838   >     env:
 839   >     - name: RTE_SDK
 840   >       value: "/usr/src/dpdk"
 841   >     command:
 842   >     - sleep
 843   >     - infinity
 844   >     stdin: true
 845   >     tty: true
 846   >     securityContext:
 847   >       privileged: true
 848   >       capabilities:
 849   >         add:
 850   >         - ALL
 851   >     resources:
 852   >       requests:
 853   >         cpu: 4
 854   >         memory: 4Gi
 855   >         intel.com/pci_sriov_net_physnet0: '1'
 856   >         intel.com/pci_sriov_net_physnet1: '1'
 857   >       limits:
 858   >         cpu: 4
 859   >         hugepages-1Gi: 2Gi
 860   >         memory: 4Gi
 861   >         intel.com/pci_sriov_net_physnet0: '1'
 862   >         intel.com/pci_sriov_net_physnet1: '1'
 863   >     volumeMounts:
 864   >     - mountPath: /mnt/huge-1048576kB
 865   >       name: hugepage
 866   >     - name: dpdk-volume
 867   >       mountPath: /usr/src/dpdk
 868   >     - name: lib-volume
 869   >       mountPath: /lib/modules
 870   >     - name: src-volume
 871   >       mountPath: /usr/src/
 872   >   volumes:
 873   >   - name: hugepage
 874   >     emptyDir:
 875   >       medium: HugePages
 876   >   - name: dpdk-volume
 877   >     hostPath:
 878   >       path: /opt/dpdk-stable/
 879   >   - name: lib-volume
 880   >     hostPath:
 881   >       path: /lib/modules
 882   >   - name: src-volume
 883   >     hostPath:
 884   >       path: /usr/src/
 885   > EOF
 886
 887 3.4 Run the VF container
 888 ````````````````````````
 889
 890 Start the VF container.
 891
 892 ::
 893
 894   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
 895   pod/pod-with-dpdk-vfs-0 created
 896
 897   [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
 898   NAME                  READY   STATUS    RESTARTS   AGE
 899   pod-with-dpdk-vfs-0   1/1     Running   0          6m40s
 900
 901 Login the VF container
 902
 903 ::
 904
 905   kubectl exec -it pod-with-dpdk-vfs-0 -- bash
 906
 907 Build the DPDK
 908
 909 ::
 910
 911   cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
 912
 913   root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
 914     HOSTCC  scripts/basic/fixdep
 915     HOSTCC  scripts/kconfig/conf.o
 916     HOSTCC  scripts/kconfig/confdata.o
 917     HOSTCC  scripts/kconfig/expr.o
 918     HOSTCC  scripts/kconfig/symbol.o
 919     HOSTCC  scripts/kconfig/preprocess.o
 920     HOSTCC  scripts/kconfig/zconf.lex.o
 921     HOSTCC  scripts/kconfig/zconf.tab.o
 922     HOSTLD  scripts/kconfig/conf
 923   scripts/kconfig/conf  --syncconfig Kconfig
 924     HOSTCC  arch/x86/tools/relocs_32.o
 925     HOSTCC  arch/x86/tools/relocs_64.o
 926     HOSTCC  arch/x86/tools/relocs_common.o
 927     HOSTLD  arch/x86/tools/relocs
 928     HOSTCC  scripts/genksyms/genksyms.o
 929     YACC    scripts/genksyms/parse.tab.c
 930     HOSTCC  scripts/genksyms/parse.tab.o
 931     LEX     scripts/genksyms/lex.lex.c
 932     YACC    scripts/genksyms/parse.tab.h
 933     HOSTCC  scripts/genksyms/lex.lex.o
 934     HOSTLD  scripts/genksyms/genksyms
 935     HOSTCC  scripts/bin2c
 936     HOSTCC  scripts/kallsyms
 937     HOSTCC  scripts/conmakehash
 938     HOSTCC  scripts/recordmcount
 939     HOSTCC  scripts/sortextable
 940     HOSTCC  scripts/asn1_compiler
 941     HOSTCC  scripts/sign-file
 942     HOSTCC  scripts/extract-cert
 943     CC      scripts/mod/empty.o
 944     HOSTCC  scripts/mod/mk_elfconfig
 945     MKELF   scripts/mod/elfconfig.h
 946     HOSTCC  scripts/mod/modpost.o
 947     CC      scripts/mod/devicetable-offsets.s
 948     UPD     scripts/mod/devicetable-offsets.h
 949     HOSTCC  scripts/mod/file2alias.o
 950     HOSTCC  scripts/mod/sumversion.o
 951     HOSTLD  scripts/mod/modpost
 952     CC      kernel/bounds.s
 953     CC      arch/x86/kernel/asm-offsets.s
 954     CALL    scripts/checksyscalls.sh
 955
 956 Build the test_pmd application
 957
 958 ::
 959
 960   cd $RTE_SDK
 961   ./usertools/dpdk-setup.sh
 962   Option: 14
 963     CC config.o
 964     CC iofwd.o
 965     CC macfwd.o
 966     CC macswap.o
 967     CC flowgen.o
 968     CC rxonly.o
 969     CC txonly.o
 970     CC csumonly.o
 971     CC icmpecho.o
 972     CC tm.o
 973     LD testpmd
 974     INSTALL-APP testpmd
 975     INSTALL-MAP testpmd.map
 976   == Build app/proc_info
 977     CC main.o
 978     LD dpdk-procinfo
 979     INSTALL-APP dpdk-procinfo
 980     INSTALL-MAP dpdk-procinfo.map
 981   == Build app/pdump
 982     CC main.o
 983     LD dpdk-pdump
 984     INSTALL-APP dpdk-pdump
 985     INSTALL-MAP dpdk-pdump.map
 986   == Build app/test-crypto-perf
 987     CC main.o
 988     CC cperf_ops.o
 989     CC cperf_options_parsing.o
 990     CC cperf_test_vectors.o
 991     CC cperf_test_throughput.o
 992     CC cperf_test_latency.o
 993     CC cperf_test_pmd_cyclecount.o
 994     CC cperf_test_verify.o
 995     CC cperf_test_vector_parsing.o
 996     CC cperf_test_common.o
 997     LD dpdk-test-crypto-perf
 998     INSTALL-APP dpdk-test-crypto-perf
 999     INSTALL-MAP dpdk-test-crypto-perf.map
1000   == Build app/test-eventdev
1001     CC evt_main.o
1002     CC evt_options.o
1003     CC evt_test.o
1004     CC parser.o
1005     CC test_order_common.o
1006     CC test_order_queue.o
1007     CC test_order_atq.o
1008     CC test_perf_common.o
1009     CC test_perf_queue.o
1010     CC test_perf_atq.o
1011     LD dpdk-test-eventdev
1012     INSTALL-APP dpdk-test-eventdev
1013     INSTALL-MAP dpdk-test-eventdev.map
1014   Build complete [x86_64-native-linuxapp-gcc]
1015   Installation cannot run with T defined and DESTDIR undefined
1016   ------------------------------------------------------------------------------
1017   RTE_TARGET exported as x86_64-native-linuxapp-gcc
1018   ------------------------------------------------------------------------------
1019
1020   Press enter to continue ...
1021
1022 Check the VF PCI information:
1023 ::
1024
1025   root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
1026   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
1027   PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
1028
1029 Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
1030 ::
1031
1032   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
1033   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
1034   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
1035   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
1036   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
1037   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
1038   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
1039
1040   [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
1041   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
1042   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
1043   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
1044   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
1045   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
1046   lrwxrwxrwx 1 root root 0 Jun  4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
1047
1048   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
1049   [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
1050
1051
1052 Get back to pod and run the test_pmd
1053
1054 ::
1055
1056   cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
1057
1058   taskset -p --cpu-list 1
1059   pid 1's current affinity list: 2,3,18,19
1060
1061   ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
1062   start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
1063   peer=1,"9e:fd:e6:dd:c1:01"
1064   EAL: Detected 32 lcore(s)
1065   EAL: No free hugepages reported in hugepages-2048kB
1066   EAL: Probing VFIO support...
1067   EAL: VFIO support initialized
1068   EAL: PCI device 0000:05:11.0 on NUMA socket 0
1069   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1070   EAL:   using IOMMU type 1 (Type 1)
1071   EAL: PCI device 0000:05:11.1 on NUMA socket 0
1072   EAL:   probe driver: 8086:10ed net_ixgbe_vf
1073   Auto-start selected
1074   Ports to start sending a burst of packets first
1075   Warning: lsc_interrupt needs to be off when  using tx_first. Disabling.
1076   USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
1077   Configuring Port 0 (socket 0)
1078   Port 0: 9E:FD:E6:DD:C1:01
1079   Configuring Port 1 (socket 0)
1080   Port 1: 9E:FD:E6:DD:C1:02
1081   Checking link statuses...
1082   Port0 Link Up. speed 10000 Mbps- full-duplex
1083   Port1 Link Up. speed 10000 Mbps- full-duplex
1084   Done
1085   No commandline core given, start packet forwarding
1086   io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
1087   disabled
1088   Logical Core 3 (socket 0) forwards packets on 2 streams:
1089     RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
1090     RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
1091
1092     io packet forwarding packets/burst=32
1093     nb forwarding cores=1 - nb forwarding ports=2
1094     port 0:
1095     CRC stripping enabled
1096     RX queues=1 - RX desc=128 - RX free threshold=32
1097     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1098     TX queues=1 - TX desc=512 - TX free threshold=32
1099     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1100     TX RS bit threshold=32 - TXQ flags=0xf01
1101     port 1:
1102     CRC stripping enabled
1103     RX queues=1 - RX desc=128 - RX free threshold=32
1104     RX threshold registers: pthresh=8 hthresh=8  wthresh=0
1105     TX queues=1 - TX desc=512 - TX free threshold=32
1106     TX threshold registers: pthresh=32 hthresh=0  wthresh=0
1107     TX RS bit threshold=32 - TXQ flags=0xf01
1108
1109   Port statistics ====================================
1110     ######################## NIC statistics for port 0  ########################
1111     RX-packets: 56         RX-missed: 0          RX-bytes:  4096
1112     RX-errors: 0
1113     RX-nombuf:  0
1114     TX-packets: 64         TX-errors: 0          TX-bytes:  4096
1115
1116     Throughput (since last show)
1117     Rx-pps:            0
1118     Tx-pps:            0
1119     ############################################################################
1120
1121     ######################## NIC statistics for port 1  ########################
1122     RX-packets: 432        RX-missed: 0          RX-bytes:  27712
1123     RX-errors: 0
1124     RX-nombuf:  0
1125     TX-packets: 461        TX-errors: 0          TX-bytes:  30080
1126
1127     Throughput (since last show)
1128     Rx-pps:            0
1129     Tx-pps:            0
1130     ############################################################################
1131
1132   Port statistics ====================================
1133     ######################## NIC statistics for port 0  ########################
1134     RX-packets: 14124641   RX-missed: 0          RX-bytes:  903977344
1135     RX-errors: 0
1136     RX-nombuf:  0
1137     TX-packets: 14170205   TX-errors: 0          TX-bytes:  906893376
1138
1139     Throughput (since last show)
1140     Rx-pps:      7068409
1141     Tx-pps:      7091206
1142     ############################################################################
1143
1144
1145
1146
1147
1148
1149 References
1150 ----------
1151
1152 - `StarlingX`_
1153
1154 .. _`StarlingX`: https://docs.starlingx.io/
1155