1 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
2 .. SPDX-License-Identifier: CC-BY-4.0
3 .. Copyright (C) 2019 Wind River Systems, Inc.
16 This document describes how to install O-RAN INF image, example configuration for better
17 real time performance, and example deployment of Kubernetes cluster and plugins.
19 The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
20 and container technology.
26 Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
32 Following minimum hardware requirements must be met for installation of O-RAN INF image:
34 +--------------------+----------------------------------------------------+
35 | **HW Aspect** | **Requirement** |
37 +--------------------+----------------------------------------------------+
38 | **# of servers** | 1 |
39 +--------------------+----------------------------------------------------+
42 +--------------------+----------------------------------------------------+
45 +--------------------+----------------------------------------------------+
46 | **Disk** | 500G * 2 |
48 +--------------------+----------------------------------------------------+
51 +--------------------+----------------------------------------------------+
53 If want to enable the HA mode which is available in Cherry release, 2 servers are needed.
56 ORAN INF Cherry test on Dell R740
57 =================================
59 1. Installation for the first server from the O-RAN INF ISO image
60 -----------------------------------------------------------------
62 - Please see the README.md file for how to build the image.
63 - The Image is a live ISO image with CLI installer:
64 inf-image-aio-installer-intel-corei7-64.iso
66 1.1 Burn the image to the USB device
67 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
69 - Assume the the usb device is /dev/sdX here
73 $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
75 1.2 Install the first Dell-R740 server (controller-0)
76 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
78 - Reboot the target from the USB device.
80 - Select “All-in-one Graphics console” or “All-in-one Serial console
81 install” and press ENTER
83 - Start the auto installation
85 - It will reboot aotumatically after installation
87 2. Configuration and initialize the bootstrap
88 ---------------------------------------------
90 2.1 First Login with sysadmin/sysadmin and change password
91 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
93 2.2 Set OAM network before bootstrap
94 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
99 export CONTROLLER0_OAM_CIDR=128.224.210.110/24
100 export DEFAULT_OAM_GATEWAY=128.224.210.1
101 sudo ip address add $CONTROLLER0_OAM_CIDR dev $OAM_DEV
102 sudo ip link set up dev $OAM_DEV
103 sudo ip route add default via $DEFAULT_OAM_GATEWAY dev $OAM_DEV
105 2.3 Login the server through SSH with sysadmin
106 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
108 2.4 Prepare the localhost.yml for bootstrap
109 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
113 cat << EOF > localhost.yml
115 management_subnet: 192.168.18.0/24
116 management_start_address: 192.168.18.2
117 management_end_address: 192.168.18.50
118 management_gateway_address: 192.168.18.1
119 external_oam_subnet: 128.224.210.0/24
120 external_oam_gateway_address: 128.224.210.1
121 external_oam_floating_address: 128.224.210.110
122 external_oam_node_0_address: 128.224.210.111
123 external_oam_node_1_address: 128.224.210.112
126 2.5 Run the ansible bootstrap
127 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
131 ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
133 After the bootstrap successfully finish, it will show as following:
137 PLAY RECAP *************************************************************************************************************
138 localhost : ok=257 changed=151 unreachable=0 failed=0 skipped=214 rescued=0 ignored=0
140 2.6 Congiure controller-0
141 ~~~~~~~~~~~~~~~~~~~~~~~~~
143 Acquire admin credentials:
147 controller-0:~$ source /etc/platform/openrc
148 [sysadmin@controller-0 ~(keystone_admin)]$
150 Configure the OAM and MGMT interfaces of controller-0 and specify the
157 system host-if-modify controller-0 lo -c none
158 IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
159 for UUID in $IFNET_UUIDS; do
160 system interface-network-remove ${UUID}
163 system host-if-modify controller-0 $OAM_IF -n oam0
164 system host-if-modify controller-0 $MGMT_IF -n pxeboot0
166 system host-if-modify controller-0 oam0 -c platform
167 system interface-network-assign controller-0 oam0 oam
169 system host-if-modify controller-0 pxeboot0 -c platform
170 system interface-network-assign controller-0 pxeboot0 pxeboot
172 system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
173 system interface-network-assign controller-0 mgmt0 mgmt
175 system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
176 system interface-network-assign controller-0 cluster0 cluster-host
182 [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
183 [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
184 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 lo -c none
185 +-----------------+--------------------------------------+
187 +-----------------+--------------------------------------+
191 | imac | 00:00:00:00:00:00 |
195 | schedpolicy | None |
196 | txhashpolicy | None |
197 | uuid | 08c95952-892b-40b5-b17a-7d2ad46e725c |
198 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
202 | created_at | 2020-11-17T00:30:45.265032+00:00 |
203 | updated_at | 2020-11-17T01:03:39.031612+00:00 |
205 | sriov_vf_driver | None |
206 +-----------------+--------------------------------------+
207 [sysadmin@controller-0 ~(keystone_admin)]$ IFNET_UUIDS=$(system interface-network-list controller-0 | awk '{if ($6=="lo") print $4;}')
208 [sysadmin@controller-0 ~(keystone_admin)]$ for UUID in $IFNET_UUIDS; do
209 > system interface-network-remove ${UUID}
211 Deleted Interface Network: 0bf11f1b-4fc6-4e97-b896-3d6393a3744e
212 Deleted Interface Network: a62d95f6-ad4e-4779-bfc0-6a885067f8d8
214 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -n oam0
215 +-----------------+--------------------------------------+
217 +-----------------+--------------------------------------+
219 | iftype | ethernet |
220 | ports | [u'eno3'] |
221 | imac | 24:6e:96:5d:0c:b2 |
225 | schedpolicy | None |
226 | txhashpolicy | None |
227 | uuid | d8a048fa-67ef-43ac-8166-671be93caa30 |
228 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
232 | created_at | 2020-11-17T00:28:32.365863+00:00 |
233 | updated_at | 2020-11-17T01:03:45.090904+00:00 |
235 | sriov_vf_driver | None |
236 | accelerated | [True] |
237 +-----------------+--------------------------------------+
238 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $MGMT_IF -n pxeboot0
239 +-----------------+--------------------------------------+
241 +-----------------+--------------------------------------+
242 | ifname | pxeboot0 |
243 | iftype | ethernet |
244 | ports | [u'eno1'] |
245 | imac | 24:6e:96:5d:0c:92 |
249 | schedpolicy | None |
250 | txhashpolicy | None |
251 | uuid | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
252 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
256 | created_at | 2020-11-17T00:28:32.612230+00:00 |
257 | updated_at | 2020-11-17T01:03:47.341003+00:00 |
259 | sriov_vf_driver | None |
260 | accelerated | [True] |
261 +-----------------+--------------------------------------+
262 [sysadmin@controller-0 ~(keystone_admin)]$
263 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 oam0 -c platform
264 +-----------------+--------------------------------------+
266 +-----------------+--------------------------------------+
268 | iftype | ethernet |
269 | ports | [u'eno3'] |
270 | imac | 24:6e:96:5d:0c:b2 |
272 | ifclass | platform |
274 | schedpolicy | None |
275 | txhashpolicy | None |
276 | uuid | d8a048fa-67ef-43ac-8166-671be93caa30 |
277 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
281 | created_at | 2020-11-17T00:28:32.365863+00:00 |
282 | updated_at | 2020-11-17T01:03:49.368879+00:00 |
284 | sriov_vf_driver | None |
285 | accelerated | [True] |
286 +-----------------+--------------------------------------+
287 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 oam0 oam
288 +--------------+--------------------------------------+
290 +--------------+--------------------------------------+
291 | hostname | controller-0 |
292 | uuid | 3c8bd181-d3f3-4e14-8e89-75a3432db1e4 |
294 | network_name | oam |
295 +--------------+--------------------------------------+
296 [sysadmin@controller-0 ~(keystone_admin)]$
297 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 pxeboot0 -c platform
298 +-----------------+--------------------------------------+
300 +-----------------+--------------------------------------+
301 | ifname | pxeboot0 |
302 | iftype | ethernet |
303 | ports | [u'eno1'] |
304 | imac | 24:6e:96:5d:0c:92 |
306 | ifclass | platform |
308 | schedpolicy | None |
309 | txhashpolicy | None |
310 | uuid | 23b5e923-1e53-4e70-a975-542d8380b7f2 |
311 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
315 | created_at | 2020-11-17T00:28:32.612230+00:00 |
316 | updated_at | 2020-11-17T01:03:53.143795+00:00 |
318 | sriov_vf_driver | None |
319 | accelerated | [True] |
320 +-----------------+--------------------------------------+
321 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 pxeboot0 pxeboot
322 +--------------+--------------------------------------+
324 +--------------+--------------------------------------+
325 | hostname | controller-0 |
326 | uuid | 6c55622d-2da4-4f4e-ab5e-f8e06e03af7c |
327 | ifname | pxeboot0 |
328 | network_name | pxeboot |
329 +--------------+--------------------------------------+
330 [sysadmin@controller-0 ~(keystone_admin)]$
331 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 18 controller-0 mgmt0 vlan pxeboot0
332 +-----------------+--------------------------------------+
334 +-----------------+--------------------------------------+
338 | imac | 24:6e:96:5d:0c:92 |
342 | schedpolicy | None |
343 | txhashpolicy | None |
344 | uuid | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 |
345 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
347 | uses | [u'pxeboot0'] |
349 | created_at | 2020-11-17T01:03:57.303000+00:00 |
350 | updated_at | None |
352 | sriov_vf_driver | None |
353 | accelerated | [True] |
354 +-----------------+--------------------------------------+
355 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 mgmt0 mgmt
356 +--------------+--------------------------------------+
358 +--------------+--------------------------------------+
359 | hostname | controller-0 |
360 | uuid | 2e93ef03-e9ee-457a-8667-05b52b7109a5 |
362 | network_name | mgmt |
363 +--------------+--------------------------------------+
364 [sysadmin@controller-0 ~(keystone_admin)]$
365 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-0 cluster0 vlan pxeboot0
366 +-----------------+--------------------------------------+
368 +-----------------+--------------------------------------+
369 | ifname | cluster0 |
372 | imac | 24:6e:96:5d:0c:92 |
376 | schedpolicy | None |
377 | txhashpolicy | None |
378 | uuid | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 |
379 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
381 | uses | [u'pxeboot0'] |
383 | created_at | 2020-11-17T01:04:02.613518+00:00 |
384 | updated_at | None |
386 | sriov_vf_driver | None |
387 | accelerated | [True] |
388 +-----------------+--------------------------------------+
389 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 cluster0 cluster-host
390 +--------------+--------------------------------------+
392 +--------------+--------------------------------------+
393 | hostname | controller-0 |
394 | uuid | fb8b6be6-1618-4662-b063-b1e8d340aa48 |
395 | ifname | cluster0 |
396 | network_name | cluster-host |
397 +--------------+--------------------------------------+
398 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-0
399 +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
400 | uuid | name | class | type | vlan id | ports | uses i/f | used by i/f | attributes |
401 +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
402 | 119bdb85-1e24-44ff-b527-fe8f167b0ad3 | mgmt0 | platform | vlan | 18 | [] | [u'pxeboot0'] | [] | MTU=1500 |
403 | 23b5e923-1e53-4e70-a975-542d8380b7f2 | pxeboot0 | platform | ethernet | None | [u'eno1'] | [] | [u'mgmt0', u'cluster0'] | MTU=1500 |
404 | 6a620c8e-4f7b-4f74-a9f4-2a91d3ae9756 | cluster0 | platform | vlan | 19 | [] | [u'pxeboot0'] | [] | MTU=1500 |
405 | d8a048fa-67ef-43ac-8166-671be93caa30 | oam0 | platform | ethernet | None | [u'eno3'] | [] | [] | MTU=1500 |
406 +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
408 Configure NTP Servers for network time synchronization:
412 system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
418 [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
419 +--------------+--------------------------------------+
421 +--------------+--------------------------------------+
422 | uuid | 3206cf01-c64a-457e-ac66-b8224c9684c3 |
423 | ntpservers | 0.pool.ntp.org,1.pool.ntp.org |
424 | isystem_uuid | cc79b616-d24e-4432-a953-85c9b242cb3a |
425 | created_at | 2020-11-17T00:27:23.529571+00:00 |
426 | updated_at | None |
427 +--------------+--------------------------------------+
429 Add an OSD on controller-0 for Ceph:
433 system host-disk-list controller-0
434 system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
435 system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
436 system host-stor-list controller-0
442 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
443 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
444 | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path |
445 | | de | num | type | gib | gib | | | |
446 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
447 | 8e2a719a-fa5a-4c25-89af-70a23fb7b238 | /dev/sda | 2048 | HDD | 893. | 644.726 | Undetermined | 00c66a07604fa8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
448 | | | | | 75 | | | | |
450 | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/sdb | 2064 | HDD | 1117. | 1117.247 | Undetermined | 00c6b9139b76a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
451 | | | | | 25 | | | | |
453 | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/sdc | 2080 | HDD | 1117. | 1117.247 | Undetermined | 0053be63c794a8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
454 | | | | | 25 | | | | |
456 | 4879b381-8e9f-48f3-84e2-f9c6a94bbfe0 | /dev/sdd | 2096 | HDD | 1117. | 0.0 | Undetermined | 0065482503bca8de2500151b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
457 | | | | | 25 | | | | |
459 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
460 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
461 +------------------+-------------------------------------------------------+
463 +------------------+-------------------------------------------------------+
466 | state | configuring-on-unlock |
467 | journal_location | 0816c72f-a4f0-49ea-9a95-0f02c880717c |
468 | journal_size_gib | 1024 |
469 | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
470 | journal_node | /dev/sdb2 |
471 | uuid | 0816c72f-a4f0-49ea-9a95-0f02c880717c |
472 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
473 | idisk_uuid | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 |
474 | tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
475 | tier_name | storage |
476 | created_at | 2020-11-17T01:05:04.063823+00:00 |
477 | updated_at | None |
478 +------------------+-------------------------------------------------------+
479 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-0 {}
480 +------------------+-------------------------------------------------------+
482 +------------------+-------------------------------------------------------+
485 | state | configuring-on-unlock |
486 | journal_location | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 |
487 | journal_size_gib | 1024 |
488 | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
489 | journal_node | /dev/sdc2 |
490 | uuid | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 |
491 | ihost_uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
492 | idisk_uuid | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 |
493 | tier_uuid | 3af8c893-9dd4-40af-afc6-30bb79048448 |
494 | tier_name | storage |
495 | created_at | 2020-11-17T01:05:06.939798+00:00 |
496 | updated_at | None |
497 +------------------+-------------------------------------------------------+
498 [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
499 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
500 | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
501 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
502 | 0816c72f-a4f0-49ea-9a95-0f02c880717c | osd | 0 | configuring-on-unlock | 61b6f262-a51f-4310-aeac-373b1c1bbbc2 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2 | 1 | storage |
503 | 7a0b3727-0e3f-4582-9415-56e44bb8f1e5 | osd | 1 | configuring-on-unlock | 81a7f4f9-dd3a-49b5-80d9-e1953aa43c79 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
504 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
506 2.7 Unlock controller-0
507 ~~~~~~~~~~~~~~~~~~~~~~~
511 system host-unlock controller-0
517 [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
518 +-----------------------+-------------------------------------------------+
520 +-----------------------+-------------------------------------------------+
522 | administrative | locked |
523 | availability | online |
526 | bm_username | None |
527 | boot_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
528 | capabilities | {u'stor_function': u'monitor'} |
529 | clock_synchronization | ntp |
530 | config_applied | 6aa15fb4-8cb3-494e-b94e-95f85b560f22 |
531 | config_status | None |
532 | config_target | c6ae9b2d-a3c4-4751-a79e-5487ba81ed82 |
533 | console | ttyS0,115200 |
534 | created_at | 2020-11-17T00:28:01.983673+00:00 |
535 | hostname | controller-0 |
537 | install_output | graphical |
538 | install_state | None |
539 | install_state_info | None |
540 | inv_state | inventoried |
541 | invprovision | provisioning |
543 | mgmt_ip | 192.168.18.3 |
544 | mgmt_mac | 24:6e:96:5d:0c:92 |
545 | operational | disabled |
546 | personality | controller |
548 | rootfs_device | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
550 | software_load | 19.12 |
551 | subfunction_avail | online |
552 | subfunction_oper | disabled |
553 | subfunctions | controller,worker,lowlatency |
557 | updated_at | 2020-11-17T01:05:07.015414+00:00 |
559 | uuid | 16afe3a2-ba50-46b8-9fd7-09010059e8b9 |
560 | vim_progress_status | None |
561 +-----------------------+-------------------------------------------------+
563 Controller-0 will reboot to apply configuration changes and come into
564 service. This can take 5-10 minutes, depending on the performance of the
567 Once the controller comes back up, check the status of controller-0
571 controller-0:~$ source /etc/platform/openrc
572 [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
573 +----+--------------+-------------+----------------+-------------+--------------+
574 | id | hostname | personality | administrative | operational | availability |
575 +----+--------------+-------------+----------------+-------------+--------------+
576 | 1 | controller-0 | controller | unlocked | enabled | available |
577 +----+--------------+-------------+----------------+-------------+--------------+
579 2. Installation for the second server (controller-1)
580 ----------------------------------------------------
582 2.1 Power on the controller-1 server and force it to network boot
583 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
585 2.2 As controller-1 boots, a message appears on its console instructing you to configure the personality of the node
586 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
588 2.3 On the console of controller-0, list hosts to see newly discovered controller-1 host (hostname=None)
589 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
593 [sysadmin@controller-0 ~(keystone_admin)]$ system host-list
594 +----+--------------+-------------+----------------+-------------+--------------+
595 | id | hostname | personality | administrative | operational | availability |
596 +----+--------------+-------------+----------------+-------------+--------------+
597 | 1 | controller-0 | controller | unlocked | enabled | degraded |
598 | 2 | None | None | locked | disabled | offline |
599 +----+--------------+-------------+----------------+-------------+--------------+
601 2.4 Using the host id, set the personality of this host to 'controller’:
602 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
606 [sysadmin@controller-0 ~(keystone_admin)]$ system host-update 2 personality=controller
607 +-----------------------+--------------------------------------+
609 +-----------------------+--------------------------------------+
611 | administrative | locked |
612 | availability | offline |
615 | bm_username | None |
616 | boot_device | /dev/sda |
617 | capabilities | {} |
618 | clock_synchronization | ntp |
619 | config_applied | None |
620 | config_status | None |
621 | config_target | None |
622 | console | ttyS0,115200 |
623 | created_at | 2020-11-17T10:17:44.387813+00:00 |
624 | hostname | controller-1 |
626 | install_output | text |
627 | install_state | None |
628 | install_state_info | None |
630 | invprovision | None |
632 | mgmt_ip | 192.168.18.4 |
633 | mgmt_mac | 24:6e:96:5d:38:ee |
634 | operational | disabled |
635 | personality | controller |
637 | rootfs_device | /dev/sda |
639 | software_load | 19.12 |
640 | subfunction_avail | not-installed |
641 | subfunction_oper | disabled |
642 | subfunctions | controller,worker,lowlatency |
646 | updated_at | None |
648 | uuid | f069381d-9743-49cc-bf8b-eb4bd3972203 |
649 | vim_progress_status | None |
650 +-----------------------+--------------------------------------+
652 2.5 Wait for the software installation on controller-1 to complete, for controller-1 to reboot, and for controller-1 to show as locked/disabled/online in 'system host-list'.
653 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
655 This can take 5-10 minutes, depending on the performance of the host
660 [root@controller-0 hieradata(keystone_admin)]$ system host-list
661 +----+--------------+-------------+----------------+-------------+--------------+
662 | id | hostname | personality | administrative | operational | availability |
663 +----+--------------+-------------+----------------+-------------+--------------+
664 | 1 | controller-0 | controller | unlocked | enabled | available |
665 | 2 | controller-1 | controller | locked | disabled | online |
666 +----+--------------+-------------+----------------+-------------+--------------+
668 2.6 Configure controller-1
669 ~~~~~~~~~~~~~~~~~~~~~~~~~~
675 system host-if-modify controller-1 $OAM_IF -n oam0
676 system host-if-modify controller-1 oam0 -c platform
677 system interface-network-assign controller-1 oam0 oam
679 system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
680 system interface-network-assign controller-1 cluster0 cluster-host
682 system host-if-list controller-1
684 system host-disk-list controller-1
685 system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
686 system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
687 system host-stor-list controller-1
693 [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno3
694 [sysadmin@controller-0 ~(keystone_admin)]$ MGMT_IF=eno1
695 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 $OAM_IF -n oam0
696 +-----------------+--------------------------------------+
698 +-----------------+--------------------------------------+
700 | iftype | ethernet |
701 | ports | [u'eno3'] |
702 | imac | 24:6e:96:5d:39:0e |
706 | schedpolicy | None |
707 | txhashpolicy | None |
708 | uuid | c2473511-d0d6-445d-9739-4d43dc029de9 |
709 | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
713 | created_at | 2020-11-22T14:01:32.365863+00:00 |
714 | updated_at | 2020-11-22T15:04:45.090904+00:00 |
716 | sriov_vf_driver | None |
717 | accelerated | [True] |
718 +-----------------+--------------------------------------+
719 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-1 oam0 -c platform
720 +-----------------+--------------------------------------+
722 +-----------------+--------------------------------------+
724 | iftype | ethernet |
725 | ports | [u'eno3'] |
726 | imac | 24:6e:96:5d:39:0e |
728 | ifclass | platform |
730 | schedpolicy | None |
731 | txhashpolicy | None |
732 | uuid | c2473511-d0d6-445d-9739-4d43dc029de9 |
733 | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
737 | created_at | 2020-11-22T14:05:16.052229+00:00 |
738 | updated_at | 2020-11-22T15:08:35.324634+00:00 |
740 | sriov_vf_driver | None |
741 | accelerated | [True] |
742 +-----------------+--------------------------------------+
743 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 oam0 oam
744 +--------------+--------------------------------------+
746 +--------------+--------------------------------------+
747 | hostname | controller-1 |
748 | uuid | f2e7f088-0dd0-4adc-8348-4e3cef23bc47 |
750 | network_name | oam |
751 +--------------+--------------------------------------+
752 [sysadmin@controller-0 ~(keystone_admin)]$
754 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-add -V 19 controller-1 cluster0 vlan pxeboot0
755 +-----------------+--------------------------------------+
757 +-----------------+--------------------------------------+
758 | ifname | cluster0 |
761 | imac | 24:6e:96:5d:38:ee |
765 | schedpolicy | None |
766 | txhashpolicy | None |
767 | uuid | b6783682-b2aa-4135-90d2-676e1db41ae8 |
768 | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
770 | uses | [u'pxeboot0'] |
772 | created_at | 2020-11-22T15:08:43.932209+00:00 |
773 | updated_at | None |
775 | sriov_vf_driver | None |
776 | accelerated | [True] |
777 +-----------------+--------------------------------------+
779 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-1 cluster0 cluster-host
780 +--------------+--------------------------------------+
782 +--------------+--------------------------------------+
783 | hostname | controller-1 |
784 | uuid | 8fc64805-b54b-45a4-b88a-e13b236abfe8 |
785 | ifname | cluster0 |
786 | network_name | cluster-host |
787 +--------------+--------------------------------------+
788 [sysadmin@controller-0 ~(keystone_admin)]$
789 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-list controller-1
790 +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
791 | uuid | name | class | type | vlan id | ports | uses i/f | used by i/f | attributes |
792 +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
793 | b6783682-b2aa-4135-90d2-676e1db41ae8 | cluster0 | platform | vlan | 19 | [] | [u'pxeboot0'] | [] | MTU=1500 |
794 | b8921960-fde5-44c3-960d-2aebf42ea400 | pxeboot0 | platform | ethernet | None | [u'eno1'] | [] | [u'mgmt0', u'cluster0'] | MTU=1500 |
795 | c103275b-2b75-4568-865f-ac6be32ecb2d | mgmt0 | platform | vlan | 18 | [] | [u'pxeboot0'] | [] | MTU=1500 |
796 | c2473511-d0d6-445d-9739-4d43dc029de9 | oam0 | platform | ethernet | None | [u'eno3'] | [] | [] | MTU=1500 |
797 +--------------------------------------+----------+----------+----------+---------+-----------+---------------+-------------------------+------------+
798 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1
799 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
800 | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_id | device_path |
801 | | de | num | type | gib | gib | | | |
802 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
803 | 5b8fade4-b048-48fa-b906-9dcbdbed8e96 | /dev/sda | 2048 | HDD | 893. | 644.726 | Undetermined | 00cbd97f3e36ccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:0:0 |
804 | | | | | 75 | | | | |
806 | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/sdb | 2064 | HDD | 1117. | 1117.247 | Undetermined | 006d0e977b5fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0 |
807 | | | | | 25 | | | | |
809 | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/sdc | 2080 | HDD | 1117. | 1117.247 | Undetermined | 003a2377ac7fccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0 |
810 | | | | | 25 | | | | |
812 | 774c3cd0-1178-4145-9573-f0d6dee2ba06 | /dev/sdd | 2096 | HDD | 1117. | 1117.247 | Undetermined | 00d7093ef0adccfa2500561b14604609 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:3:0 |
813 | | | | | 25 | | | | |
815 | 00361302-8d55-4730-855c-b0098c73ab7e | /dev/sde | 2112 | SSD | 223. | 223.568 | N/A | PHDW730104QM240E | /dev/disk/by-path/pci-0000:d8:00.0-ata-1 |
816 | | | | | 57 | | | | |
818 | 7ce735e6-920f-4424-a890-a7a7f48d7632 | /dev/sdf | 2128 | SSD | 223. | 223.568 | N/A | PHDW730104LL240E | /dev/disk/by-path/pci-0000:d8:00.0-ata-2 |
819 | | | | | 57 | | | | |
821 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+----------------------------------+-------------------------------------------------+
822 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-1 {}
823 +------------------+-------------------------------------------------------+
825 +------------------+-------------------------------------------------------+
828 | state | configuring-on-unlock |
829 | journal_location | 54a218d8-0466-4366-9ef0-3ec5a952fde7 |
830 | journal_size_gib | 1024 |
831 | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 |
832 | journal_node | /dev/sdb2 |
833 | uuid | 54a218d8-0466-4366-9ef0-3ec5a952fde7 |
834 | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
835 | idisk_uuid | 1a3f0a36-5961-42e5-a271-e71db1c25d42 |
836 | tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
837 | tier_name | storage |
838 | created_at | 2020-11-22T15:11:55.641193+00:00 |
839 | updated_at | None |
840 +------------------+-------------------------------------------------------+
842 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-1 | awk '/\/dev\/sdc/{print $2}' | xargs -i system host-stor-add controller-1 {}
843 +------------------+-------------------------------------------------------+
845 +------------------+-------------------------------------------------------+
848 | state | configuring-on-unlock |
849 | journal_location | 5be88c7a-3a94-4b97-9da5-b247bb89406c |
850 | journal_size_gib | 1024 |
851 | journal_path | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 |
852 | journal_node | /dev/sdc2 |
853 | uuid | 5be88c7a-3a94-4b97-9da5-b247bb89406c |
854 | ihost_uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
855 | idisk_uuid | eddd732f-2cea-49b3-86db-b722c0b1a1ae |
856 | tier_uuid | 06b4740e-29db-4896-9600-03ee40fe0d6c |
857 | tier_name | storage |
858 | created_at | 2020-11-22T15:12:04.274839+00:00 |
859 | updated_at | None |
860 +------------------+-------------------------------------------------------+
862 [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-1
863 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
864 | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
865 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
866 | 54a218d8-0466-4366-9ef0-3ec5a952fde7 | osd | 2 | configuring-on-unlock | 1a3f0a36-5961-42e5-a271-e71db1c25d42 | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:1:0-part2 | /dev/sdb2 | 1 | storage |
867 | 5be88c7a-3a94-4b97-9da5-b247bb89406c | osd | 3 | configuring-on-unlock | eddd732f-2cea-49b3-86db-b722c0b1a1ae | /dev/disk/by-path/pci-0000:86:00.0-scsi-0:2:2:0-part2 | /dev/sdc2 | 1 | storage |
868 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
870 2.7 Unlock controller-1
871 ~~~~~~~~~~~~~~~~~~~~~~~
873 Unlock controller-1 in order to bring it into service:
877 [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-1
878 +-----------------------+--------------------------------------+
880 +-----------------------+--------------------------------------+
882 | administrative | locked |
883 | availability | online |
886 | bm_username | None |
887 | boot_device | /dev/sda |
888 | capabilities | {u'stor_function': u'monitor'} |
889 | clock_synchronization | ntp |
890 | config_applied | None |
891 | config_status | Config out-of-date |
892 | config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
893 | console | ttyS0,115200 |
894 | created_at | 2020-11-22T12:58:11.630526+00:00 |
895 | hostname | controller-1 |
897 | install_output | text |
898 | install_state | None |
899 | install_state_info | None |
900 | inv_state | inventoried |
901 | invprovision | unprovisioned |
903 | mgmt_ip | 192.168.18.4 |
904 | mgmt_mac | 24:6e:96:5d:38:ee |
905 | operational | disabled |
906 | personality | controller |
908 | rootfs_device | /dev/sda |
910 | software_load | 19.12 |
911 | subfunction_avail | online |
912 | subfunction_oper | disabled |
913 | subfunctions | controller,worker,lowlatency |
917 | updated_at | 2020-11-22T15:13:09.716324+00:00 |
919 | uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
920 | vim_progress_status | None |
921 +-----------------------+--------------------------------------+
923 Controller-1 will reboot in order to apply configuration changes and
924 come into service. This can take 5-10 minutes, depending on the
925 performance of the host machine.
929 [root@controller-0 hieradata(keystone_admin)]$ system host-list
930 +----+--------------+-------------+----------------+-------------+--------------+
931 | id | hostname | personality | administrative | operational | availability |
932 +----+--------------+-------------+----------------+-------------+--------------+
933 | 1 | controller-0 | controller | unlocked | enabled | available |
934 | 2 | controller-1 | controller | unlocked | enabled | available |
935 +----+--------------+-------------+----------------+-------------+--------------+
937 [sysadmin@controller-0 ~(keystone_admin)]$ system host-show controller-1
938 +-----------------------+-----------------------------------------------------------------------+
940 +-----------------------+-----------------------------------------------------------------------+
942 | administrative | unlocked |
943 | availability | available |
946 | bm_username | None |
947 | boot_device | /dev/sda |
948 | capabilities | {u'stor_function': u'monitor', u'Personality': u'Controller-Standby'} |
949 | clock_synchronization | ntp |
950 | config_applied | 9747e0ce-2319-409d-b75c-2475bc5065ac |
951 | config_status | None |
952 | config_target | 9747e0ce-2319-409d-b75c-2475bc5065ac |
953 | console | ttyS0,115200 |
954 | created_at | 2020-11-22T12:58:11.630526+00:00 |
955 | hostname | controller-1 |
957 | install_output | text |
958 | install_state | None |
959 | install_state_info | None |
960 | inv_state | inventoried |
961 | invprovision | provisioned |
963 | mgmt_ip | 192.168.18.4 |
964 | mgmt_mac | 24:6e:96:5d:38:ee |
965 | operational | enabled |
966 | personality | controller |
968 | rootfs_device | /dev/sda |
970 | software_load | 19.12 |
971 | subfunction_avail | available |
972 | subfunction_oper | enabled |
973 | subfunctions | controller,worker,lowlatency |
977 | updated_at | 2020-11-22T23:59:07.787759+00:00 |
979 | uuid | 63c930c7-2195-4d5a-870c-be610fd6b4fc |
980 | vim_progress_status | services-enabled |
981 +-----------------------+-----------------------------------------------------------------------+
984 3. Simple use case for sriov
985 ````````````````````````````
987 3.1 After controller-0 is rebooted and up running, download the DPDK
988 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
992 [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
993 [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
995 --2020-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
996 Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
997 Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
999 HTTP request sent, awaiting response... 200 OK
1000 Length: 10251680 (9.8M) [application/octet-stream]
1001 Saving to: ‘dpdk-17.11.10.tar.xz’
1003 dpdk-17.11.10.tar.xz 100%
1004 [========================================================================================>] 9.78M
1007 2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
1009 sudo tar xvf dpdk-17.11.10.tar.xz
1011 sudo ln -s dpdk-stable-17.11.10 dpdk-stable
1013 3.2 Prepare the yaml file for the network assignment container
1014 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
1016 The following the exmaple of the yaml file:
1020 [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
1021 > apiVersion: "k8s.cni.cncf.io/v1"
1022 > kind: NetworkAttachmentDefinition
1024 > name: sriov-data-dpdk-0
1026 > k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
1030 > "name": "sriov-data-dpdk-0"
1034 > apiVersion: "k8s.cni.cncf.io/v1"
1035 > kind: NetworkAttachmentDefinition
1037 > name: sriov-data-dpdk-1
1039 > k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
1043 > "name": "sriov-data-dpdk-1"
1047 3.3 Run the network assignent container for the 2 VFs
1048 '''''''''''''''''''''''''''''''''''''''''''''''''''''
1052 [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
1053 networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
1054 networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
1056 3.4 Prepare the VF container yaml file
1057 ''''''''''''''''''''''''''''''''''''''
1061 [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
1066 > name: pod-with-dpdk-vfs-0
1068 > k8s.v1.cni.cncf.io/networks: '[
1069 > { "name": "sriov-data-dpdk-0" },
1070 { "name": "sriov-data-dpdk-1" }
1071 > { "name": "sriov-data-dpdk-1" }
1074 > restartPolicy: Never
1076 > - name: pod-with-dpdk-vfs-0
1077 > image: wrsnfv/ubuntu-dpdk-build:v0.3
1080 > value: "/usr/src/dpdk"
1095 > intel.com/pci_sriov_net_physnet0: '1'
1096 > intel.com/pci_sriov_net_physnet1: '1'
1099 > hugepages-1Gi: 2Gi
1101 > intel.com/pci_sriov_net_physnet0: '1'
1102 > intel.com/pci_sriov_net_physnet1: '1'
1104 > - mountPath: /mnt/huge-1048576kB
1106 > - name: dpdk-volume
1107 > mountPath: /usr/src/dpdk
1108 > - name: lib-volume
1109 > mountPath: /lib/modules
1110 > - name: src-volume
1111 > mountPath: /usr/src/
1116 > - name: dpdk-volume
1118 > path: /opt/dpdk-stable/
1119 > - name: lib-volume
1121 > path: /lib/modules
1122 > - name: src-volume
1127 3.5 Run the VF container
1128 ''''''''''''''''''''''''
1130 Start the VF container.
1134 [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
1135 pod/pod-with-dpdk-vfs-0 created
1137 [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
1138 NAME READY STATUS RESTARTS AGE
1139 pod-with-dpdk-vfs-0 1/1 Running 0 6m40s
1141 Login the VF container
1145 kubectl exec -it pod-with-dpdk-vfs-0 -- bash
1151 cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
1153 root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
1154 HOSTCC scripts/basic/fixdep
1155 HOSTCC scripts/kconfig/conf.o
1156 HOSTCC scripts/kconfig/confdata.o
1157 HOSTCC scripts/kconfig/expr.o
1158 HOSTCC scripts/kconfig/symbol.o
1159 HOSTCC scripts/kconfig/preprocess.o
1160 HOSTCC scripts/kconfig/zconf.lex.o
1161 HOSTCC scripts/kconfig/zconf.tab.o
1162 HOSTLD scripts/kconfig/conf
1163 scripts/kconfig/conf --syncconfig Kconfig
1164 HOSTCC arch/x86/tools/relocs_32.o
1165 HOSTCC arch/x86/tools/relocs_64.o
1166 HOSTCC arch/x86/tools/relocs_common.o
1167 HOSTLD arch/x86/tools/relocs
1168 HOSTCC scripts/genksyms/genksyms.o
1169 YACC scripts/genksyms/parse.tab.c
1170 HOSTCC scripts/genksyms/parse.tab.o
1171 LEX scripts/genksyms/lex.lex.c
1172 YACC scripts/genksyms/parse.tab.h
1173 HOSTCC scripts/genksyms/lex.lex.o
1174 HOSTLD scripts/genksyms/genksyms
1175 HOSTCC scripts/bin2c
1176 HOSTCC scripts/kallsyms
1177 HOSTCC scripts/conmakehash
1178 HOSTCC scripts/recordmcount
1179 HOSTCC scripts/sortextable
1180 HOSTCC scripts/asn1_compiler
1181 HOSTCC scripts/sign-file
1182 HOSTCC scripts/extract-cert
1183 CC scripts/mod/empty.o
1184 HOSTCC scripts/mod/mk_elfconfig
1185 MKELF scripts/mod/elfconfig.h
1186 HOSTCC scripts/mod/modpost.o
1187 CC scripts/mod/devicetable-offsets.s
1188 UPD scripts/mod/devicetable-offsets.h
1189 HOSTCC scripts/mod/file2alias.o
1190 HOSTCC scripts/mod/sumversion.o
1191 HOSTLD scripts/mod/modpost
1193 CC arch/x86/kernel/asm-offsets.s
1194 CALL scripts/checksyscalls.sh
1196 Build the test_pmd application
1201 ./usertools/dpdk-setup.sh
1215 INSTALL-MAP testpmd.map
1216 == Build app/proc_info
1219 INSTALL-APP dpdk-procinfo
1220 INSTALL-MAP dpdk-procinfo.map
1224 INSTALL-APP dpdk-pdump
1225 INSTALL-MAP dpdk-pdump.map
1226 == Build app/test-crypto-perf
1229 CC cperf_options_parsing.o
1230 CC cperf_test_vectors.o
1231 CC cperf_test_throughput.o
1232 CC cperf_test_latency.o
1233 CC cperf_test_pmd_cyclecount.o
1234 CC cperf_test_verify.o
1235 CC cperf_test_vector_parsing.o
1236 CC cperf_test_common.o
1237 LD dpdk-test-crypto-perf
1238 INSTALL-APP dpdk-test-crypto-perf
1239 INSTALL-MAP dpdk-test-crypto-perf.map
1240 == Build app/test-eventdev
1245 CC test_order_common.o
1246 CC test_order_queue.o
1248 CC test_perf_common.o
1249 CC test_perf_queue.o
1251 LD dpdk-test-eventdev
1252 INSTALL-APP dpdk-test-eventdev
1253 INSTALL-MAP dpdk-test-eventdev.map
1254 Build complete [x86_64-native-linuxapp-gcc]
1255 Installation cannot run with T defined and DESTDIR undefined
1256 ------------------------------------------------------------------------------
1257 RTE_TARGET exported as x86_64-native-linuxapp-gcc
1258 ------------------------------------------------------------------------------
1260 Press enter to continue ...
1262 Check the VF PCI information:
1266 root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
1267 PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
1268 PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
1270 Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
1274 [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
1275 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
1276 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
1277 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
1278 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
1279 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
1280 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
1282 [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
1283 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
1284 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
1285 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
1286 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
1287 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
1288 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
1290 [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
1291 [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
1294 Get back to pod and run the test_pmd
1298 cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
1300 taskset -p --cpu-list 1
1301 pid 1's current affinity list: 2,3,18,19
1303 ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
1304 start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
1305 peer=1,"9e:fd:e6:dd:c1:01"
1306 EAL: Detected 32 lcore(s)
1307 EAL: No free hugepages reported in hugepages-2048kB
1308 EAL: Probing VFIO support...
1309 EAL: VFIO support initialized
1310 EAL: PCI device 0000:05:11.0 on NUMA socket 0
1311 EAL: probe driver: 8086:10ed net_ixgbe_vf
1312 EAL: using IOMMU type 1 (Type 1)
1313 EAL: PCI device 0000:05:11.1 on NUMA socket 0
1314 EAL: probe driver: 8086:10ed net_ixgbe_vf
1316 Ports to start sending a burst of packets first
1317 Warning: lsc_interrupt needs to be off when using tx_first. Disabling.
1318 USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
1319 Configuring Port 0 (socket 0)
1320 Port 0: 9E:FD:E6:DD:C1:01
1321 Configuring Port 1 (socket 0)
1322 Port 1: 9E:FD:E6:DD:C1:02
1323 Checking link statuses...
1324 Port0 Link Up. speed 10000 Mbps- full-duplex
1325 Port1 Link Up. speed 10000 Mbps- full-duplex
1327 No commandline core given, start packet forwarding
1328 io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
1330 Logical Core 3 (socket 0) forwards packets on 2 streams:
1331 RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
1332 RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
1334 io packet forwarding packets/burst=32
1335 nb forwarding cores=1 - nb forwarding ports=2
1337 CRC stripping enabled
1338 RX queues=1 - RX desc=128 - RX free threshold=32
1339 RX threshold registers: pthresh=8 hthresh=8 wthresh=0
1340 TX queues=1 - TX desc=512 - TX free threshold=32
1341 TX threshold registers: pthresh=32 hthresh=0 wthresh=0
1342 TX RS bit threshold=32 - TXQ flags=0xf01
1344 CRC stripping enabled
1345 RX queues=1 - RX desc=128 - RX free threshold=32
1346 RX threshold registers: pthresh=8 hthresh=8 wthresh=0
1347 TX queues=1 - TX desc=512 - TX free threshold=32
1348 TX threshold registers: pthresh=32 hthresh=0 wthresh=0
1349 TX RS bit threshold=32 - TXQ flags=0xf01
1351 Port statistics ====================================
1352 ######################## NIC statistics for port 0 ########################
1353 RX-packets: 56 RX-missed: 0 RX-bytes: 4096
1356 TX-packets: 64 TX-errors: 0 TX-bytes: 4096
1358 Throughput (since last show)
1361 ############################################################################
1363 ######################## NIC statistics for port 1 ########################
1364 RX-packets: 432 RX-missed: 0 RX-bytes: 27712
1367 TX-packets: 461 TX-errors: 0 TX-bytes: 30080
1369 Throughput (since last show)
1372 ############################################################################
1374 Port statistics ====================================
1375 ######################## NIC statistics for port 0 ########################
1376 RX-packets: 14124641 RX-missed: 0 RX-bytes: 903977344
1379 TX-packets: 14170205 TX-errors: 0 TX-bytes: 906893376
1381 Throughput (since last show)
1384 ############################################################################
1396 .. _`StarlingX`: https://docs.starlingx.io/