OpenStack Epoxy : Add Compute Nodes (GPU) |
Add GPU attached Compute Nodes to use GPU on virtual machine instances. On this example, add [node02.srv.world] like follows. ------------+--------------------------+--------------------------+------------
| | |
eth0|10.0.0.30 eth0|10.0.0.50 eth0|10.0.0.51
+-----------+-----------+ +-----------+-----------+ +-----------+-----------+
| [ dlp.srv.world ] | | [ network.srv.world ] | | [ node01.srv.world ] |
| (Control Node) | | (Network Node) | | (Compute Node) |
| | | | | |
| MariaDB RabbitMQ | | Open vSwitch | | Libvirt |
| Memcached Nginx | | Neutron Server | | Nova Compute |
| Keystone httpd | | OVN-Northd | | Open vSwitch |
| Glance Nova API | | Nginx | | OVN Metadata Agent |
| | | | | OVN-Controller |
+-----------------------+ +-----------------------+ +-----------------------+
------------+------------
|
eth0|10.0.0.52
+-----------+-----------+
| [ node02.srv.world ] |
| (Compute Node (GPU)) |
| |
| Libvirt |
| Nova Compute |
| Open vSwitch |
| OVN Metadata Agent |
| OVN-Controller |
+-----------------------+
|
| [1] | On the new Node with GPU, Configure GPU Passthrough setting, refer to here for [1]. |
| [2] | On the new Node with GPU, add it to Openstack Cluster as a Compute Node, refer to here. |
| [3] | On the new Node with GPU, Configure additional settings for Nova-Compute. |
| root@node02:~# lspci -nn | grep -i nvidia 81:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA104 [GeForce RTX 3060] [10de:2487] (rev a1) 81:00.1 Audio device [0403]: NVIDIA Corporation GA104 High Definition Audio Controller [10de:228b] (rev a1) root@node02:~# vi /etc/nova/nova.conf # add to last line
# add [vendor_id], [product_id] of passthrough device
[pci]
passthrough_whitelist = { "vendor_id": "10de", "product_id": "2487" }
systemctl restart nova-compute |
| [4] | On Control Node, Change Nova settings. |
root@dlp ~(keystone)# vi /etc/nova/nova.conf # add to last line
# add [vendor_id], [product_id] that are the same one on compute node
# for the [name], set any name you like
[pci]
alias: { "vendor_id":"10de", "product_id":"2487", "device_type":"type-PCI", "name":"RTX-3060" }
[filter_scheduler]
enabled_filters = PciPassthroughFilter
root@dlp ~(keystone)# systemctl restart nova-api nova-scheduler root@dlp ~(keystone)# su -s /bin/bash nova -c "nova-manage cell_v2 discover_hosts" # create [flavor] for GPU instance root@dlp ~(keystone)# openstack flavor create --id 6 --vcpus 4 --ram 8192 --disk 20 --property "pci_passthrough:alias"="RTX-3060:1" gpu1.small +----------------------------+------------------------------------+ | Field | Value | +----------------------------+------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | description | None | | disk | 20 | | id | 6 | | name | gpu1.small | | os-flavor-access:is_public | True | | properties | pci_passthrough:alias='RTX-3060:1' | | ram | 8192 | | rxtx_factor | 1.0 | | swap | 0 | | vcpus | 4 | +----------------------------+------------------------------------+root@dlp ~(keystone)# openstack flavor list +----+------------+-------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +----+------------+-------+------+-----------+-------+-----------+ | 1 | m1.tiny | 2048 | 10 | 0 | 1 | True | | 2 | m1.small | 4096 | 10 | 0 | 2 | True | | 3 | m1.medium | 8192 | 10 | 0 | 4 | True | | 4 | m1.large | 16384 | 10 | 0 | 8 | True | | 5 | m2.medium | 8192 | 10 | 10 | 4 | True | | 6 | gpu1.small | 8192 | 20 | 0 | 4 | True | +----+------------+-------+------+-----------+-------+-----------+ |
| [5] | Verify settings to create a GPU instance with any Openstack user. |
| debian@dlp ~(keystone)$ openstack image list +--------------------------------------+----------+--------+ | ID | Name | Status | +--------------------------------------+----------+--------+ | 33def798-3361-483b-9de1-8c2c6e1c840c | Debian13 | active | +--------------------------------------+----------+--------+debian@dlp ~(keystone)$ openstack network list +--------------------------------------+---------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+---------+--------------------------------------+ | 7de3878f-814f-4909-b4e6-d4dc1c740577 | public | 319013fd-5412-4cce-bb87-49f5a0c91b0e | | d442015a-b6f6-4349-890b-c08eb5366a4d | private | bb5efd0a-ea4d-42ee-99ca-97cee2f56ca2 | +--------------------------------------+---------+--------------------------------------+debian@dlp ~(keystone)$ netID=$(openstack network list | grep private | awk '{ print $2 }') debian@dlp ~(keystone)$ debian@dlp ~(keystone)$ openstack server create --flavor gpu1.small --image Debian13 --security-group secgroup01 --nic net-id=$netID --key-name mykey Debian-13GPU openstack server list +--------------------------------------+--------------+---------+------------------------------------+----------+------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+--------------+---------+------------------------------------+----------+------------+ | 8b270701-c516-4f7e-979b-53d7186a6e4d | Debian-13GPU | ACTIVE | private=192.168.100.97 | Debian13 | gpu1.small | | 2deabceb-e220-4e5c-b8dd-320683d0bf0b | Debian-13 | SHUTOFF | private=10.0.0.249, 192.168.100.39 | Debian13 | m1.small | +--------------------------------------+--------------+---------+------------------------------------+----------+------------+debian@dlp ~(keystone)$ openstack floating ip create public +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | created_at | 2025-08-29T05:06:04Z | | description | | | dns_domain | | | dns_name | | | fixed_ip_address | None | | floating_ip_address | 10.0.0.251 | | floating_network_id | 7de3878f-814f-4909-b4e6-d4dc1c740577 | | id | ec9e221a-68a6-4ce7-a149-47cf4f62eaef | | name | 10.0.0.251 | | port_details | None | | port_id | None | | project_id | ecfa98ba82de421e8f16c3d862b5ab04 | | qos_policy_id | None | | revision_number | 0 | | router_id | None | | status | DOWN | | subnet_id | None | | tags | [] | | updated_at | 2025-08-29T05:06:04Z | +---------------------+--------------------------------------+debian@dlp ~(keystone)$ openstack server add floating ip Debian-13GPU 10.0.0.251 debian@dlp ~(keystone)$ ssh debian@10.0.0.251 The authenticity of host '10.0.0.251 (10.0.0.251)' can't be established.
ED25519 key fingerprint is SHA256:Vl3vZ+KUqZ0fiuiDUR7gsOQ1fnyCFAsQ833tz59TVQY.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.0.0.251' (ED25519) to the list of known hosts.
Linux debian-13gpu 6.12.41+deb13-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.41-1 (2025-08-12) x86_64
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
debian@debian-13gpu:~$debian@debian-13gpu:~$ lspci | grep -i nvidia 00:05.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3060] (rev a1) |
Matched Content
No comments:
Post a Comment