VFIO GPU with pass thru with two nvidia cards

Good morning, I had to reinstall due to my own error and I am now in the process of setting VFIO GPUPassthrough backup. I am missing a step and not sure what. As I have a nvidia 1050 and 2080 in the system. I want to pass thru the 1050 only. I have been following the guides I see posted in the forums, and recall I had this issue before and adding one line someplace fixed it.

The current issue is the 1050 is not loading the VFIO drivers. I have created a vfio file in /etc and added the following to it:

/etc/modprobe.d/vfio.conf:

options vfio-pci.ids=10de:1c82,10de:0fb9

grub boot line:

rd.driver.pre=vfio-pci intel_iommu=on rd.udev.log_priority=3 vt.global_cursor_default=0 loglevel=3 vfio_pci.ids=10de:1c82,10de:0fb9 pcie_acs_override=downstream,multifunction video=efifb:off iommu=pt

/etc/dracut.conf.d/10-vfio.conf
force_drivers+=" vfio_pci vfio vfio_iommu_type1 "

Still reading items in the forum and some guides. I think I have most of what applies in the arch wiki for VFIO that pairs with Garuda. I just recall someone said. Yep add this to this and reboot, but for the life me I can see what I am missing. Never talk to your kids and wife while expanding your root partition ;-).

garuda-inxi:
System:
Kernel: 6.4.12-zen1-1-zen arch: x86_64 bits: 64 compiler: gcc v: 13.2.1
clocksource: tsc available: hpet,acpi_pm
parameters: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen
root=UUID=85d10ba7-1da5-4b61-a934-a0dd37c75c5e rw rootflags=subvol=@
rd.driver.pre=vfio-pci intel_iommu=on rd.udev.log_priority=3
vt.global_cursor_default=0 loglevel=3 vfio_pci.ids=10de:1c82,10de:0fb9
pcie_acs_override=downstream,multifunction video=efifb:off iommu=pt
ibt=off
Desktop: KDE Plasma v: 5.27.7 tk: Qt v: 5.15.10 wm: kwin_x11 dm: SDDM
Distro: Garuda Linux base: Arch Linux
Machine:
Type: Desktop System: ASUS product: N/A v: N/A serial: N/A
Mobo: ASUSTeK model: TUF GAMING Z590-PLUS WIFI v: Rev 1.xx
serial: <filter> UEFI: American Megatrends v: 1801 date: 12/26/2022
Battery:
Device-1: hid-0003:256F:C62E.0002-battery model: 3Dconnexion SpaceMouse
Wireless serial: N/A charge: N/A status: discharging
CPU:
Info: model: 11th Gen Intel Core i9-11900KF socket: LGA1200 bits: 64
type: MT MCP arch: Rocket Lake gen: core 11 level: v4 note: check
built: 2021+ process: Intel 14nm family: 6 model-id: 0xA7 (167)
stepping: 1 microcode: 0x59
Topology: cpus: 1x cores: 8 tpc: 2 threads: 16 smt: enabled cache:
L1: 640 KiB desc: d-8x48 KiB; i-8x32 KiB L2: 4 MiB desc: 8x512 KiB
L3: 16 MiB desc: 1x16 MiB
Speed (MHz): avg: 4278 high: 5091 min/max: 800/5100:5300
base/boost: 3465/8500 scaling: driver: intel_pstate governor: powersave
volts: 0.9 V ext-clock: 100 MHz cores: 1: 3742 2: 4919 3: 3500 4: 3500
5: 5035 6: 4104 7: 3608 8: 3500 9: 4888 10: 4934 11: 4583 12: 3500
13: 3500 14: 5091 15: 5038 16: 5010 bogomips: 112128
Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Vulnerabilities: <filter>
Graphics:
Device-1: NVIDIA TU104 [GeForce RTX 2080 Rev. A] vendor: ZOTAC
driver: nvidia v: 535.104.05 alternate: nouveau,nvidia_drm non-free: 535.xx+
status: current (as of 2023-08) arch: Turing code: TUxxx
process: TSMC 12nm FF built: 2018-22 pcie: gen: 3 speed: 8 GT/s lanes: 16
bus-ID: 01:00.0 chip-ID: 10de:1e87 class-ID: 0300
Device-2: NVIDIA GP107 [GeForce GTX 1050 Ti] vendor: Micro-Star MSI
driver: nvidia v: 535.104.05 alternate: nouveau,nvidia_drm non-free: 535.xx+
status: current (as of 2023-08) arch: Pascal code: GP10x
process: TSMC 16nm built: 2016-21 pcie: gen: 1 speed: 2.5 GT/s lanes: 4
link-max: gen: 3 speed: 8 GT/s lanes: 16 bus-ID: 04:00.0
chip-ID: 10de:1c82 class-ID: 0300
Device-3: Adomax Nuroum C40
driver: hid-generic,snd-usb-audio,usbhid,uvcvideo type: USB rev: 2.0
speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 1-1.1:4 chip-ID: 0627:a6bf
class-ID: 0102 serial: <filter>
Device-4: Logitech HD Pro Webcam C920 driver: snd-usb-audio,uvcvideo
type: USB rev: 2.0 speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 5-3.4.2:5
chip-ID: 046d:082d class-ID: 0102 serial: <filter>
Display: server: X.Org v: 21.1.8 with: Xwayland v: 23.2.0
compositor: kwin_x11 driver: X: loaded: modesetting,nvidia
alternate: fbdev,nouveau,nv,vesa gpu: nvidia,nvidia display-ID: :0
screens: 1
Screen-1: 0 s-res: 8320x1440 s-dpi: 92 s-size: 2297x393mm (90.43x15.47")
s-diag: 2330mm (91.75")
Monitor-1: DP-1 pos: 2-4 res: 1920x1080 hz: 60 dpi: 82
size: 598x336mm (23.54x13.23") diag: 686mm (27.01") modes: N/A
Monitor-2: DP-3 pos: 2-2 res: 1920x1080 hz: 60 dpi: 82
size: 598x336mm (23.54x13.23") diag: 686mm (27.01") modes: N/A
Monitor-3: DP-4 pos: primary,1-3 res: 2560x1440 hz: 60 dpi: 93
size: 698x392mm (27.48x15.43") diag: 801mm (31.52") modes: N/A
Monitor-4: HDMI-1-0 size-res: N/A modes: N/A
Monitor-5: None-2-1 size-res: N/A modes: N/A
Monitor-6: USB-C-0 pos: 2-1 res: 1920x1080 dpi: 163
size: 300x260mm (11.81x10.24") diag: 397mm (15.63") modes: N/A
API: OpenGL v: 4.6.0 NVIDIA 535.104.05 renderer: NVIDIA GeForce RTX
2080/PCIe/SSE2 direct-render: Yes
Audio:
Device-1: Intel Tiger Lake-H HD Audio vendor: ASUSTeK driver: snd_hda_intel
v: kernel alternate: snd_sof_pci_intel_tgl bus-ID: 00:1f.3
chip-ID: 8086:43c8 class-ID: 0403
Device-2: NVIDIA TU104 HD Audio vendor: ZOTAC driver: snd_hda_intel
v: kernel pcie: gen: 3 speed: 8 GT/s lanes: 16 bus-ID: 01:00.1
chip-ID: 10de:10f8 class-ID: 0403
Device-3: NVIDIA GP107GL High Definition Audio vendor: Micro-Star MSI
driver: snd_hda_intel v: kernel pcie: gen: 1 speed: 2.5 GT/s lanes: 4
link-max: gen: 3 speed: 8 GT/s lanes: 16 bus-ID: 04:00.1
chip-ID: 10de:0fb9 class-ID: 0403
Device-4: Adomax Nuroum C40
driver: hid-generic,snd-usb-audio,usbhid,uvcvideo type: USB rev: 2.0
speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 1-1.1:4 chip-ID: 0627:a6bf
class-ID: 0102 serial: <filter>
Device-5: Giga-Byte ESS SABRE HiFi USB Audio Device
driver: hid-generic,snd-usb-audio,usbhid type: USB rev: 2.0 speed: 480 Mb/s
lanes: 1 mode: 2.0 bus-ID: 1-12.2:13 chip-ID: 0414:a00c class-ID: 0300
Device-6: Giga-Byte USB Audio driver: snd-usb-audio type: USB rev: 2.0
speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 1-12.3:15 chip-ID: 0414:a008
class-ID: 0102 serial: <filter>
Device-7: Logitech HD Pro Webcam C920 driver: snd-usb-audio,uvcvideo
type: USB rev: 2.0 speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 5-3.4.2:5
chip-ID: 046d:082d class-ID: 0102 serial: <filter>
Device-8: GN Netcom Jabra Engage 75 driver: jabra,snd-usb-audio,usbhid
type: USB rev: 2.0 speed: 12 Mb/s lanes: 1 mode: 1.1 bus-ID: 5-3.4.3:6
chip-ID: 0b0e:1112 class-ID: 0300 serial: <filter>
API: ALSA v: k6.4.12-zen1-1-zen status: kernel-api tools: N/A
Server-1: PipeWire v: 0.3.78 status: n/a (root, process) with:
1: pipewire-pulse status: active 2: wireplumber status: active
3: pipewire-alsa type: plugin 4: pw-jack type: plugin
tools: pactl,pw-cat,pw-cli,wpctl
Network:
Device-1: Intel Tiger Lake PCH CNVi WiFi driver: iwlwifi v: kernel
bus-ID: 00:14.3 chip-ID: 8086:43f0 class-ID: 0280
IF: wlo1 state: down mac: <filter>
Device-2: Intel Ethernet I225-V vendor: ASUSTeK driver: igc v: kernel
pcie: gen: 2 speed: 5 GT/s lanes: 1 port: N/A bus-ID: 07:00.0
chip-ID: 8086:15f3 class-ID: 0200
IF: enp7s0 state: up speed: 1000 Mbps duplex: full mac: <filter>
Bluetooth:
Device-1: Intel AX201 Bluetooth driver: btusb v: 0.8 type: USB rev: 2.0
speed: 12 Mb/s lanes: 1 mode: 1.1 bus-ID: 1-14:14 chip-ID: 8087:0026
class-ID: e001
Report: btmgmt ID: hci0 rfk-id: 0 state: up address: <filter> bt-v: 5.2
lmp-v: 11 status: discoverable: no pairing: no class-ID: 7c0104
Drives:
Local Storage: total: 15.94 TiB used: 17.78 GiB (0.1%)
ID-1: /dev/nvme0n1 maj-min: 259:3 vendor: Samsung model: SSD 970 EVO 1TB
size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
lanes: 4 tech: SSD serial: <filter> fw-rev: 2B2QEXE7 temp: 37.9 C
scheme: GPT
SMART: yes health: PASSED on: 1y 146d 16h cycles: 1,369
read-units: 82,581,186 [42.2 TB] written-units: 55,208,180 [28.2 TB]
ID-2: /dev/nvme1n1 maj-min: 259:0 vendor: Western Digital
model: WDS100T2B0C-00PXH0 size: 931.51 GiB block-size: physical: 512 B
logical: 512 B speed: 31.6 Gb/s lanes: 4 tech: SSD serial: <filter>
fw-rev: 233010WD temp: 29.9 C scheme: MBR
SMART: yes health: PASSED on: 1y 199d 19h cycles: 424
read-units: 8,773,797 [4.49 TB] written-units: 11,099,011 [5.68 TB]
ID-3: /dev/sda maj-min: 8:0 vendor: SanDisk model: SSD PLUS 1000GB
family: Marvell based SSDs size: 931.52 GiB block-size: physical: 512 B
logical: 512 B sata: 3.2 speed: 6.0 Gb/s tech: SSD serial: <filter>
fw-rev: 00RL temp: 38 C scheme: MBR
SMART: yes state: enabled health: PASSED on: 71d 22h cycles: 353
ID-4: /dev/sdb maj-min: 8:16 vendor: SanDisk model: SSD PLUS 1000GB
family: Marvell based SSDs size: 931.52 GiB block-size: physical: 512 B
logical: 512 B sata: 3.2 speed: 6.0 Gb/s tech: SSD serial: <filter>
fw-rev: 00RL temp: 34 C scheme: GPT
SMART: yes state: enabled health: PASSED on: 261d 19h cycles: 580
ID-5: /dev/sdc maj-min: 8:32 vendor: SanDisk model: SDSSDH3 512G
size: 476.94 GiB block-size: physical: 512 B logical: 512 B sata: 3.3
speed: 6.0 Gb/s tech: SSD serial: <filter> fw-rev: 1000 temp: 33 C
scheme: MBR
SMART: yes state: enabled health: PASSED on: 3y 179d 14h cycles: 599
read: 1.4 MiB written: 1.6 MiB
ID-6: /dev/sdd maj-min: 8:48 vendor: Silicon Power model: SPCC M.2 SSD
size: 119.24 GiB block-size: physical: 512 B logical: 512 B sata: 3.2
speed: 6.0 Gb/s tech: SSD serial: <filter> fw-rev: 1B0 temp: 40 C
scheme: GPT
SMART: yes state: enabled health: PASSED on: 1y 161d 6h cycles: 678
read: 10.5 MiB written: 11.9 MiB
ID-7: /dev/sde maj-min: 8:64 vendor: ASMedia model: T ASM236X NVME
size: 119.24 GiB block-size: physical: 512 B logical: 512 B type: USB
rev: 2.1 spd: 480 Mb/s lanes: 1 mode: 2.0 tech: SSD serial: <filter>
drive serial: <filter> fw-rev: E8FK11.C temp: 69 Celsius C scheme: GPT
SMART: yes health: PASSED on: 110d 14h cycles: 542
read-units: 2,505,306 [1.28 TB] written-units: 2,673,141 [1.36 TB]
ID-8: /dev/sdf maj-min: 8:80 vendor: Seagate model: Expansion SW
size: 7.28 TiB block-size: physical: 4096 B logical: 512 B type: USB
rev: 3.2 spd: 5 Gb/s lanes: 1 mode: 3.2 gen-1x1 tech: N/A serial: <filter>
fw-rev: 1801 scheme: GPT
SMART Message: A mandatory SMART command failed. Various possible causes.
ID-9: /dev/sdg maj-min: 8:96 vendor: Seagate model: Backup+ Hub BK
size: 3.64 TiB block-size: physical: 4096 B logical: 512 B type: USB
rev: 3.1 spd: 5 Gb/s lanes: 1 mode: 3.2 gen-1x1 tech: N/A serial: <filter>
fw-rev: D781 scheme: GPT
SMART Message: A mandatory SMART command failed. Various possible causes.
ID-10: /dev/sdh maj-min: 8:112 vendor: Samsung model: SSD 850 PRO 256GB
size: 238.47 GiB block-size: physical: 512 B logical: 512 B type: USB
rev: 3.0 spd: 5 Gb/s lanes: 1 mode: 3.2 gen-1x1 tech: SSD serial: <filter>
fw-rev: TB08 scheme: GPT
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
ID-11: /dev/sdi maj-min: 8:128 vendor: Samsung model: SSD 860 EVO 500GB
size: 465.76 GiB block-size: physical: 512 B logical: 512 B type: USB
rev: 3.0 spd: 5 Gb/s lanes: 1 mode: 3.2 gen-1x1 tech: SSD serial: <filter>
fw-rev: TB08 scheme: GPT
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition:
ID-1: / raw-size: 159.23 GiB size: 159.23 GiB (100.00%)
used: 17.76 GiB (11.2%) fs: btrfs block-size: 4096 B dev: /dev/nvme0n1p5
maj-min: 259:8
ID-2: /boot/efi raw-size: 100 MiB size: 96 MiB (96.00%)
used: 25.9 MiB (27.0%) fs: vfat block-size: 512 B dev: /dev/nvme0n1p1
maj-min: 259:4
ID-3: /home raw-size: 159.23 GiB size: 159.23 GiB (100.00%)
used: 17.76 GiB (11.2%) fs: btrfs block-size: 4096 B dev: /dev/nvme0n1p5
maj-min: 259:8
ID-4: /var/log raw-size: 159.23 GiB size: 159.23 GiB (100.00%)
used: 17.76 GiB (11.2%) fs: btrfs block-size: 4096 B dev: /dev/nvme0n1p5
maj-min: 259:8
ID-5: /var/tmp raw-size: 159.23 GiB size: 159.23 GiB (100.00%)
used: 17.76 GiB (11.2%) fs: btrfs block-size: 4096 B dev: /dev/nvme0n1p5
maj-min: 259:8
Swap:
Kernel: swappiness: 133 (default 60) cache-pressure: 100 (default) zswap: no
ID-1: swap-1 type: zram size: 31.15 GiB used: 0 KiB (0.0%) priority: 100
comp: zstd avail: lzo,lzo-rle,lz4,lz4hc,842 max-streams: 16 dev: /dev/zram0
Sensors:
System Temperatures: cpu: 37.0 C mobo: N/A gpu: nvidia temp: 38 C
Fan Speeds (rpm): N/A gpu: nvidia fan: 33%
Info:
Processes: 397 Uptime: 9m wakeups: 3 Memory: total: 32 GiB
available: 31.15 GiB used: 3.41 GiB (10.9%) Init: systemd v: 254
default: graphical tool: systemctl Compilers: gcc: 13.2.1 clang: 15.0.7
Packages: pm: pacman pkgs: 1658 libs: 419
tools: gnome-software,octopi,pamac,paru Shell: Bash (sudo) v: 5.1.16
running-in: konsole inxi: 3.3.29
Garuda (2.6.16-1):
System install date:     2023-08-25
Last full system update: 2023-08-25
Is partially upgraded:   No
Relevant software:       snapper NetworkManager dracut nvidia-dkms
Windows dual boot:       Yes
Failed units:

Solved:

The missing line was: softdep

/etc/modprobe.d/vfio.conf

softdep drm pre: vfio-pci
options vfio-pci ids=10de:1c82,10de:0fb9

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.