Everything renders on GPU0 despite image being on monitor connected to GPU1

[DZHEX]

So, once again, a weird issue with specific details…
I have two GTX1080tis (GPU0 in PCIe1 and GPU1 in PCIe4) with 3 monitors/displays (and VR) connected to them, the monitor I am playing on is connected to GPU1, but I am only getting load on GPU0 - it seems that GPU1 is just handing everything over to GPU0, which is generating a lot of heat, since it’s mounted pretty tightly with the other GPU (a different mount is in the works, but that’s a different problem). I tried the prime-run script, which didn’t seem to do much of anything.

Is there a way to make an application to use the GPU that the monitor is connected to? Tried to make this work with BAR (appimage) and minecraft, zero load on GPU1.
Aside from misc searching, been looking at this, but not getting much help: Making sure you're not a bot!

garuda-inxi                                                                                                                                                                   (base)
System:
Kernel: 6.14.4-zen1-2-zen arch: x86_64 bits: 64 compiler: gcc v: 15.1.1
clocksource: tsc avail: hpet,acpi_pm
parameters: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen
root=UUID=dce9a305-635c-4c6b-856f-79a5114c2f5e rw rootflags=subvol=@
loglevel=3 ibt=off nvidia-drm.modeset=1 nvidia_drm.fbdev=1
Desktop: KDE Plasma v: 6.3.4 tk: Qt v: N/A info: frameworks v: 6.13.0
wm: kwin_wayland tools: xscreensaver,xscreensaver-systemd vt: 1 dm: SDDM
Distro: Garuda base: Arch Linux
Machine:
Type: Desktop Mobo: ASUSTeK model: ROG STRIX Z370-E GAMING v: Rev X.0x
serial: <superuser required> part-nu: SKU uuid: <superuser required>
UEFI: American Megatrends v: 3005 date: 09/30/2021
Battery:
ID-1: hidpp_battery_0 charge: 100% condition: N/A volts: 4.2 min: N/A
model: Logitech G903 LIGHTSPEED Wireless Gaming Mouse w/ HERO type: N/A
serial: <filter> status: full
CPU:
Info: model: Intel Core i7-8700K bits: 64 type: MT MCP arch: Coffee Lake
gen: core 8 level: v3 note: check built: 2018 process: Intel 14nm family: 6
model-id: 0x9E (158) stepping: 0xA (10) microcode: 0xFA
Topology: cpus: 1x dies: 1 clusters: 6 cores: 6 threads: 12 tpc: 2
smt: enabled cache: L1: 384 KiB desc: d-6x32 KiB; i-6x32 KiB L2: 1.5 MiB
desc: 6x256 KiB L3: 12 MiB desc: 1x12 MiB
Speed (MHz): avg: 4901 min/max: 800/4700:4900 scaling:
driver: intel_pstate governor: performance cores: 1: 4901 2: 4901 3: 4901
4: 4901 5: 4901 6: 4901 7: 4901 8: 4901 9: 4901 10: 4901 11: 4901 12: 4901
bogomips: 88796
Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Vulnerabilities: <filter>
Graphics:
Device-1: Intel CoffeeLake-S GT2 [UHD Graphics 630] vendor: ASUSTeK
driver: i915 v: kernel arch: Gen-9.5 process: Intel 14nm built: 2016-20
ports: active: none off: HDMI-A-6 empty: DP-5,HDMI-A-5,HDMI-A-7
bus-ID: 00:02.0 chip-ID: 8086:3e92 class-ID: 0380
Device-2: NVIDIA GP102 [GeForce GTX 1080 Ti] vendor: ASUSTeK
driver: nvidia v: 570.144 alternate: nouveau,nvidia_drm
non-free: 550-570.xx+ status: current (as of 2025-04; EOL~2026-12-xx)
arch: Pascal code: GP10x process: TSMC 16nm built: 2016-2021 pcie: gen: 3
speed: 8 GT/s lanes: 8 link-max: lanes: 16 ports: active: none
off: DP-1,DP-2 empty: DVI-D-1,HDMI-A-1,HDMI-A-2 bus-ID: 01:00.0
chip-ID: 10de:1b06 class-ID: 0300
Device-3: NVIDIA GP102 [GeForce GTX 1080 Ti] vendor: ASUSTeK
driver: nvidia v: 570.144 alternate: nouveau,nvidia_drm
non-free: 550-570.xx+ status: current (as of 2025-04; EOL~2026-12-xx)
arch: Pascal code: GP10x process: TSMC 16nm built: 2016-2021 pcie: gen: 3
speed: 8 GT/s lanes: 8 link-max: lanes: 16 ports: active: none
off: DP-3,DP-4 empty: DVI-D-2,HDMI-A-3,HDMI-A-4 bus-ID: 02:00.0
chip-ID: 10de:1b06 class-ID: 0300
Device-4: Valve 3D Camera driver: uvcvideo type: USB rev: 3.2
speed: 5 Gb/s lanes: 1 mode: 3.2 gen-1x1 bus-ID: 2-6.1:3 chip-ID: 28de:2400
class-ID: 0e02 serial: <filter>
Device-5: Logitech HD Webcam C615 driver: snd-usb-audio,uvcvideo type: USB
rev: 2.0 speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 3-1.2.2:6
chip-ID: 046d:082c class-ID: 0e02 serial: <filter>
Display: wayland server: X.org v: 1.21.1.16 with: Xwayland v: 24.1.6
compositor: kwin_wayland driver: X: loaded: modesetting,nvidia
alternate: fbdev,intel,nouveau,nv,vesa dri: iris
gpu: i915,nvidia,nvidia-nvswitch display-ID: 0
Monitor-1: DP-1 model: Nvidia built: 1990 size-res: N/A gamma: 1.2
modes: 640x480
Monitor-2: DP-2 model: AOC Q27G2WG4 serial: <filter> built: 2021 res:
mode: 2560x1440 hz: 120 scale: 100% (1) dpi: 109 gamma: 1.2
size: 597x336mm (23.5x13.23") diag: 685mm (27") ratio: 16:9 modes:
max: 2560x1440 min: 640x480
Monitor-3: DP-3 model: AOC Q27G2WG4 serial: <filter> built: 2021 res:
mode: 2560x1440 hz: 120 scale: 100% (1) dpi: 109 gamma: 1.2
size: 597x336mm (23.5x13.23") diag: 685mm (27") ratio: 16:9 modes:
max: 2560x1440 min: 640x480
Monitor-4: DP-4 model: Gigabyte G32QC A serial: <filter> built: 2021 res:
mode: 2560x1440 hz: 165 scale: 100% (1) dpi: 93 gamma: 1.2
size: 697x392mm (27.44x15.43") diag: 800mm (31.5") ratio: 16:9 modes:
max: 2560x1440 min: 640x480
Monitor-5: HDMI-A-6 model: Sony TV serial: <filter> built: 2013
res: 1920x1080 dpi: 69 gamma: 1.2 size: 708x398mm (27.87x15.67")
diag: 812mm (32") ratio: 16:9 modes: max: 1920x1080 min: 640x480
API: EGL v: 1.5 hw: drv: intel iris drv: nvidia nouveau drv: nvidia
platforms: device: 0 drv: nvidia device: 1 drv: nvidia device: 2 drv: iris
device: 3 drv: nouveau device: 4 drv: nouveau device: 5 drv: swrast gbm:
drv: iris surfaceless: drv: nvidia wayland: drv: nvidia x11: drv: nvidia
API: OpenGL v: 4.6.0 compat-v: 4.5 vendor: nvidia mesa v: 570.144
glx-v: 1.4 direct-render: yes renderer: NVIDIA GeForce GTX 1080 Ti/PCIe/SSE2
memory: 10.74 GiB display-ID: :1.0
API: Vulkan v: 1.4.309 layers: 9 device: 0 type: discrete-gpu name: NVIDIA
GeForce GTX 1080 Ti driver: nvidia v: 570.144 device-ID: 10de:1b06
surfaces: xcb,xlib,wayland device: 1 type: discrete-gpu name: NVIDIA
GeForce GTX 1080 Ti driver: nvidia v: 570.144 device-ID: 10de:1b06
surfaces: wayland
Info: Tools: api: clinfo, eglinfo, glxinfo, vulkaninfo
de: kscreen-console,kscreen-doctor gpu: corectrl, nvidia-settings,
nvidia-smi wl: wayland-info x11: xdpyinfo, xprop, xrandr
Audio:
Device-1: Intel 200 Series PCH HD Audio vendor: ASUSTeK
driver: snd_hda_intel v: kernel alternate: snd_soc_avs bus-ID: 00:1f.3
chip-ID: 8086:a2f0 class-ID: 0403
Device-2: NVIDIA GP102 HDMI Audio vendor: ASUSTeK driver: snd_hda_intel
v: kernel pcie: gen: 3 speed: 8 GT/s lanes: 8 link-max: lanes: 16
bus-ID: 01:00.1 chip-ID: 10de:10ef class-ID: 0403
Device-3: NVIDIA GP102 HDMI Audio vendor: ASUSTeK driver: snd_hda_intel
v: kernel pcie: gen: 3 speed: 8 GT/s lanes: 8 link-max: lanes: 16
bus-ID: 02:00.1 chip-ID: 10de:10ef class-ID: 0403
Device-4: Valve VR Radio driver: cdc_acm,hid-generic,usbhid type: USB
rev: 2.0 speed: 12 Mb/s lanes: 1 mode: 1.1 bus-ID: 1-6.3.2:8
chip-ID: 28de:2102 class-ID: 0a00 serial: <filter>
Device-5: Valve VR Radio & HMD Mic
driver: hid-generic,snd-usb-audio,usbhid type: USB rev: 2.0 speed: 12 Mb/s
lanes: 1 mode: 1.1 bus-ID: 1-6.3.3:11 chip-ID: 28de:2102 class-ID: 0102
serial: <filter>
Device-6: Logitech HD Webcam C615 driver: snd-usb-audio,uvcvideo type: USB
rev: 2.0 speed: 480 Mb/s lanes: 1 mode: 2.0 bus-ID: 3-1.2.2:6
chip-ID: 046d:082c class-ID: 0e02 serial: <filter>
Device-7: Kingston HyperX QuadCast S
driver: hid-generic,snd-usb-audio,usbhid type: USB rev: 1.1 speed: 12 Mb/s
lanes: 1 mode: 1.1 bus-ID: 3-1.4.4.1:11 chip-ID: 0951:171d class-ID: 0300
serial: <filter>
API: ALSA v: k6.14.4-zen1-2-zen status: kernel-api with: aoss
type: oss-emulator tools: N/A
Server-1: PipeWire v: 1.4.2 status: active with: 1: pipewire-pulse
status: active 2: wireplumber status: active 3: pipewire-alsa type: plugin
4: pw-jack type: plugin tools: pactl,pw-cat,pw-cli,wpctl
Network:
Device-1: Intel Ethernet I219-V vendor: ASUSTeK driver: e1000e v: kernel
port: N/A bus-ID: 00:1f.6 chip-ID: 8086:15b8 class-ID: 0200
IF: enp0s31f6 state: up speed: 100 Mbps duplex: full mac: <filter>
Device-2: Realtek RTL8822BE 802.11a/b/g/n/ac WiFi adapter vendor: ASUSTeK
driver: rtw_8822be v: N/A modules: rtw88_8822be pcie: gen: 1 speed: 2.5 GT/s
lanes: 1 port: c000 bus-ID: 05:00.0 chip-ID: 10ec:b822 class-ID: 0280
IF: wlp5s0 state: up mac: <filter>
Info: services: NetworkManager, smbd, systemd-timesyncd, wpa_supplicant
Bluetooth:
Device-1: ASUSTek Bluetooth Radio driver: btusb v: 0.8 type: USB rev: 1.1
speed: 12 Mb/s lanes: 1 mode: 1.1 bus-ID: 1-7:3 chip-ID: 0b05:185c
class-ID: e001 serial: <filter>
Report: btmgmt ID: hci0 rfk-id: 0 state: up address: <filter> bt-v: 4.2
lmp-v: 8 status: discoverable: no pairing: no class-ID: 6c0104
Drives:
Local Storage: total: 3.18 TiB used: 316.31 GiB (9.7%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Samsung model: SSD 960 EVO 500GB
size: 465.76 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
lanes: 4 tech: SSD serial: <filter> fw-rev: 3B7QCXE7 temp: 42.9 C
scheme: GPT
ID-2: /dev/sda maj-min: 8:0 vendor: Samsung model: SSD 860 EVO 1TB
size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s
tech: SSD serial: <filter> fw-rev: 1B6Q scheme: GPT
ID-3: /dev/sdb maj-min: 8:16 vendor: Seagate model: ST2000DX002-2DV164
size: 1.82 TiB block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s
tech: HDD rpm: 7200 serial: <filter> fw-rev: CC41 scheme: GPT
Partition:
ID-1: / raw-size: 120.03 GiB size: 120.03 GiB (100.00%)
used: 57.32 GiB (47.8%) fs: btrfs dev: /dev/nvme0n1p7 maj-min: 259:7
ID-2: /boot/efi raw-size: 300 MiB size: 299.4 MiB (99.80%)
used: 612 KiB (0.2%) fs: vfat dev: /dev/nvme0n1p8 maj-min: 259:8
ID-3: /home raw-size: 200.32 GiB size: 200.32 GiB (100.00%)
used: 193.56 GiB (96.6%) fs: btrfs dev: /dev/sda3 maj-min: 8:3
ID-4: /var/log raw-size: 120.03 GiB size: 120.03 GiB (100.00%)
used: 57.32 GiB (47.8%) fs: btrfs dev: /dev/nvme0n1p7 maj-min: 259:7
ID-5: /var/tmp raw-size: 120.03 GiB size: 120.03 GiB (100.00%)
used: 57.32 GiB (47.8%) fs: btrfs dev: /dev/nvme0n1p7 maj-min: 259:7
Swap:
Kernel: swappiness: 133 (default 60) cache-pressure: 100 (default) zswap: no
ID-1: swap-1 type: zram size: 31.19 GiB used: 9.84 GiB (31.5%)
priority: 100 comp: zstd avail: lzo-rle,lzo,lz4,lz4hc,deflate,842
max-streams: 12 dev: /dev/zram0
ID-2: swap-2 type: partition size: 31.3 GiB used: 0 KiB (0.0%)
priority: -2 dev: /dev/nvme0n1p6 maj-min: 259:6
Sensors:
System Temperatures: cpu: 51.0 C mobo: N/A
Fan Speeds (rpm): N/A
Info:
Memory: total: 32 GiB available: 31.19 GiB used: 23.55 GiB (75.5%)
Processes: 559 Power: uptime: 3d 19h 36m states: freeze,mem,disk
suspend: deep avail: s2idle wakeups: 0 hibernate: platform avail: shutdown,
reboot, suspend, test_resume image: 12.4 GiB services: org_kde_powerdevil,
power-profiles-daemon, upowerd Init: systemd v: 257 default: graphical
tool: systemctl
Packages: 2290 pm: pacman pkgs: 2273 libs: 611 tools: octopi,pamac,paru
pm: flatpak pkgs: 17 Compilers: clang: 19.1.7 gcc: 15.1.1 Shell: garuda-inxi
default: fish v: 4.0.2 running-in: konsole inxi: 3.3.38
Garuda (2.7.2-1):
System install date:     2024-04-25
Last full system update: 2025-05-08 ↻
Is partially upgraded:   No
Relevant software:       snapper NetworkManager dracut nvidia-dkms
Windows dual boot:       Probably (Run as root to verify)
Failed units:

[gampel]

nvidia-prime

• also set __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0 for the first gpu or __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G1 for the second
  (for vulkan apps is this not enough, what i know)

• for vulkan you could install vulkan device chooser layer
  Can´t test this, sorry no 2x nvidia gpus
  GitHub - aejsmith/vkdevicechooser: Vulkan layer to force a specific device to be used
  paru -S vkdevicechooser-git

• about nvidia-smi you see both gpus ?

• log file /var/log/Xorg.0.log no probs ?

• over nvidia-settings (with root rights) = ?

• you use wayland and if you use X11 result = ?
  Perhaps inside X11 you need a /etc/X11/xorg.conf.d/xorg.conf file for both nvidia gpus.
  2x device section with the correct bus-id + your screen sections (depedands from your hardware)

Section "Device"                                         -->  Section "Device"
    Identifier     "Device0"                                   Identifier "Device1"
    Driver         "nvidia"                                 -->         
    VendorName     "NVIDIA Corporation"  -->        
    BusID          "1:0:0" (your id)                           BusID "2:0:0" (your id)
EndSection                                                   -->

[DZHEX]

• no change
• no change
• I can see both of them in nvidia-smi:

Fri May  9 22:37:52 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.144                Driver Version: 570.144        CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1080 Ti     Off |   00000000:01:00.0  On |                  N/A |
| 36%   64C    P0             95W /  275W |    4536MiB /  11264MiB |     33%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce GTX 1080 Ti     Off |   00000000:02:00.0  On |                  N/A |
| 23%   43C    P0             64W /  275W |     107MiB /  11264MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

• errors loading modules mesa, noveau, nv, fbdev, vesa (module does not exist, 0), looks like modules “nvidia” and “modesetting” did load. Also some errors about “parent device not found” for my mouse and mic and whatnot, but nothing marked as error that seemed relevant. I did see the following near the end of the log file:

[   629.351] (WW) NVIDIA(G0): Failed to set the display configuration
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 0 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 1 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 2 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 3 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0): Failed to set the display configuration
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 0 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 1 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 2 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0):  - Setting a mode on head 3 failed: Insufficient permissions
[   629.351] (WW) NVIDIA(G0): Failed to set DPMS to off

• nvidia-settings not really applicable on wayland
• X11 - I was seeing some load on GPU1, but it still looked like GPU0 was doing all the heavy lifting, despite any variables I tried to set. I didn’t get into configuring xorg.conf yet - I would want something that works with wayland. With wayland, load actually reads as zero, which kinda doesn’t make sense…

Edit with additional info:
It seems that this is some sort of early boot thing - if the BIOS screen and, subsequently, GRUB (yes, the bootloader) shows on a monitor that was running on GPU1, then everything is rendered on GPU1 and GPU0 is getting zero load. Since the weird problem I had with wayland (monitors on non-grub GPU looking like a memdump with cursor over it/them) is no longer present (got fixed), this works as a very stupid and very fragile hack, but I am beginning to feel like there might be something that I need to configure in the bios and it will magic this problem away…

fuuuu*k, I don’t want to get to “just buy a new mobo”…

[gampel]

Did you try to : DRI_PRIME=pci-0000_02_00_0 ?
For nvidia ↔ wayland with 2x nvidia gpu current i found this info

[SOLVED] Multiple Nvidia GPUs with 5 screens in Wayland / Kernel &...

But i`m not a arch nvidia guru..perhaps other knows more.

[DZHEX]

I did try it, the syntax of the PCI port was kinda confusing, since in most other places they use colons instead of underscores (and the amount of digits was different), but none of the parameters seem to have any effect in wayland.
Your link refers to the other problem I had, which has since been resolved with a driver update:

Wayland black/weird/colorful/glitchy screen on GPU2, with cursor (KDE)

Hello, I’m having a pretty significant problem with Wayland on KDE plasma - I have two GPUs, in PCIe slots 1 (“GPU1”) and 2 (“GPU2”), and 3 displays connected to them, two (“sides”) to the GPU1 and third (“main”/“primary”/“center”) to GPU2. The displays connected to GPU1 work fine, but the one connected to the GPU2 some time ago was showing a black screen, but now it usually looks like some sort of graphical memory dump (different on every session). The windows that are supposed to be on that s…

…from that angle, it’s starting to look like I just have to wait for someone who can actually do it, to make multi-GPU setups work actually, not just technically