Random GPU usage drop while gaming

System:
Kernel: 5.16.5-zen1-1-zen x86_64 bits: 64 compiler: gcc v: 11.1.0
parameters: BOOT_IMAGE=/@/boot/vmlinuz-linux-zen
root=UUID=2230677d-5ca8-44dd-a833-30d1b43c58a6 rw rootflags=subvol=@
quiet splash rd.udev.log_priority=3 vt.global_cursor_default=0 loglevel=3
Desktop: KDE Plasma 5.23.5 tk: Qt 5.15.2 info: latte-dock wm: kwin_x11
vt: 1 dm: SDDM Distro: Garuda Linux base: Arch Linux
Machine:
Type: Desktop Mobo: Micro-Star model: B450M-A PRO MAX (MS-7C52) v: 1.0
serial: <superuser required> UEFI: American Megatrends LLC. v: 3.C0
date: 05/19/2021
CPU:
Info: model: AMD Ryzen 5 3600 bits: 64 type: MT MCP arch: Zen 2
family: 0x17 (23) model-id: 0x71 (113) stepping: 0 microcode: 0x8701021
Topology: cpus: 1x cores: 6 tpc: 2 threads: 12 smt: enabled cache:
L1: 384 KiB desc: d-6x32 KiB; i-6x32 KiB L2: 3 MiB desc: 6x512 KiB
L3: 32 MiB desc: 2x16 MiB
Speed (MHz): avg: 3876 high: 4173 min/max: 2200/4208 boost: enabled
scaling: driver: acpi-cpufreq governor: performance cores: 1: 3607 2: 3563
3: 4096 4: 3618 5: 3905 6: 4173 7: 4035 8: 4045 9: 3978 10: 3714 11: 3846
12: 3932 bogomips: 86395
Flags: avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm
Vulnerabilities:
Type: itlb_multihit status: Not affected
Type: l1tf status: Not affected
Type: mds status: Not affected
Type: meltdown status: Not affected
Type: spec_store_bypass
mitigation: Speculative Store Bypass disabled via prctl
Type: spectre_v1
mitigation: usercopy/swapgs barriers and __user pointer sanitization
Type: spectre_v2 mitigation: Full AMD retpoline, IBPB: conditional,
STIBP: conditional, RSB filling
Type: srbds status: Not affected
Type: tsx_async_abort status: Not affected
Graphics:
Device-1: NVIDIA TU116 [GeForce GTX 1660 SUPER] vendor: Micro-Star MSI
driver: nvidia v: 510.47.03 alternate: nouveau,nvidia_drm bus-ID: 29:00.0
chip-ID: 10de:21c4 class-ID: 0300
Device-2: KYE Systems (Mouse Systems) Genius Webcam type: USB
driver: snd-usb-audio,uvcvideo bus-ID: 1-5:2 chip-ID: 0458:6006
class-ID: 0102 serial: <filter>
Display: x11 server: X.Org 1.21.1.3 compositor: kwin_x11 driver:
loaded: nvidia unloaded: modesetting alternate: fbdev,nouveau,nv,vesa
display-ID: :0 screens: 1
Screen-1: 0 s-res: 3840x2160 s-dpi: 162 s-size: 602x341mm (23.7x13.4")
s-diag: 692mm (27.2")
Monitor-1: HDMI-0 res: 3840x2160 hz: 60 dpi: 163
size: 600x340mm (23.6x13.4") diag: 690mm (27.2")
OpenGL: renderer: NVIDIA GeForce GTX 1660 SUPER/PCIe/SSE2
v: 4.6.0 NVIDIA 510.47.03 direct render: Yes
Audio:
Device-1: NVIDIA TU116 High Definition Audio vendor: Micro-Star MSI
driver: snd_hda_intel v: kernel bus-ID: 29:00.1 chip-ID: 10de:1aeb
class-ID: 0403
Device-2: AMD Starship/Matisse HD Audio vendor: Micro-Star MSI
driver: snd_hda_intel v: kernel bus-ID: 2b:00.4 chip-ID: 1022:1487
class-ID: 0403
Device-3: KYE Systems (Mouse Systems) Genius Webcam type: USB
driver: snd-usb-audio,uvcvideo bus-ID: 1-5:2 chip-ID: 0458:6006
class-ID: 0102 serial: <filter>
Sound Server-1: ALSA v: k5.16.5-zen1-1-zen running: yes
Sound Server-2: PulseAudio v: 15.0 running: no
Sound Server-3: PipeWire v: 0.3.45 running: yes
Network:
Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
vendor: Micro-Star MSI driver: r8169 v: kernel port: f000 bus-ID: 25:00.0
chip-ID: 10ec:8168 class-ID: 0200
IF: enp37s0 state: up speed: 100 Mbps duplex: full mac: <filter>
Drives:
Local Storage: total: 931.51 GiB used: 163.73 GiB (17.6%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/sda maj-min: 8:0 vendor: Western Digital
model: WDS100T2B0A-00SM50 size: 931.51 GiB block-size: physical: 512 B
logical: 512 B speed: 6.0 Gb/s type: SSD serial: <filter> rev: 00WD
scheme: GPT
Partition:
ID-1: / raw-size: 931.22 GiB size: 931.22 GiB (100.00%)
used: 163.73 GiB (17.6%) fs: btrfs dev: /dev/sda7 maj-min: 8:7
ID-2: /boot/efi raw-size: 300 MiB size: 299.4 MiB (99.80%)
used: 576 KiB (0.2%) fs: vfat dev: /dev/sda6 maj-min: 8:6
ID-3: /home raw-size: 931.22 GiB size: 931.22 GiB (100.00%)
used: 163.73 GiB (17.6%) fs: btrfs dev: /dev/sda7 maj-min: 8:7
ID-4: /var/log raw-size: 931.22 GiB size: 931.22 GiB (100.00%)
used: 163.73 GiB (17.6%) fs: btrfs dev: /dev/sda7 maj-min: 8:7
ID-5: /var/tmp raw-size: 931.22 GiB size: 931.22 GiB (100.00%)
used: 163.73 GiB (17.6%) fs: btrfs dev: /dev/sda7 maj-min: 8:7
Swap:
Kernel: swappiness: 133 (default 60) cache-pressure: 100 (default)
ID-1: swap-1 type: zram size: 15.61 GiB used: 6.2 MiB (0.0%)
priority: 100 dev: /dev/zram0
Sensors:
System Temperatures: cpu: 44.0 C mobo: N/A gpu: nvidia temp: 44 C
Fan Speeds (RPM): fan-1: 0 fan-2: 1880 fan-3: 0 fan-4: 0 fan-5: 0
gpu: nvidia fan: 44%
Power: 12v: N/A 5v: N/A 3.3v: 3.34 vbat: 3.36
Info:
Processes: 319 Uptime: 45m wakeups: 0 Memory: 15.61 GiB
used: 5.45 GiB (34.9%) Init: systemd v: 250 tool: systemctl Compilers:
gcc: 11.1.0 Packages: 1913 pacman: 1906 lib: 541 snap: 7 Shell: fish
v: 3.3.1 default: Bash v: 5.1.16 running-in: konsole inxi: 3.3.12
Garuda (2.5.4-2):
System install date:     2022-01-27
Last full system update: 2022-02-04
Is partially upgraded:   Yes
Relevant software:       NetworkManager
Windows dual boot:       No/Undetected
Snapshots:               Snapper
Failed units:

Hi, while gaming my GPU usage drops below 99-100% and my FPS with it, this happens especially in areas where it is not so much to render. Overall the games run in more FPS than on Windows but this stuttering makes the games kinda unplayable.
And I don't understand why it keeps saying it is partially upgraded, I fully updated the system twice and it keeps saying this. Edit: I think it may be because I downgraded libusb to 1.0.23-3. I m using xow driver and with libusb 1.0.24-2 it uses half of my cpu.