[GTX 1650 4GB] VRAM fragmentation causes nvAssertFailedNoLog in virt_mem_allocator_gm107.c leading to Wayland crash and system deadlock on shutdown

[Mvkweb]

NVIDIA Open GPU Kernel Modules Version

595.71.05

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

• I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Linux 7.0.9-1-Cachyos

Kernel Release

Linux Mvk 7.0.9-1-cachyos #1 SMP PREEMPT_DYNAMIC Mon, 18 May 2026 21:32:51 +0000 x86_64 GNU/Linux

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

• I am running on a stable kernel release.

Hardware: GPU

GPU 0: NVIDIA GeForce GTX 1650 (UUID: GPU-712129c1-1ac3-db2e-3e17-c6807d96a0fa)

Describe the bug

When playing proton games, specifically the game (Wuthering Waves via Proton/DXVK), hitting the 4GB VRAM limit causes catastrophic Virtual Address Space fragmentation in the open kernel module on GM107 architecture.

Instead of gracefully crashing the game with an Out-Of-Memory error, the NVIDIA driver hits severe mapping assertion failures (nvAssertFailedNoLog). This instantly starves and kills the Wayland compositor (kwin_wayland).

Because the NVIDIA kernel module enters a deadlocked panic state, the Linux kernel gets permanently stuck waiting for the GPU during the shutdown sequence. The system fails to power off (screen goes black, but PC fans/lights remain on infinitely) until forced via a hard reset.

This happens even on lowest graphics when using less or half the VRAM. So i can't really find a workaround for this.

To Reproduce

1. Use a 4GB GPU (e.g., GTX 1650) on a Wayland session (KDE Plasma).
2. Launch a VRAM-heavy game (Wuthering Waves via Proton/DXVK).
3. Play until VRAM hits maximum capacity (4GB) and fragments the VA space.
4. Open a GPU accelerated app e.g ghostty or a browser, try to fullscreen, notice freezing as the Wayland context is dropped by kwin_wayland.
5. Attempt to issue a system shutdown. The OS will close user space apps successfully, but the PC will hang infinitely waiting for the NVIDIA driver to release the hardware.

Note: While this is highly reproducible on a 4GB card like the GTX 1650, this appears to be a driver Virtual Address (VA) space mapping/fragmentation issue rather than strictly physical VRAM exhaustion, as it can sometimes trigger even when physical VRAM is not fully saturated.
I cannot try with other GPU's but this is definetely happening on 1650 and likely 1660.

Please also track the logs in your terminal while the game is running, to see it live: sudo journalctl -kf | grep -iE "nvidia|nvkms|fail|alloc"

Or if already closed the game you may check with journalctl -b 0 | grep -iE "nvidia|nvkms|failed|timeout|stop job"

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

Here is the exact chain of events during the crash from journalctl.

1. GPU runs out of space for display objects:
   kernel: [drm:nv_drm_gem_alloc_nvkms_memory_ioctl [nvidia_drm]] ERROR [nvidia-drm] [GPU ID 0x00000100] Failed to allocate NVKMS memory for GEM object

2. Wayland compositor gets starved and drops context:
   kwin_wayland[1057]: QRhiGles2: Failed to make context current. Expect bad things to happen.
   kwin_wayland[1057]: Failed to create RHI (backend 2)

3. NVIDIA driver hits severe VA space fragmentation and assertion fails (deadlock):
   kernel: NVRM: intermapRegisterDmaMapping: Failed to insert new mapping node for range 0xDFB7320000-0xDFB732FFFF!
   kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: NV_OK == status @ gpu_vaspace.c:4631
   kernel: NVRM: GPU0 nvAssertFailedNoLog: Assertion failed: NV_OK == status @ gpu_vaspace.c:2070
   kernel: NVRM: nvCheckFailedNoLog: Check failed: 0 == (pMapNode->gpuMask & gpuMask) @ gpu_vaspace.c:4548
   kernel: NVRM: nvCheckOkFailedNoLog: Check failed: Invalid argument to call [NV_ERR_INVALID_ARGUMENT] (0x0000001F) returned from _gvaspaceMappingInsert(pGVAS, pGpu, pVASBlock, vaLo, vaHi, flags) @ gpu_vaspace.c:2026
   kernel: NVRM: nvCheckFailedNoLog: Check failed: NV_OK == status @ virt_mem_allocator_gm107.c:2552

This loop continues endlessly untill i reboot my pc, if i try to shutdown, it completely breaks the system shutdown and never actually shuts down. It's left on hang.

These are the only relevant logs i have found to this issue of shutdown, but even if it is not from this, it is a real bug where i can't use certain apps while on Wuthering Waves and Proton together.