Fix DLSS on Windows: CUDA interop warm-up and NGX application identity #6

Open
sajittarius wants to merge 2 commits from sajittarius/the-beast-roofer-edition:dlss-cuda-warmup-and-ngx-identity into master
2 Commits
Author SHA1 Message Date
sajittariusandClaude Fable 5 b882b46e53 Remove diagnostic checkpoints left from the DLSS investigation
Drops the RYUJINX_VK_DIAG lifecycle probes (Program.cs, AppHost.cs,
VulkanInitialization.cs) and the failure-time feature-bisect loop in
CreateDevice; they existed only to locate the CUDA init failure. Strips
VkDiagProbe down to the WarmUpCudaInterop path that the fix uses.

The staged device-creation fallback and per-extension probe stay: they
are the runtime safety net when a driver refuses an extension.

Verified after cleanup: warm-up OK, optical-flow queue acquired, DLSS
available on a plain launch with no env vars.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-26 20:33:35 -04:00
sajittariusandClaude Fable 5 45d51ccf34 Fix DLSS: CUDA interop warm-up and NGX application identity
DLSS never worked on Windows: vkCreateDevice refused VK_NVX_binary_import
and VK_NV_optical_flow with ErrorInitializationFailed, and NGX refused to
initialize even when the extensions were present.

Root cause 1: once guest process creation (Device.LoadNsp) reserves the
guest address space, cuInit fails with CUDA_ERROR_OUT_OF_MEMORY for the
rest of the process lifetime, so the driver refuses the two CUDA/OFA-
backed device extensions. CUDA state initialized before the reservation
stays valid all session. Fix: the VulkanRenderer constructor (which runs
before game load) creates and destroys a throwaway device requesting
VK_NVX_binary_import when DLSS is enabled, forcing CUDA init first.

Root cause 2: Streamline.Initialize passed no application identity, and
NGX disables all NGX-based features without one ("Please provide correct
application id when calling slInit" in sl.log). Fix: pass the Streamline
sample ApplicationId plus a ProjectId GUID and EngineVersion, selecting
the NGX project-id init path.

Also in this change:

- Staged device-creation fallback: retry without optical flow, then
  probe each DLSS device extension individually and keep the accepted
  ones, instead of crashing on ErrorInitializationFailed.
- GPU selection: prefer a discrete GPU in the fallback path and log why
  a preferred device was skipped.
- VkDiagProbe: raw P/Invoke probe used for the warm-up, plus lifecycle
  diagnostic checkpoints gated on RYUJINX_VK_DIAG (off by default),
  appending to vkdiag.txt next to the exe.

Verified end to end on an RTX 4090 (driver 616.56): all 30 device
extensions accepted, dedicated optical-flow queue acquired, DLSS
reports available, and DLSS4 transformer upscaling runs in-game.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-26 20:30:04 -04:00