[AI] Add ONNX Runtime GPU acceleration install flow to AI preferences

[andriiryzhkov]

Adds a GPU ONNX Runtime installer to the AI preferences tab as an alternative to the shell/PowerShell scripts in tools/ai/, for users who'd rather not run them from a terminal. Open preferences → AI tab → click install → restart darktable. Both paths consume the same data/ort_gpu.json manifest and produce the same install layout – they need to stay in sync.

What it does

• Detects GPU (NVIDIA via nvidia-smi, AMD via rocminfo + multi-source ROCm version cascade, Intel via lspci + libze_loader).
• Downloads and SHA-256-verifies the matching archive, extracts via libarchive, probes the installed library to confirm it loads.
• On Windows, bundles cuDNN + CUDA-runtime DLLs from the system toolkit; pulls the OpenVINO runtime as a second download for Intel/Windows.
• On Linux, clears PT_GNU_STACK PF_X on installed .so files so glibc ≥ 2.41 will dlopen AMD's MIGraphX wheels (they ship with the flag set; newer glibc otherwise rejects them silently).
• Auto-redetects after install and writes plugins/ai/ort_library_path.

Parity with the scripts

The two paths are peers, not master/slave. Both must agree on package selection, install paths, checksum verification, archive layout (including preserve_layout for auditwheel-bundled wheels), and post-install validation. They both read data/ort_gpu.json; any change to detection rules, install destinations, or supported variants needs landing in both.

[TurboGit]

That's a lot of code to do something once and especially since it can be done already using a script.

[TurboGit]

Issue 1: Absolute-Path Traversal in Archive Extraction (CRITICAL)

File: ort_install.c:1010

if(preserve_layout)
    out_path = g_build_filename(destdir, name, NULL);

g_build_filename discards destdir when name is an absolute path (e.g., /etc/passwd). The .. check on line 966 (if(strstr(name, ".."))) does NOT catch absolute paths. An attacker who controls the package URL (MITM, or gains write access to ort_gpu.json) can craft an archive that overwrites arbitrary files on the user's system.

Fix: Reject absolute paths before constructing the output path:

if(name[0] == G_DIR_SEPARATOR) continue;  // or g_path_is_absolute(name)

---

Issue 2: CMake HAVE_AI_DOWNLOAD Mismatch (HIGH)

preferences_ai.c gets HAVE_AI_DOWNLOAD from the global add_definitions("-DHAVE_AI_DOWNLOAD") in src/CMakeLists.txt:475, which only checks the USE_AI_DOWNLOAD CMake option. But ort_install.c's real implementation requires CURL_FOUND && LibArchive_FOUND (checked later in src/ai/CMakeLists.txt:78-87). When curl/libarchive are missing, preferences_ai.c still compiles the install button but dt_ort_install_gpu only provides the stub → "download is not available in this build".

Fix: Either move the curl/LibArchive check to the top-level CMakeLists.txt alongside the global definition, or set a global HAVE_AI_DOWNLOAD only when both conditions are met.

---

Issue 3: _install_progress_idle Timer Never Self-Removes (MEDIUM)

ort_install.c lines 1489-1498 / preferences_ai.c ~1560:

if(finished)
    gtk_dialog_response(GTK_DIALOG(ui->dialog), GTK_RESPONSE_OK);
return G_SOURCE_CONTINUE;  // <-- should be G_SOURCE_REMOVE when finished

The timer continues firing every 100ms even after the dialog closes. While g_source_remove(timer) in _step_run_install_with_progress typically catches it, there is a window between gtk_dialog_run returning and g_source_remove where the callback can execute with ui pointing to a destroyed dialog and freed _ort_install_ui_t.

Fix: Return G_SOURCE_REMOVE when finished is true.

---

Issue 4: _verify_sha256 Reads Entire File Into Memory (MEDIUM)

ort_install.c:1066-1083 calls g_file_get_contents(path, &contents, &length, ...) which loads the entire downloaded package into memory. GPU ORT packages can be hundreds of MB. Use a streaming checksum via GChecksum + chunked fread() instead.

---

Issue 5: Unchecked archive_write_data_block Return Value (MEDIUM)

ort_install.c:1032-1033:

while(archive_read_data_block(a, &buf, &size, &offset) == ARCHIVE_OK)
    archive_write_data_block(ext, buf, size, offset);

If the disk is full or the write fails, the error is silently ignored — the extraction reports success (extracted++) with partial data.

---

Issue 6: _extract_archive Path Filter Bypass (MEDIUM)

ort_install.c:960:

if(strstr(name, "..")) continue;

This is a substring match, not a path-component match. foo..bar is falsely rejected while foo/..bar/../baz might pass depending on the specific separator situation. More importantly, this does not use libarchive's own ARCHIVE_EXTRACT_SECURE_NODOTDOT or ARCHIVE_EXTRACT_SECURE_NOABSOLUTEPATH flags on the write-disk handle (line 1025). Even if the manual check passes, libarchive's write-disk machinery would apply its own security checks if configured.

Fix: Add ARCHIVE_EXTRACT_SECURE_NODOTDOT | ARCHIVE_EXTRACT_SECURE_NOABSOLUTEPATH to archive_write_disk_set_options.

---

Issue 7: Missing #include <stdint.h> (LOW)

ort_install.c:1390-1435 uses uint8_t, uint16_t, uint32_t, uint64_t but never explicitly includes <stdint.h>. It compiles only via transitive inclusion from system headers (curl/archive/gio). Explicit include for portability and maintainability.

---

Issue 8: _find_package Can Match on Bad Version Data (LOW)

ort_install.c:341:

int ax = 0, ay = 0, bx = 0, by = 0;
if(a) sscanf(a, "%d.%d", &ax, &ay);

If _get_cuda_version() or _get_rocm_version() returns garbage, sscanf leaves ax/ay at 0, which may cause unexpected filtering (e.g., cuda_min: "12.0" vs detected "0.0" silently skips all packages). The caller should validate version strings before using them.

---

Issue 9: Double Probe on _step_apply_install_result (LOW)

preferences_ai.c lines 1640-1644 (_step_apply_install_result) calls dt_ai_ort_probe_library_full a second time on the same installed_path that dt_ort_install_gpu already validated via _find_valid_library. Redundant probe — the version and EP data could be returned from the install function instead.

[andriiryzhkov]

That's a lot of code to do something once and especially since it can be done already using a script.

Agree. And it's mostly useful on Linux. Both macOS and Windows have solid GPU acceleration bundled by default.

I don't have strong opinion on whether we should include it in the next release. It might be better to release with scripts support initially and collect more feedback from users.