Skip to content

feat: native Wayland support and NixOS dev setup#22

Open
yuuhikaze wants to merge 6 commits into
Open-LLM-VTuber:mainfrom
yuuhikaze:feat/wayland-pet-mode
Open

feat: native Wayland support and NixOS dev setup#22
yuuhikaze wants to merge 6 commits into
Open-LLM-VTuber:mainfrom
yuuhikaze:feat/wayland-pet-mode

Conversation

@yuuhikaze
Copy link
Copy Markdown

@yuuhikaze yuuhikaze commented May 1, 2026

Summary

  • Wayland pet mode: strip { forward: true } from setIgnoreMouseEvents on native Wayland — this flag invokes XFixesSetWindowShapeRegion (X11-only) which corrupts compositor state and logs the user out of Hyprland on drag
  • GPU stability: disable UseChromeOSDirectVideoDecoder, VaapiVideoDecoder, and zero-copy DMA-buf scanout path to prevent Failed to create BO with modifiers GPU process crashes during resize on Wayland
  • Live2D scale fix: stop the animateEase animation loop on convergence (was running forever at 60fps); sync lastScaleRef with the actual model matrix after resize/reload to prevent scale desync causing the avatar to render at wrong size
  • NixOS dev setup: replace manual LD_LIBRARY_PATH export with ELECTRON_EXEC_PATH pointing to pkgs.electron (nixpkgs-patched, native Wayland capable); add .envrc (use flake .) so direnv users get the environment automatically on cd, making npm run dev work with no wrapper

Context

Reported from a NixOS/Hyprland setup running Electron under the native Wayland ozone backend (ELECTRON_OZONE_PLATFORM_HINT=wayland). Dragging the Live2D avatar in pet mode crashed the Wayland session entirely.

Test plan

  • Switch to pet mode — window spans full display(s)
  • Drag the avatar — model repositions freely, Hyprland session does not crash
  • Scroll to resize the avatar — no GPU process crash in logs
  • On NixOS: cd into repo (direnv loads), npm run dev launches without any LD_LIBRARY_PATH export

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation

    • Expanded setup instructions with Nix-based workflow options and environment variable documentation.
  • Bug Fixes

    • Fixed video decoder compatibility issues on Linux Wayland environments.
    • Improved Live2D model scaling synchronization and animation convergence to prevent visual jumps.
  • Chores

    • Added Nix flake configuration and direnv support files for streamlined development environment provisioning.

yuuhikaze and others added 6 commits April 25, 2026 18:59
Add recommended Nix flake workflow for NixOS users, allowing them to quickly
set up the development environment without manual system dependency installation.
Maintain backward compatibility with manual setup instructions.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Detect native Wayland via WAYLAND_DISPLAY/XDG_SESSION_TYPE
- Pet mode: use small repositionable window (400x600) instead of
  full-screen overlay, which relies on the X11 input shape extension
  ({ forward: true }) that does not exist in Wayland and causes
  Hyprland to crash when toggled rapidly during drag operations
- Replace all setIgnoreMouseEvents({ forward: true }) calls with a
  platform-aware helper that omits the flag on Wayland/macOS
- Add movePetWindow() IPC for window repositioning from the renderer
- Renderer drag on Wayland: move the window via BrowserWindow.setPosition()
  instead of manipulating the Live2D model matrix within a full-screen canvas
- Fix GBM buffer object creation failures (GL_INVALID_FRAMEBUFFER_OPERATION,
  gbm_wrapper BO modifier errors) by disabling UseChromeOSDirectVideoDecoder
  and VaapiVideoDecoder on Wayland before app.whenReady()

X11 behavior is unchanged.
The animation loop called requestAnimationFrame unconditionally, running
forever after the first scroll-to-resize and continuously overwriting the
model matrix scale at 60fps.

Added SCALE_EPSILON convergence check so the loop terminates once the
scale difference is negligible. Added readModelScale() to read the
actual model matrix scale and sync lastScaleRef/targetScaleRef in
handleResize and handleWheel, preventing desync after model reloads or
external scale mutations (e.g. onUpdate setWidth) that could cause the
avatar to render at an unexpected size.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the LD_LIBRARY_PATH library list in the flake with
ELECTRON_OVERRIDE_DIST_PATH pointing to pkgs.electron. The NixOS-packaged
Electron has RPATHs baked in for all Nix store paths and Wayland/ozone
support compiled in, eliminating the need for any LD_LIBRARY_PATH export.

Add .envrc (use flake .) so direnv users get the environment automatically
on cd, making npm run dev work with no wrapper. Add .direnv/ to .gitignore.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
electron-vite has its own getElectronPath() that ignores the electron
npm package's ELECTRON_OVERRIDE_DIST_PATH. It checks ELECTRON_EXEC_PATH
first, then falls back to node_modules/electron/dist/electron.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The nixpkgs electron wrapper at bin/electron sets CHROME_DEVEL_SANDBOX,
GDK_PIXBUF_MODULE_FILE, GIO_EXTRA_MODULES and other required env vars
before exec-ing the real binary. Pointing directly to
libexec/electron/electron bypassed all of that, causing silent startup
failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 1, 2026

📝 Walkthrough

Walkthrough

This pull request introduces Nix flake-based development environment setup with direnv integration, updates documentation for Nix workflows, adds Linux Wayland video decoder compatibility fixes, refactors mouse-ignore event handling into a centralized helper, and improves Live2D model scaling convergence and resynchronization logic.

Changes

Cohort / File(s) Summary
Environment Configuration
.envrc, .gitignore, README.md, flake.nix
Establishes Nix flake setup with direnv support for reproducible development environments targeting x86_64-linux and aarch64-linux. Includes NodeJS and Electron provisioning, ignores direnv cache, and documents both direnv-enabled and manual Nix workflows in setup instructions.
Electron Platform Compatibility
src/main/index.ts
Adds conditional Linux Wayland detection to disable problematic video decoders (UseChromeOSDirectVideoDecoder, VaapiVideoDecoder) and zero-copy buffering before app initialization, addressing GBM buffer allocation failures during window resizing.
Input Event Handling
src/main/window-manager.ts
Consolidates platform-specific mouse-ignore behavior into centralized applyIgnoreMouseEvents helper that conditionally applies { forward: true } only on non-Mac, non-Wayland systems. Simplifies pet-mode bounds calculation by inlining width/height computations.
Live2D Rendering & Scaling
src/renderer/src/hooks/canvas/use-live2d-model.ts, src/renderer/src/hooks/canvas/use-live2d-resize.ts
Adds scaling convergence threshold for early loop termination, implements Live2D model matrix synchronization to prevent scale desyncs after reloads or external changes, and normalizes scale refs during wheel and resize events.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Nix shells and flakes now line the warren,
Wayland glitches vanish with the dawn,
Mouse events hop through platforms neat,
Live2D scaling finds its rhythmic beat,
Environment bundled, code runs true!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and clearly summarizes the main changes: native Wayland support and NixOS development setup, which are the primary objectives of the PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 0/1 reviews remaining, refill in 60 minutes.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/main/window-manager.ts (1)

210-235: ⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Wayland still enters the full-screen overlay path instead of the small pet window.

This code unconditionally expands pet mode to the combined bounds of every display, but the PR objective says native Wayland should use a repositionable 400x600 window. Without that branch, applyIgnoreMouseEvents(true) also drops { forward: true } on Wayland while the app is still relying on overlay-style hover/drag behavior, so the native-Wayland flow described in the PR is not actually implemented here.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main/window-manager.ts` around lines 210 - 235, The current pet mode
always expands the window to cover all displays and then calls
applyIgnoreMouseEvents(true), which forces overlay behavior even on native
Wayland; change the logic to detect native Wayland (e.g., XDG_SESSION_TYPE ===
'wayland' or your existing native-Wayland flag) and, when on Wayland, set the
pet window to a repositionable 400x600 rectangle (centered on the primary
display) instead of spanning all displays, do not call
applyIgnoreMouseEvents(true) for the Wayland path (or call it only in a way that
preserves intended hover/drag semantics), and ensure the Wayland branch still
sets the same visibility/resizability/focus flags (the code around
this.window.setBounds, this.window.setResizable, this.window.setSkipTaskbar,
this.window.setFocusable and applyIgnoreMouseEvents should be split so the
full-display expansion runs only for non-Wayland and the 400x600 window creation
runs for Wayland).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@README.md`:
- Around line 29-30: Update the README to reference the actual environment
variable exported by the flake: replace mentions of ELECTRON_OVERRIDE_DIST_PATH
with ELECTRON_EXEC_PATH so docs match flake.nix; search for the README text that
currently says "ELECTRON_OVERRIDE_DIST_PATH" and change it to
"ELECTRON_EXEC_PATH" (and optionally add a brief parenthetical if needed to
clarify it is provided by the flake) to ensure users are directed to the correct
knob.

In `@src/main/index.ts`:
- Around line 7-14: The current env-based predicate falsely assumes Wayland is
in use; replace it with a backend-aware check before app.whenReady() by creating
a shared helper (e.g., isWaylandBackend()) and use it in both src/main/index.ts
(the app.commandLine.appendSwitch block) and src/main/window-manager.ts (the
setIgnoreMouseEvents logic). Implement isWaylandBackend() to detect Electron's
actual backend by checking explicit backend signals available before ready—first
look for explicit ozone flags (process.env.OZONE_PLATFORM === 'wayland' or
process.argv includes '--ozone-platform=wayland'), then fall back to the session
env vars only as a last resort; call that helper to gate the appendSwitch calls
and the setIgnoreMouseEvents behavior so the switches are only disabled when the
Wayland/Ozone backend is actually in use.

---

Outside diff comments:
In `@src/main/window-manager.ts`:
- Around line 210-235: The current pet mode always expands the window to cover
all displays and then calls applyIgnoreMouseEvents(true), which forces overlay
behavior even on native Wayland; change the logic to detect native Wayland
(e.g., XDG_SESSION_TYPE === 'wayland' or your existing native-Wayland flag) and,
when on Wayland, set the pet window to a repositionable 400x600 rectangle
(centered on the primary display) instead of spanning all displays, do not call
applyIgnoreMouseEvents(true) for the Wayland path (or call it only in a way that
preserves intended hover/drag semantics), and ensure the Wayland branch still
sets the same visibility/resizability/focus flags (the code around
this.window.setBounds, this.window.setResizable, this.window.setSkipTaskbar,
this.window.setFocusable and applyIgnoreMouseEvents should be split so the
full-display expansion runs only for non-Wayland and the 400x600 window creation
runs for Wayland).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 412b6a53-f094-4420-a4b5-6a3b7a69beef

📥 Commits

Reviewing files that changed from the base of the PR and between d176e7d and 4a9b352.

⛔ Files ignored due to path filters (1)
  • flake.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • .envrc
  • .gitignore
  • README.md
  • flake.nix
  • src/main/index.ts
  • src/main/window-manager.ts
  • src/renderer/src/hooks/canvas/use-live2d-model.ts
  • src/renderer/src/hooks/canvas/use-live2d-resize.ts

Comment thread README.md
Comment on lines +29 to +30
The flake provides a NixOS-patched Electron binary via `ELECTRON_OVERRIDE_DIST_PATH`,
so no `LD_LIBRARY_PATH` exports are needed.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix env var name mismatch in docs (ELECTRON_OVERRIDE_DIST_PATH vs ELECTRON_EXEC_PATH).

Line 29 currently documents ELECTRON_OVERRIDE_DIST_PATH, but flake.nix exports ELECTRON_EXEC_PATH. This will send users to the wrong knob.

Suggested doc fix
-The flake provides a NixOS-patched Electron binary via `ELECTRON_OVERRIDE_DIST_PATH`,
+The flake provides a NixOS-patched Electron binary via `ELECTRON_EXEC_PATH`,
 so no `LD_LIBRARY_PATH` exports are needed.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
The flake provides a NixOS-patched Electron binary via `ELECTRON_OVERRIDE_DIST_PATH`,
so no `LD_LIBRARY_PATH` exports are needed.
The flake provides a NixOS-patched Electron binary via `ELECTRON_EXEC_PATH`,
so no `LD_LIBRARY_PATH` exports are needed.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` around lines 29 - 30, Update the README to reference the actual
environment variable exported by the flake: replace mentions of
ELECTRON_OVERRIDE_DIST_PATH with ELECTRON_EXEC_PATH so docs match flake.nix;
search for the README text that currently says "ELECTRON_OVERRIDE_DIST_PATH" and
change it to "ELECTRON_EXEC_PATH" (and optionally add a brief parenthetical if
needed to clarify it is provided by the flake) to ensure users are directed to
the correct knob.

Comment thread src/main/index.ts
Comment on lines +7 to +14
// Fix GBM buffer object creation failures with DRM format modifiers on Wayland.
// These switches must be set before app.whenReady().
if (process.platform === 'linux' && (process.env.WAYLAND_DISPLAY || process.env.XDG_SESSION_TYPE === 'wayland')) {
app.commandLine.appendSwitch('disable-features', 'UseChromeOSDirectVideoDecoder,VaapiVideoDecoder');
// Disable DMA-buf zero-copy scanout path that triggers GBM BO modifier allocation failures
// on some GPU/driver combinations when Electron resizes GPU-backed surfaces on Wayland.
app.commandLine.appendSwitch('disable-zero-copy');
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

For Electron on Linux, do WAYLAND_DISPLAYorXDG_SESSION_TYPE=wayland guarantee that the app is using the native Wayland backend, or can Electron still run under X11/XWayland unless Ozone platform flags or environment variables are set?

💡 Result:

They do not guarantee it. On Linux, Electron’s choice between native Wayland and X11/XWayland depends on Ozone platform selection (historically via flags and/or environment variables), and environment variables like WAYLAND_DISPLAY and XDG_SESSION_TYPE=wayland can be insufficient by themselves. 1) What the env vars do (and don’t) do - WAYLAND_DISPLAY and/or XDG_SESSION_TYPE=wayland indicate you are in a Wayland session, but they are not, on their own, a contractual “Electron must use the native Wayland Ozone backend” guarantee. - Electron’s backend selection is controlled by Ozone platform settings (e.g. --ozone-platform=wayland or --ozone-platform=x11), and/or Ozone platform “hint” logic that inspects XDG_SESSION_TYPE (or had similar behavior historically). 2) Native Wayland in Electron depends on Ozone/flags (and can still fall back to XWayland) - Electron documents that running on Wayland sessions historically did not automatically mean native Wayland; instead apps could use X11 via Xwayland unless the Ozone/Wayland platform path is selected. It explicitly contrasts “native Wayland” vs “running inside an invisible X server called Xwayland.” - Electron 38.0.0+ introduced native Wayland by default in Wayland sessions, but you can still force X11 compatibility mode with --ozone-platform=x11. 3) Evidence that “setting WAYLAND/XDG vars” alone can still result in XWayland - Real-world reports show Electron apps can choose XWayland even when started in a Wayland session, due to launchers/packaging setting Ozone hints to auto (or using other launcher logic), or due to Electron/version-specific Wayland detection issues. Practical conclusion - If you need to ensure native Wayland (no XWayland), set Electron explicitly to Wayland using Ozone flags, e.g. --enable-features=UseOzonePlatform --ozone-platform=wayland (or set the relevant Ozone hint/auto logic if your Electron version supports it as the default). - Do not assume that merely having WAYLAND_DISPLAY set or exporting XDG_SESSION_TYPE=wayland will always force native Wayland in Electron; Electron can still run via X11/XWayland unless its Ozone platform selection is aligned with Wayland. How to verify at runtime - Check whether the window is registered as an X11/XWayland surface (e.g., using tools like xwininfo/xprop/xlsclients) rather than relying solely on session env vars. - If you see X11/XWayland indicators, you’ll need to adjust Electron launch options (Ozone flags/hints) for your specific app/version.

Citations:


🏁 Script executed:

rg -n "WAYLAND_DISPLAY|XDG_SESSION_TYPE" src/main/window-manager.ts -A 3 -B 3

Repository: Open-LLM-VTuber/Open-LLM-VTuber-Web

Length of output: 352


🏁 Script executed:

rg -n "isWayland" src/main/window-manager.ts -A 2 -B 2

Repository: Open-LLM-VTuber/Open-LLM-VTuber-Web

Length of output: 471


Gate these switches on Electron's actual backend, not just the desktop session.

WAYLAND_DISPLAY / XDG_SESSION_TYPE=wayland do not guarantee that Electron is using the native Wayland backend—Electron can still run under X11/XWayland. This predicate disables decoders and zero-copy on the wrong backend. The same unreliable check is duplicated in src/main/window-manager.ts (lines 8–10), where it controls setIgnoreMouseEvents() behavior (line 246), so a false positive here becomes a cross-file regression affecting both GPU decoding and mouse-event forwarding.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/main/index.ts` around lines 7 - 14, The current env-based predicate
falsely assumes Wayland is in use; replace it with a backend-aware check before
app.whenReady() by creating a shared helper (e.g., isWaylandBackend()) and use
it in both src/main/index.ts (the app.commandLine.appendSwitch block) and
src/main/window-manager.ts (the setIgnoreMouseEvents logic). Implement
isWaylandBackend() to detect Electron's actual backend by checking explicit
backend signals available before ready—first look for explicit ozone flags
(process.env.OZONE_PLATFORM === 'wayland' or process.argv includes
'--ozone-platform=wayland'), then fall back to the session env vars only as a
last resort; call that helper to gate the appendSwitch calls and the
setIgnoreMouseEvents behavior so the switches are only disabled when the
Wayland/Ozone backend is actually in use.

@yuuhikaze
Copy link
Copy Markdown
Author

yuuhikaze commented May 6, 2026

I'll address the review soon

thr3a pushed a commit to thr3a/Open-LLM-VTuber-Web that referenced this pull request May 10, 2026
…Open-LLM-VTuber#22)

* update

* smooth-action

---------

Co-authored-by: MrXnneHang <XnneHang@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant