Open
Conversation
The NVIDIA NVENC encoder code already exists in webrtc-sys (h264/h265 encoder impls, cuda_context with Windows LoadLibrary support, NvCodec SDK) but was only compiled on Linux. This enables it on Windows too. Requires CUDA Toolkit headers at build time (set CUDA_HOME or use default install path). At runtime, loads nvcuda.dll from the GPU driver. The VideoEncoderFactory composite pattern automatically tries NVIDIA first, falling back to software encoders if CUDA is unavailable.
Include winsock2.h before cuda.h in all NVIDIA headers to prevent winsock.h/winsock2.h struct redefinition errors when compiling with WebRTC headers on Windows. Also fix NvCodec Logger.h to use winsock2.h instead of winsock.h. Note: build still fails due to Chromium libc++ include path pulling in windows.h without WIN32_LEAN_AND_MEAN. Needs separate cc::Build for NVIDIA files or patched prebuilt headers.
The NVIDIA NVENC encoder code compiles fine with manual cl.exe but fails when compiled via cc-rs due to the crate's INCLUDE env var construction causing Windows SDK header conflicts. All NVIDIA source patches and build.rs changes are in place. The conflict is between cc-rs's environment and Windows SDK 26100. Patched files: nvEncodeAPI.h, NvEncoder.cpp, Logger.h, cuda_context, encoder impl headers. Added win_compat.h and factory bridge.
The NVIDIA encoder files compile successfully when the build runs from a Visual Studio Developer Command Prompt (vcvarsall x64). The winsock.h/winsock2.h conflict was caused by cc-rs's find-msvc-tools constructing a different INCLUDE environment than vcvarsall. Build requirements: - CUDA Toolkit installed (CUDA_HOME or auto-detected) - Run cargo from a VS Developer Command Prompt (vcvarsall x64) Header patches for robustness: - nvEncodeAPI.h, NvEncoder.cpp, Logger.h, cuda_context.cpp: All patched to use WIN32_LEAN_AND_MEAN before windows.h - win_compat.h: Establishes correct winsock2.h include order - nvidia_factory_bridge: Safe factory bridge without CUDA headers
The build.rs now automatically runs vcvarsall.bat and captures the resulting INCLUDE/LIB/VCINSTALLDIR env vars when not already in a Developer Command Prompt. This means NVENC compilation works from any terminal — no need to manually run vcvarsall first. Also cleaned up debug logging from previous iterations.
Revert to original USE_NVIDIA_VIDEO_CODEC define (encoder + decoder). The nvcuvid.lib import library is generated from nvcuvid.dll (ships with the NVIDIA GPU driver) and bundled in lib/x64/ since it's not part of the CUDA Toolkit. This enables: - NVENC H264/H265 hardware encoding (screenshare) - NVDEC H264/H265/AV1 hardware decoding (receiving streams)
- Changed H265 profile from Main to Main 10 (NV_ENC_HEVC_PROFILE_MAIN10_GUID) - Set 10-bit output pixel depth (pixelBitDepthMinus8 = 2) - Added VUI parameters for HDR signaling: - Color primaries: BT.2020 (9) - Transfer characteristics: ST.2084/PQ (16) - Matrix coefficients: BT.2020 non-constant (9) - NVENC accepts 8-bit I420 input and internally upscales to 10-bit This means H265 streams now carry HDR metadata in the HEVC SPS, allowing receivers with HDR displays to render in wide color gamut.
Both H264 and H265 encoders stored the new bitrate/framerate from WebRTC's bandwidth estimator but never actually reconfigured the running NVENC encoder. This caused the encoder to keep producing at the initial bitrate regardless of network conditions, leading to progressive jitter buffer growth on receivers. Now calls NvEncoder::Reconfigure() in SetRates() to dynamically adjust bitrate and framerate, enabling proper WebRTC congestion control.
…ottles The WebRTC bandwidth estimator aggressively reduces bitrate from 120kbps to 27kbps over 30 seconds (even on localhost), causing the encoder to produce only 17fps. Reverting to fixed initial bitrate which maintained stable 55fps. The root cause is the bandwidth estimator's interaction with the loopback path. Proper fix needs adaptive streaming support in the Rust SDK (not yet implemented).
…odecs - SetRates: clamp minimum bitrate to 1.5 Mbps to prevent BWE from throttling NVENC to unusable levels (27kbps for 1080p screenshare) - SetRates: actually call Reconfigure() so NVENC adapts to new bitrate (previously the encoder was stuck at initial startBitrate forever) - InitEncode: add maxQP=28 cap to prevent quality collapse at low bitrate - GetEncoderInfo: set has_trusted_rate_controller=true to disable libwebrtc's frame dropper which was dropping frames behind NVENC's back - SDP: extend x-google-start-bitrate munging to H264/H265 (was VP9/AV1 only), so BWE starts at ~5.6 Mbps instead of default 300kbps
Remove bitrate floor and has_trusted_rate_controller — these caused a death spiral where NVENC produced 1.5Mbps but BWE only allowed ~100kbps through the pacer, causing massive jitter which made BWE throttle even harder. Now the encoder follows BWE's rate via Reconfigure (can ramp up as BWE probes higher), and maxQP=28 prevents quality collapse at low bitrates during the initial probing phase.
maxQP=28 prevented NVENC from compressing below ~2Mbps at 1920x808, but BWE only allows ~800kbps during initial probing. The encoder couldn't reach the target rate, causing pacer buffering, growing jitter (150→519ms), and frame drops (74 frames in 45s). Without the cap, NVENC can use whatever QP is needed to match BWE's rate. Quality starts lower but improves as BWE probes upward.
Add min_bitrate field to RtpEncodingParameters and wire it to the native webrtc-sys layer. Set 2 Mbps min for screenshare tracks so the BitrateAllocator won't let BWE throttle below usable levels. This fixes the localhost screenshare lag where BWE's delay-based estimator incorrectly detects congestion and throttles to <100kbps.
No changeset foundThis PR modifies the following packages but doesn't include a changeset: Directly changed:
Downstream dependencies (also need a version bump):
Click here to create a changeset The link pre-populates a changeset file with If this change doesn't require a version bump, add the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.