HMD Optical Hand tracking and MANUS automatic installation#235
HMD Optical Hand tracking and MANUS automatic installation#235timvw-manusvr wants to merge 3 commits intoNVIDIA:mainfrom
Conversation
Added installation bash script HMD hand tracking is now used when available to position the hands
📝 WalkthroughWalkthroughThe Manus plugin now integrates native OpenXR XDev hand tracking with graceful controller fallback, adds automated installation scripts for MANUS SDK and gRPC dependencies, expands documentation with prerequisites and troubleshooting guidance, and standardizes runtime logging with Manus prefixes. Changes
Sequence Diagram(s)sequenceDiagram
participant OpenXR as OpenXR Session
participant XDev as XDev Hand Tracking
participant Controller as Controller Input
participant Plugin as Manus Plugin
participant Injector as Hand Injector
Plugin->>OpenXR: Initialize XDev space & extensions
activate OpenXR
OpenXR-->>Plugin: Extensions loaded
deactivate OpenXR
Plugin->>XDev: Enumerate & create hand trackers
activate XDev
alt XDev Available
XDev-->>Plugin: Native hand tracker created
Plugin->>XDev: update_xdev_hand(time)
XDev-->>Plugin: Wrist pose + joint data
else XDev Unavailable
XDev-->>Plugin: Fallback to controllers
end
deactivate XDev
alt Hand Tracking Available
Plugin->>Injector: Inject XDev hand data
else Hand Tracking Unavailable
Plugin->>Controller: get_controller_wrist_pose()
Controller-->>Plugin: Synthesized wrist pose
Plugin->>Injector: Inject controller-based data
end
activate Injector
Injector-->>Plugin: Skeleton injection complete
deactivate Injector
Plugin->>XDev: cleanup_xdev_hand_trackers()
activate XDev
XDev-->>Plugin: Resources released
deactivate XDev
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes The changes span multiple heterogeneous areas: substantial refactoring of hand tracking initialization and data injection logic with new XDev-based pathways; two new installation scripts with multi-step orchestration, architecture-specific logic, and dependency patching; and expanded documentation. The core plugin changes introduce state management for dual input sources, multiple new private methods, and conditional fallback paths that require careful verification of error handling and OpenXR API correctness. Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Comment |
There was a problem hiding this comment.
Pull request overview
This PR adds support for using the HMD's built-in optical hand tracking (via the XR_MNDX_xdev_space extension) as the wrist position source when using Manus gloves, alongside two new automated installation scripts and expanded documentation.
Changes:
- Adds
initialize_xdev_hand_trackers()/cleanup_xdev_hand_trackers()/update_xdev_hand()/get_controller_wrist_pose()to the plugin, with automatic fallback from HMD optical tracking to controller-based wrist positioning. - Introduces two new shell scripts (
install_manus.sh,install-dependencies.sh) for end-to-end automated installation including SDK download, dependency setup (with optional gRPC), and plugin build. - Updates
README.mdwith automated installation instructions, CloudXR setup steps, a new section on the two wrist-tracking modes, and an expanded troubleshooting guide.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
src/plugins/manus/core/manus_hand_tracking_plugin.cpp |
Adds XDev/hand tracking initialization, wrist-source auto-selection, and simplifies injector lifecycle |
src/plugins/manus/inc/core/manus_hand_tracking_plugin.hpp |
Adds new private methods, XDev handle/function-pointer members, and HandTracker member |
src/plugins/manus/tools/manus_hand_tracker_printer.cpp |
Adds [Manus] prefix to all console output lines |
src/plugins/manus/install_manus.sh |
New all-in-one installation and build script |
src/plugins/manus/install-dependencies.sh |
New script for building gRPC v1.28.1 and system dependencies |
src/plugins/manus/README.md |
Expanded with automated install, CloudXR setup, tracking modes, and troubleshooting |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| std::string serial_str = properties.serial ? properties.serial : ""; | ||
| if (serial_str == "Head Device (0)") | ||
| { | ||
| left_xdev_id = xdev_id; | ||
| } | ||
| else if (serial_str == "Head Device (1)") | ||
| { | ||
| right_xdev_id = xdev_id; | ||
| } | ||
| } |
There was a problem hiding this comment.
The code uses hardcoded serial strings "Head Device (0)" and "Head Device (1)" to identify the left and right optical hand-tracking XDevs. This string is not part of any published XR_MNDX_xdev_space specification and is specific to a particular runtime implementation. If the runtime changes these display names (e.g., across firmware updates), both trackers would fail silently to be found (both left_xdev_id and right_xdev_id remain 0), and the system would silently fall back to controllers with only a generic error message. At a minimum, a comment explaining that these strings are runtime-specific and the source of this assumption should be added; ideally the matching should be made more robust or the fallback more informative.
|
|
||
| // Add XDev extension for HMD hand tracking | ||
| extensions.push_back(XR_MNDX_XDEV_SPACE_EXTENSION_NAME); | ||
|
|
There was a problem hiding this comment.
The XR_MNDX_XDEV_SPACE_EXTENSION_NAME is pushed into the extensions vector and passed to OpenXRSession as a required extension (it is listed directly in enabledExtensionNames in xrCreateInstance). If the runtime does not support this extension, xrCreateInstance will fail with XR_ERROR_EXTENSION_NOT_PRESENT, causing the plugin to crash entirely rather than falling back to controllers.
The intent described in the PR is that this extension is optional with a graceful fallback. To achieve this, the extension should be queried first via xrEnumerateInstanceExtensionProperties, and only added to the list if actually supported by the runtime. If not supported, the code should skip initialize_xdev_hand_trackers() and proceed with the controller fallback.
| if (XR_FAILED(result) || xdev_count == 0) | ||
| { | ||
| std::cerr << "[Manus] No XDevs found, falling back to controllers" << std::endl; | ||
| return; | ||
| } | ||
|
|
||
| std::vector<XrXDevIdMNDX> xdev_ids(xdev_count); | ||
| result = m_pfn_enumerate_xdevs(m_xdev_list, xdev_count, &xdev_count, xdev_ids.data()); | ||
| if (XR_FAILED(result)) | ||
| { | ||
| return; | ||
| } |
There was a problem hiding this comment.
When xdev_count == 0 (the "No XDevs found" early return) or when the second m_pfn_enumerate_xdevs call fails, the function returns without destroying m_xdev_list, leaking the handle created on line 381. Both early-return paths (lines 391–394 and 399–401) should call cleanup_xdev_hand_trackers() instead of a bare return, consistent with how errors are handled later in the same function (line 460).
| make install | ||
| ldconfig |
There was a problem hiding this comment.
The gRPC build step calls make install directly without a preceding parallel make -j$(nproc). While make install will trigger the build implicitly, it will do so using only a single thread, making the gRPC compilation (which is already a large codebase) significantly slower than necessary. A make -j$(nproc) step should be added before make install, consistent with the protobuf build steps on lines 82–83.
| if [ "$ARCH" = "x86_64" ]; then | ||
| SDK_CLIENT_DIR="SDKClient_Linux" | ||
| elif [ "$ARCH" = "aarch64" ]; then | ||
| SDK_CLIENT_DIR="SDKClient_Linux" | ||
| else | ||
| echo "Warning: Unsupported architecture $ARCH. Attempting to use SDKClient_Linux..." | ||
| SDK_CLIENT_DIR="SDKClient_Linux" | ||
| fi |
There was a problem hiding this comment.
The architecture detection block for SDK_CLIENT_DIR is redundant dead code: all three branches (x86_64, aarch64, and the unsupported fallback) assign the same value "SDKClient_Linux". If the SDK zip contains architecture-specific subdirectories (e.g., SDKClient_Linux_aarch64), the logic should handle them differently; otherwise the entire if-elif-else block can be replaced with a single SDK_CLIENT_DIR="SDKClient_Linux".
| #!/bin/bash | ||
| set -e # Exit on error | ||
| set -u # Exit on undefined variable |
There was a problem hiding this comment.
The install-dependencies.sh script is missing the SPDX license header that is present in install_manus.sh (lines 2–3). All other scripts in this repository include the SPDX-FileCopyrightText and SPDX-License-Identifier header lines immediately after the shebang. The header should be added to install-dependencies.sh as well.
| // Extract glove IDs from landscape data | ||
| for (uint32_t i = 0; i < gloves.gloveCount; i++) | ||
| { | ||
| const GloveLandscapeData& glove = gloves.gloves[i]; | ||
| if (glove.side == Side::Side_Left) | ||
| { | ||
| tracker.left_glove_id = glove.id; | ||
| left_present = true; | ||
| } | ||
| else if (glove.side == Side::Side_Right) | ||
| { | ||
| tracker.right_glove_id = glove.id; | ||
| right_present = true; | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
The OnLandscapeStream callback previously cleared stale glove IDs and skeleton node data when a glove disappeared from the landscape (i.e., disconnected). That cleanup logic was removed in this PR. Now, if a glove disconnects, left_glove_id/right_glove_id are never reset and m_left_hand_nodes/m_right_hand_nodes are never cleared, so inject_hand_data() will keep pushing the last known stale pose indefinitely after disconnect. The nodes.empty() guard in process_hand will never trigger after the first successful data reception. The disconnect-detection logic (checking which sides are absent from the landscape update) and the corresponding cache-clearing should be restored.
| if [ ! -f "/etc/ld.so.conf.d/aarch64-local.conf" ]; then | ||
| echo "/usr/local/lib" | tee /etc/ld.so.conf.d/aarch64-local.conf > /dev/null | ||
| ldconfig | ||
| echo "Added /usr/local/lib to ld.so.conf" |
There was a problem hiding this comment.
The ld.so.conf.d configuration file is hardcoded as aarch64-local.conf (line 120), but this script runs on any architecture (x86_64 and aarch64). On an x86_64 system, the file will be created with an architecturally misleading name. The filename should be architecture-agnostic (e.g., local-libs.conf) or dynamically derived from $(uname -m) to avoid confusion.
| - Use the MANUS Quest 3 controller adapters to attach the Quest 3 controllers to the MANUS Universal Mount on the back of the glove. | ||
| - Use the HMD's optical hand tracking to position the hands. | ||
|
|
||
| The system will switch dynamically based on the available tracking source. When using controllers it's advised to turn of hand tracking entirely or turn off automatic switching. |
There was a problem hiding this comment.
The sentence "When using controllers it's advised to turn of hand tracking entirely or turn off automatic switching." contains a typo: "turn of" should be "turn off".
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 99-100: The printed diagnostic incorrectly says "Z-up" while the
coordinate system is actually configured with up = AxisPolarity_PositiveY;
update the log string printed before calling
CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true) (the std::cout line) to
reflect "Y-up, right-handed, meters" so the diagnostic matches the SDK
configuration (referencing AxisPolarity_PositiveY and the call to
CoreSdk_InitializeCoordinateSystemWithVUH).
- Around line 507-514: The code currently treats a wrist pose as "valid" in
update_xdev_hand by checking XR_SPACE_LOCATION_POSITION_VALID_BIT and
XR_SPACE_LOCATION_ORIENTATION_VALID_BIT but does not ensure the pose is actively
tracked; change the logic in update_xdev_hand where wrist is read (using
joint_locations[XR_HAND_JOINT_WRIST_EXT] and wrist.locationFlags) to also check
XR_SPACE_LOCATION_POSITION_TRACKED_BIT and
XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT before returning success/setting
out_wrist_pose, or alternatively return/propagate both a valid and a tracked
boolean so inject_hand_data (which uses is_root_tracked) can add tracked bits
only when the wrist is actually tracked; update references to is_root_tracked
accordingly so last-known-but-untracked poses are not advertised as fully
tracked.
- Around line 116-129: The code currently unconditionally adds
XR_EXT_HAND_TRACKING_EXTENSION_NAME (via including core::HandTracker in
trackers) and XR_MNDX_XDEV_SPACE_EXTENSION_NAME to extensions before
constructing the OpenXR session, which causes xrCreateInstance() to fail if the
runtime lacks those optional extensions; modify the flow to query the runtime's
supported extensions (or probe via core::OpenXRSession/DeviceIOSession helper)
before pushing optional names: only include XR_EXT_HAND_TRACKING_EXTENSION_NAME
if the runtime reports it supported (or defer adding core::HandTracker to
trackers until after support is confirmed), and only push
XR_MNDX_XDEV_SPACE_EXTENSION_NAME if supported; construct m_session =
std::make_shared<core::OpenXRSession>(app_name, extensions) after gating
optional extensions and then call initialize_xdev_hand_trackers()/create hand
trackers as needed so controller fallback remains possible.
In `@src/plugins/manus/install_manus.sh`:
- Around line 53-64: The apt install block in install_manus.sh currently omits a
downloader and unzip and doesn't validate downloads; update the apt-get install
list (the sudo apt-get install -y ... block) to include a downloader (curl or
wget) and unzip, change the SDK fetch command (the curl/wget invocation in the
SDK download/extract step around lines referenced) to use a failing download
mode (curl -fL or wget --server-response -O) so HTTP errors fail, save to a
deterministic filename, then compute and verify the archive checksum (e.g.
sha256sum) against a known expected value before extracting, and only run
unzip/tar after the checksum matches; if verification fails, abort with a clear
error.
- Around line 181-187: The script currently builds only the manus_hand_plugin
target but README expects the manus_hand_tracker_printer binary; update the
build/install steps to include the printer binary: adjust the cmake --build
invocation (or add a second cmake --build call) to also build the
manus_hand_tracker_printer target and ensure the corresponding binary/component
is installed (e.g., include its name alongside manus or call cmake --install for
that component). Locate the build/install lines referencing targets
manus_hand_plugin and component manus and add the manus_hand_tracker_printer
target/component so the documented ./build/bin/manus_hand_tracker_printer exists
after the script runs.
- Around line 173-179: The script currently skips CMake configuration when
build/CMakeCache.txt exists, which leaves stale discovery results after copying
the ManusSDK; change the logic so CMake is forced to reconfigure after the SDK
copy by removing or invalidating build/CMakeCache.txt (or always running the
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release command) instead of conditionally
skipping it; update the block that checks "build/CMakeCache.txt" and ensure the
cmake invocation is executed unconditionally (or after deleting the cache) so
the project is deterministically reconfigured.
In `@src/plugins/manus/install-dependencies.sh`:
- Around line 14-22: The script currently writes "deb
http://deb.debian.org/debian bookworm main" directly into
/etc/apt/sources.list.d/bookworm-gcc.list which can allow unrelated Bookworm
packages to be installed; instead, add an apt pin/priority file that pins only
the specific packages you need (e.g., gcc-11, g++-11, libgcc-11) with a high
priority for the target release, or use an isolated toolchain source (PPA or apt
repository that provides only GCC packages). Update the Debian branch of the
install step (the block that runs apt-get install -y debian-archive-keyring and
echoes the repo) to also create a /etc/apt/preferences.d/bookworm-gcc.pref entry
that pins the listed package names to release "n=bookworm" with a high
Pin-Priority, and ensure apt-get update is called before installing the pinned
packages so only those specific packages are allowed from Bookworm rather than
the whole main archive.
- Around line 38-44: The existing check that skips cloning when the grpc repo
directory exists can leave a stale or dirty checkout; change the else branch to
normalize the existing checkout: fetch all tags and branches, checkout the
pinned tag v1.28.1, hard-reset the worktree to that tag, remove untracked files,
and ensure submodules are synchronized and updated/initialized recursively so
the repo is deterministic before continuing with build steps that expect
v1.28.1.
In `@src/plugins/manus/README.md`:
- Around line 44-46: Update the MANUS version references in the README: change
both URLs and any text that point to 3.1.0 to 3.1.1 (specifically replace
https://docs.manus-meta.com/3.1.0/Resources/ and
https://docs.manus-meta.com/3.1.0/Plugins/SDK/Linux/ with the 3.1.1 equivalents)
so the manual-install instructions match the automated installer; ensure the
mention of the ManusSDK folder and MANUS_SDK_ROOT remains the same but reflects
the 3.1.1 archive name where applicable.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c7bcbfd3-5e8e-4006-9d8b-2a78c775fabb
📒 Files selected for processing (6)
src/plugins/manus/README.mdsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/plugins/manus/inc/core/manus_hand_tracking_plugin.hppsrc/plugins/manus/install-dependencies.shsrc/plugins/manus/install_manus.shsrc/plugins/manus/tools/manus_hand_tracker_printer.cpp
| std::cout << "[Manus] Setting up coordinate system (Z-up, right-handed, meters)..." << std::endl; | ||
| const SDKReturnCode t_CoordinateResult = CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true); |
There was a problem hiding this comment.
Fix the coordinate-system log text.
Line 95 sets up = AxisPolarity_PositiveY, so this diagnostic should say Y-up, not Z-up. Otherwise the log contradicts the actual SDK configuration.
📝 Keep the log aligned with the configured axes
- std::cout << "[Manus] Setting up coordinate system (Z-up, right-handed, meters)..." << std::endl;
+ std::cout << "[Manus] Setting up coordinate system (Y-up, right-handed, meters)..." << std::endl;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| std::cout << "[Manus] Setting up coordinate system (Z-up, right-handed, meters)..." << std::endl; | |
| const SDKReturnCode t_CoordinateResult = CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true); | |
| std::cout << "[Manus] Setting up coordinate system (Y-up, right-handed, meters)..." << std::endl; | |
| const SDKReturnCode t_CoordinateResult = CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 99 - 100,
The printed diagnostic incorrectly says "Z-up" while the coordinate system is
actually configured with up = AxisPolarity_PositiveY; update the log string
printed before calling CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true)
(the std::cout line) to reflect "Y-up, right-handed, meters" so the diagnostic
matches the SDK configuration (referencing AxisPolarity_PositiveY and the call
to CoreSdk_InitializeCoordinateSystemWithVUH).
| // Create ControllerTracker, HandTracker and DeviceIOSession | ||
| m_controller_tracker = std::make_shared<core::ControllerTracker>(); | ||
| std::vector<std::shared_ptr<core::ITracker>> trackers = { m_controller_tracker }; | ||
| m_hand_tracker = std::make_shared<core::HandTracker>(); | ||
| std::vector<std::shared_ptr<core::ITracker>> trackers = { m_controller_tracker, m_hand_tracker }; | ||
|
|
||
| // Get required extensions from trackers | ||
| auto extensions = core::DeviceIOSession::get_required_extensions(trackers); | ||
| extensions.push_back(XR_NVX1_DEVICE_INTERFACE_BASE_EXTENSION_NAME); | ||
|
|
||
| // Add XDev extension for HMD hand tracking | ||
| extensions.push_back(XR_MNDX_XDEV_SPACE_EXTENSION_NAME); | ||
|
|
||
| // Create session with required extensions - constructor automatically begins the session | ||
| m_session = std::make_shared<core::OpenXRSession>(app_name, extensions); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
grep -n "get_required_extensions\|initialize_xdev\|controller fallback" src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 249
🏁 Script executed:
find . -type f \( -name "*.cpp" -o -name "*.h" \) -path "*/core/*" | xargs grep -l "class.*HandTracker\|class.*DeviceIOSession\|class.*OpenXRSession" 2>/dev/null | head -10Repository: NVIDIA/IsaacTeleop
Length of output: 149
🏁 Script executed:
find . -type f \( -name "*.cpp" -o -name "*.h" \) | xargs grep -l "get_required_extensions" 2>/dev/nullRepository: NVIDIA/IsaacTeleop
Length of output: 930
🏁 Script executed:
cat -n src/plugins/manus/core/manus_hand_tracking_plugin.cpp | sed -n '110,160p'Repository: NVIDIA/IsaacTeleop
Length of output: 2555
🏁 Script executed:
grep -n "class HandTracker\|required_extensions" src/core/deviceio/cpp/hand_tracker.cpp | head -20Repository: NVIDIA/IsaacTeleop
Length of output: 135
🏁 Script executed:
cat -n src/core/deviceio/cpp/deviceio_session.cpp | grep -A 20 "get_required_extensions"Repository: NVIDIA/IsaacTeleop
Length of output: 1714
🏁 Script executed:
find . -type f \( -name "*.cpp" -o -name "*.h" \) | xargs grep -l "class OpenXRSession" 2>/dev/nullRepository: NVIDIA/IsaacTeleop
Length of output: 46
🏁 Script executed:
find . -type f \( -name "*.cpp" -o -name "*.h" \) | xargs grep -l "OpenXRSession" 2>/dev/null | head -10Repository: NVIDIA/IsaacTeleop
Length of output: 523
🏁 Script executed:
cat -n src/core/deviceio/cpp/hand_tracker.cpp | sed -n '210,230p'Repository: NVIDIA/IsaacTeleop
Length of output: 1007
🏁 Script executed:
cat -n src/plugins/manus/core/manus_hand_tracking_plugin.cpp | sed -n '352,380p'Repository: NVIDIA/IsaacTeleop
Length of output: 1871
🏁 Script executed:
find . -type f \( -name "*.cpp" -o -name "*.h" \) | xargs grep -l "class OpenXRSession" 2>/dev/nullRepository: NVIDIA/IsaacTeleop
Length of output: 46
🏁 Script executed:
find . -path "*/oxr/*" -type f \( -name "*.cpp" -o -name "*.h" \) | grep -i session | head -10Repository: NVIDIA/IsaacTeleop
Length of output: 139
🏁 Script executed:
cat -n src/core/oxr/cpp/oxr_session.cpp | head -100Repository: NVIDIA/IsaacTeleop
Length of output: 4126
🏁 Script executed:
grep -n "OpenXRSession::" src/core/oxr/cpp/oxr_session.cpp | head -20Repository: NVIDIA/IsaacTeleop
Length of output: 499
🏁 Script executed:
wc -l src/core/oxr/cpp/oxr_session.cppRepository: NVIDIA/IsaacTeleop
Length of output: 98
🏁 Script executed:
cat -n src/core/oxr/cpp/oxr_session.cpp | sed -n '91,145p'Repository: NVIDIA/IsaacTeleop
Length of output: 2530
🏁 Script executed:
cat -n src/core/oxr/cpp/oxr_session.cpp | sed -n '100,135p'Repository: NVIDIA/IsaacTeleop
Length of output: 1592
🏁 Script executed:
cat -n src/plugins/manus/core/manus_hand_tracking_plugin.cpp | sed -n '140,175p'Repository: NVIDIA/IsaacTeleop
Length of output: 1221
🏁 Script executed:
grep -n "catch\|try" src/plugins/manus/core/manus_hand_tracking_plugin.cpp | head -20Repository: NVIDIA/IsaacTeleop
Length of output: 504
🏁 Script executed:
grep -B5 -A15 "Failed to initialize OpenXR" src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 473
🏁 Script executed:
grep -n "XR_ERROR_EXTENSION_NOT_PRESENT\|XR_FAILED" src/core/oxr/cpp/oxr_session.cppRepository: NVIDIA/IsaacTeleop
Length of output: 304
🏁 Script executed:
cat -n src/plugins/manus/core/manus_hand_tracking_plugin.cpp | sed -n '1,50p' | grep -i "include\|extension"Repository: NVIDIA/IsaacTeleop
Length of output: 569
Gate optional OpenXR extensions before constructing the session.
Adding core::HandTracker to trackers makes XR_EXT_HAND_TRACKING_EXTENSION_NAME part of the required extension set, and line 126 unconditionally adds XR_MNDX_XDEV_SPACE_EXTENSION_NAME. If the runtime lacks either extension, xrCreateInstance() fails at line 119 of oxr_session.cpp, throwing an exception before initialize_xdev_hand_trackers() runs, preventing the controller fallback from executing.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 116 -
129, The code currently unconditionally adds XR_EXT_HAND_TRACKING_EXTENSION_NAME
(via including core::HandTracker in trackers) and
XR_MNDX_XDEV_SPACE_EXTENSION_NAME to extensions before constructing the OpenXR
session, which causes xrCreateInstance() to fail if the runtime lacks those
optional extensions; modify the flow to query the runtime's supported extensions
(or probe via core::OpenXRSession/DeviceIOSession helper) before pushing
optional names: only include XR_EXT_HAND_TRACKING_EXTENSION_NAME if the runtime
reports it supported (or defer adding core::HandTracker to trackers until after
support is confirmed), and only push XR_MNDX_XDEV_SPACE_EXTENSION_NAME if
supported; construct m_session = std::make_shared<core::OpenXRSession>(app_name,
extensions) after gating optional extensions and then call
initialize_xdev_hand_trackers()/create hand trackers as needed so controller
fallback remains possible.
| const auto& wrist = joint_locations[XR_HAND_JOINT_WRIST_EXT]; | ||
| bool is_valid = (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) && | ||
| (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT); | ||
|
|
||
| if (is_valid) | ||
| { | ||
| out_wrist_pose = wrist.pose; | ||
| return true; |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's confirm the file exists and look at the code around lines 507-514
wc -l src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 119
🏁 Script executed:
# Read the specific section mentioned in the review
sed -n '500,530p' src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 969
🏁 Script executed:
# Let's also search for the inject_hand_data function to understand how the return value is used
rg -n "inject_hand_data" src/plugins/manus/core/manus_hand_tracking_plugin.cpp -A 20Repository: NVIDIA/IsaacTeleop
Length of output: 1243
🏁 Script executed:
# Search for the helper function containing lines 507-514
rg -n "XR_HAND_JOINT_WRIST_EXT" src/plugins/manus/core/manus_hand_tracking_plugin.cpp -B 10 -A 15Repository: NVIDIA/IsaacTeleop
Length of output: 2033
🏁 Script executed:
# Find the function name that contains lines 507-514
sed -n '480,520p' src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 1343
🏁 Script executed:
# Get more context from inject_hand_data() to see how it processes return values
sed -n '543,650p' src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 3791
🏁 Script executed:
# Let's see the complete joint assignment to understand the full context
sed -n '630,645p' src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 834
🏁 Script executed:
# Search for XR_SPACE_LOCATION flag definitions to understand the semantics
rg -i "XR_SPACE_LOCATION_POSITION_TRACKED_BIT|XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT" src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 169
🏁 Script executed:
# Check how is_root_tracked is used and set
rg -n "is_root_tracked" src/plugins/manus/core/manus_hand_tracking_plugin.cpp -B 2 -A 2Repository: NVIDIA/IsaacTeleop
Length of output: 1092
🏁 Script executed:
# Let's verify the logic flow more completely - see what happens after is_root_tracked is set
sed -n '645,660p' src/plugins/manus/core/manus_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 594
🏁 Script executed:
# Check if there are any comments or documentation about VALID vs TRACKED semantics in the codebase
rg -i "tracked|valid" src/plugins/manus/core/manus_hand_tracking_plugin.cpp | head -30Repository: NVIDIA/IsaacTeleop
Length of output: 1103
🏁 Script executed:
# Let's also check what the OpenXR spec says about these flags by looking for any documentation
find . -name "*.h" -o -name "*.hpp" | xargs grep -l "XR_SPACE_LOCATION" 2>/dev/null | head -5Repository: NVIDIA/IsaacTeleop
Length of output: 44
🏁 Script executed:
# Let's look for XR header files or any OpenXR documentation references
find . -type f \( -name "*.h" -o -name "*.hpp" -o -name "*.txt" -o -name "*.md" \) | xargs grep -l "TRACKED_BIT\|VALID_BIT" 2>/dev/nullRepository: NVIDIA/IsaacTeleop
Length of output: 46
🏁 Script executed:
# Let's check the header includes to see what OpenXR version or definitions are being used
sed -n '1,50p' src/plugins/manus/core/manus_hand_tracking_plugin.cpp | grep -i "include\|xr"Repository: NVIDIA/IsaacTeleop
Length of output: 683
Check if wrist pose is currently tracked, not just valid.
The update_xdev_hand() function returns true when the wrist pose is valid but does not verify that it is actively tracked. The return value is used in inject_hand_data() (line 583) to set is_root_tracked, which controls whether XR_SPACE_LOCATION_POSITION_TRACKED_BIT and XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT are added to all injected hand joints (lines 647-650). A last-known wrist pose that is valid but not currently tracked will therefore be advertised as fully tracked hand data.
🔧 Preserve tracked-vs-valid semantics
- bool is_valid = (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) &&
- (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT);
-
- if (is_valid)
+ bool is_tracked = (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) &&
+ (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) &&
+ (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_TRACKED_BIT) &&
+ (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT);
+
+ if (is_tracked)
{
out_wrist_pose = wrist.pose;
return true;
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const auto& wrist = joint_locations[XR_HAND_JOINT_WRIST_EXT]; | |
| bool is_valid = (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) && | |
| (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT); | |
| if (is_valid) | |
| { | |
| out_wrist_pose = wrist.pose; | |
| return true; | |
| const auto& wrist = joint_locations[XR_HAND_JOINT_WRIST_EXT]; | |
| bool is_tracked = (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_VALID_BIT) && | |
| (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_VALID_BIT) && | |
| (wrist.locationFlags & XR_SPACE_LOCATION_POSITION_TRACKED_BIT) && | |
| (wrist.locationFlags & XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT); | |
| if (is_tracked) | |
| { | |
| out_wrist_pose = wrist.pose; | |
| return true; |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 507 -
514, The code currently treats a wrist pose as "valid" in update_xdev_hand by
checking XR_SPACE_LOCATION_POSITION_VALID_BIT and
XR_SPACE_LOCATION_ORIENTATION_VALID_BIT but does not ensure the pose is actively
tracked; change the logic in update_xdev_hand where wrist is read (using
joint_locations[XR_HAND_JOINT_WRIST_EXT] and wrist.locationFlags) to also check
XR_SPACE_LOCATION_POSITION_TRACKED_BIT and
XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT before returning success/setting
out_wrist_pose, or alternatively return/propagate both a valid and a tracked
boolean so inject_hand_data (which uses is_root_tracked) can add tracked bits
only when the wrist is actually tracked; update references to is_root_tracked
accordingly so last-known-but-untracked poses are not advertised as fully
tracked.
| sudo apt-get update | ||
| sudo apt-get install -y \ | ||
| build-essential \ | ||
| cmake \ | ||
| git \ | ||
| libssl-dev \ | ||
| zlib1g-dev \ | ||
| libc-ares-dev \ | ||
| libzmq3-dev \ | ||
| libncurses-dev \ | ||
| libudev-dev \ | ||
| libusb-1.0-0-dev |
There was a problem hiding this comment.
Harden the SDK download/extract step.
The minimal dependency branch does not install a downloader or unzip, even though the next phases require them, and curl -L will happily save an HTTP error page as the ZIP. On a clean machine the recommended installer can fail late or unpack a bad payload. Please install one download client plus unzip, use a failing download mode, and verify the SDK archive checksum before extracting it.
🔧 Minimal hardening for the current flow
sudo apt-get install -y \
build-essential \
cmake \
+ curl \
git \
libssl-dev \
zlib1g-dev \
libc-ares-dev \
libzmq3-dev \
libncurses-dev \
libudev-dev \
- libusb-1.0-0-dev
+ libusb-1.0-0-dev \
+ unzip
...
if command -v wget &> /dev/null; then
wget "$MANUS_SDK_URL" -O "$MANUS_SDK_ZIP"
elif command -v curl &> /dev/null; then
- curl -L "$MANUS_SDK_URL" -o "$MANUS_SDK_ZIP"
+ curl -fL "$MANUS_SDK_URL" -o "$MANUS_SDK_ZIP"Also applies to: 92-118
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install_manus.sh` around lines 53 - 64, The apt install
block in install_manus.sh currently omits a downloader and unzip and doesn't
validate downloads; update the apt-get install list (the sudo apt-get install -y
... block) to include a downloader (curl or wget) and unzip, change the SDK
fetch command (the curl/wget invocation in the SDK download/extract step around
lines referenced) to use a failing download mode (curl -fL or wget
--server-response -O) so HTTP errors fail, save to a deterministic filename,
then compute and verify the archive checksum (e.g. sha256sum) against a known
expected value before extracting, and only run unzip/tar after the checksum
matches; if verification fails, abort with a clear error.
| # Configure with CMake (only if needed) | ||
| if [ ! -f "build/CMakeCache.txt" ]; then | ||
| echo "Configuring CMake..." | ||
| cmake -S . -B build -DCMAKE_BUILD_TYPE=Release | ||
| else | ||
| echo "CMake already configured, skipping configuration..." | ||
| fi |
There was a problem hiding this comment.
Force a CMake reconfigure after copying the SDK.
ManusSDK is copied into the tree immediately before this block, but an existing build/CMakeCache.txt short-circuits configuration. That leaves stale discovery results in place if the user already had a build directory from a previous run or before the SDK existed.
🔧 Reconfigure deterministically
-# Configure with CMake (only if needed)
-if [ ! -f "build/CMakeCache.txt" ]; then
- echo "Configuring CMake..."
- cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
-else
- echo "CMake already configured, skipping configuration..."
-fi
+# Configure with CMake
+echo "Configuring CMake..."
+cmake -S . -B build -DCMAKE_BUILD_TYPE=Release📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Configure with CMake (only if needed) | |
| if [ ! -f "build/CMakeCache.txt" ]; then | |
| echo "Configuring CMake..." | |
| cmake -S . -B build -DCMAKE_BUILD_TYPE=Release | |
| else | |
| echo "CMake already configured, skipping configuration..." | |
| fi | |
| # Configure with CMake | |
| echo "Configuring CMake..." | |
| cmake -S . -B build -DCMAKE_BUILD_TYPE=Release |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install_manus.sh` around lines 173 - 179, The script
currently skips CMake configuration when build/CMakeCache.txt exists, which
leaves stale discovery results after copying the ManusSDK; change the logic so
CMake is forced to reconfigure after the SDK copy by removing or invalidating
build/CMakeCache.txt (or always running the cmake -S . -B build
-DCMAKE_BUILD_TYPE=Release command) instead of conditionally skipping it; update
the block that checks "build/CMakeCache.txt" and ensure the cmake invocation is
executed unconditionally (or after deleting the cache) so the project is
deterministically reconfigured.
| # Build only the manus plugin (and its dependencies) | ||
| echo "Building..." | ||
| cmake --build build --target manus_hand_plugin -j$(nproc) | ||
|
|
||
| # Install only the manus component | ||
| echo "Installing..." | ||
| cmake --install build --component manus |
There was a problem hiding this comment.
Build the printer tool in the recommended flow.
src/plugins/manus/README.md later tells users to run ./build/bin/manus_hand_tracker_printer, but this script only builds manus_hand_plugin. Unless that binary was built earlier, the documented verification step fails immediately.
🔧 Build both documented binaries
# Build only the manus plugin (and its dependencies)
echo "Building..."
-cmake --build build --target manus_hand_plugin -j$(nproc)
+cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j"$(nproc)"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Build only the manus plugin (and its dependencies) | |
| echo "Building..." | |
| cmake --build build --target manus_hand_plugin -j$(nproc) | |
| # Install only the manus component | |
| echo "Installing..." | |
| cmake --install build --component manus | |
| # Build only the manus plugin (and its dependencies) | |
| echo "Building..." | |
| cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j"$(nproc)" | |
| # Install only the manus component | |
| echo "Installing..." | |
| cmake --install build --component manus |
🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 183-183: Quote this to prevent word splitting.
(SC2046)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install_manus.sh` around lines 181 - 187, The script
currently builds only the manus_hand_plugin target but README expects the
manus_hand_tracker_printer binary; update the build/install steps to include the
printer binary: adjust the cmake --build invocation (or add a second cmake
--build call) to also build the manus_hand_tracker_printer target and ensure the
corresponding binary/component is installed (e.g., include its name alongside
manus or call cmake --install for that component). Locate the build/install
lines referencing targets manus_hand_plugin and component manus and add the
manus_hand_tracker_printer target/component so the documented
./build/bin/manus_hand_tracker_printer exists after the script runs.
| if grep -qi "ubuntu" /etc/os-release; then | ||
| echo "Adding Ubuntu Toolchain PPA for GCC-11..." | ||
| apt-get install -y software-properties-common | ||
| add-apt-repository -y ppa:ubuntu-toolchain-r/test | ||
| else | ||
| echo "Adding Debian 12 (Bookworm) repository for GCC-11..." | ||
| apt-get install -y debian-archive-keyring | ||
| echo "deb http://deb.debian.org/debian bookworm main" | tee /etc/apt/sources.list.d/bookworm-gcc.list | ||
| fi |
There was a problem hiding this comment.
Don't add Bookworm main without pinning it.
Writing bookworm main straight into /etc/apt/sources.list.d can let future apt operations pull unrelated packages from a foreign release. That's a risky host-wide side effect for an installer. If you need GCC-11 from Bookworm, please add an apt preference that pins only the required compiler packages or use an isolated toolchain source instead.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install-dependencies.sh` around lines 14 - 22, The script
currently writes "deb http://deb.debian.org/debian bookworm main" directly into
/etc/apt/sources.list.d/bookworm-gcc.list which can allow unrelated Bookworm
packages to be installed; instead, add an apt pin/priority file that pins only
the specific packages you need (e.g., gcc-11, g++-11, libgcc-11) with a high
priority for the target release, or use an isolated toolchain source (PPA or apt
repository that provides only GCC packages). Update the Debian branch of the
install step (the block that runs apt-get install -y debian-archive-keyring and
echoes the repo) to also create a /etc/apt/preferences.d/bookworm-gcc.pref entry
that pins the listed package names to release "n=bookworm" with a high
Pin-Priority, and ensure apt-get update is called before installing the pinned
packages so only those specific packages are allowed from Bookworm rather than
the whole main archive.
| echo "[2/7] Cloning gRPC repository..." | ||
| mkdir -p /var/local/git | ||
| if [ ! -d "/var/local/git/grpc" ]; then | ||
| git clone --recurse-submodules -b v1.28.1 --depth 1 https://github.com/grpc/grpc /var/local/git/grpc | ||
| else | ||
| echo "gRPC repository already exists, skipping clone" | ||
| fi |
There was a problem hiding this comment.
Refresh existing gRPC checkouts to the pinned tag.
The rest of the script assumes v1.28.1, but this block skips clone as soon as /var/local/git/grpc exists. A stale branch, dirty worktree, or missing submodules makes the build nondeterministic.
🔧 Normalize the checkout before building
if [ ! -d "/var/local/git/grpc" ]; then
git clone --recurse-submodules -b v1.28.1 --depth 1 https://github.com/grpc/grpc /var/local/git/grpc
else
- echo "gRPC repository already exists, skipping clone"
+ echo "Refreshing gRPC repository to v1.28.1..."
+ git -C /var/local/git/grpc fetch --depth 1 origin tag v1.28.1
+ git -C /var/local/git/grpc checkout --force v1.28.1
+ git -C /var/local/git/grpc submodule update --init --recursive --depth 1
fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| echo "[2/7] Cloning gRPC repository..." | |
| mkdir -p /var/local/git | |
| if [ ! -d "/var/local/git/grpc" ]; then | |
| git clone --recurse-submodules -b v1.28.1 --depth 1 https://github.com/grpc/grpc /var/local/git/grpc | |
| else | |
| echo "gRPC repository already exists, skipping clone" | |
| fi | |
| echo "[2/7] Cloning gRPC repository..." | |
| mkdir -p /var/local/git | |
| if [ ! -d "/var/local/git/grpc" ]; then | |
| git clone --recurse-submodules -b v1.28.1 --depth 1 https://github.com/grpc/grpc /var/local/git/grpc | |
| else | |
| echo "Refreshing gRPC repository to v1.28.1..." | |
| git -C /var/local/git/grpc fetch --depth 1 origin tag v1.28.1 | |
| git -C /var/local/git/grpc checkout --force v1.28.1 | |
| git -C /var/local/git/grpc submodule update --init --recursive --depth 1 | |
| fi |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install-dependencies.sh` around lines 38 - 44, The existing
check that skips cloning when the grpc repo directory exists can leave a stale
or dirty checkout; change the else branch to normalize the existing checkout:
fetch all tags and branches, checkout the pinned tag v1.28.1, hard-reset the
worktree to that tag, remove untracked files, and ensure submodules are
synchronized and updated/initialized recursively so the repo is deterministic
before continuing with build steps that expect v1.28.1.
| 1. Download the MANUS Core SDK from [MANUS Downloads](https://docs.manus-meta.com/3.1.0/Resources/) | ||
| 2. Extract and place the `ManusSDK` folder inside `src/plugins/manus/`, or set the `MANUS_SDK_ROOT` environment variable to your installation path | ||
| 3. Follow the [MANUS Getting Started guide for Linux](https://docs.manus-meta.com/3.1.0/Plugins/SDK/Linux/) to install the dependencies and setup device permissions. |
There was a problem hiding this comment.
Point the manual-install docs at 3.1.1.
This section still links to the 3.1.0 MANUS docs while the automated path and installer pin 3.1.1. That is likely to send users to mismatched archive names or setup steps.
📝 Keep the docs aligned with the scripted install
-1. Download the MANUS Core SDK from [MANUS Downloads](https://docs.manus-meta.com/3.1.0/Resources/)
+1. Download the MANUS Core SDK from [MANUS Downloads](https://docs.manus-meta.com/3.1.1/Resources/)
...
-3. Follow the [MANUS Getting Started guide for Linux](https://docs.manus-meta.com/3.1.0/Plugins/SDK/Linux/) to install the dependencies and setup device permissions.
+3. Follow the [MANUS Getting Started guide for Linux](https://docs.manus-meta.com/3.1.1/Plugins/SDK/Linux/) to install the dependencies and setup device permissions.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| 1. Download the MANUS Core SDK from [MANUS Downloads](https://docs.manus-meta.com/3.1.0/Resources/) | |
| 2. Extract and place the `ManusSDK` folder inside `src/plugins/manus/`, or set the `MANUS_SDK_ROOT` environment variable to your installation path | |
| 3. Follow the [MANUS Getting Started guide for Linux](https://docs.manus-meta.com/3.1.0/Plugins/SDK/Linux/) to install the dependencies and setup device permissions. | |
| 1. Download the MANUS Core SDK from [MANUS Downloads](https://docs.manus-meta.com/3.1.1/Resources/) | |
| 2. Extract and place the `ManusSDK` folder inside `src/plugins/manus/`, or set the `MANUS_SDK_ROOT` environment variable to your installation path | |
| 3. Follow the [MANUS Getting Started guide for Linux](https://docs.manus-meta.com/3.1.1/Plugins/SDK/Linux/) to install the dependencies and setup device permissions. |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/README.md` around lines 44 - 46, Update the MANUS version
references in the README: change both URLs and any text that point to 3.1.0 to
3.1.1 (specifically replace https://docs.manus-meta.com/3.1.0/Resources/ and
https://docs.manus-meta.com/3.1.0/Plugins/SDK/Linux/ with the 3.1.1 equivalents)
so the manual-install instructions match the automated installer; ensure the
mention of the ManusSDK folder and MANUS_SDK_ROOT remains the same but reflects
the 3.1.1 archive name where applicable.
HMD Optical Hand Tracking
Summary
This PR adds support for using the HMD's built-in optical hand tracking (via the
XR_MNDX_xdev_spaceextension) as the wrist position source when using Manus gloves, along with automated installation scripts and improved documentation.Changes
HMD optical hand tracking wrist source (
XR_MNDX_xdev_space)XR_MNDX_xdev_spaceandXR_EXT_hand_trackingextension function pointers. If both are available, it creates two nativeXrHandTrackerEXThandles backed by the HMD's optical tracking ("Head Device (0)" = left, "Head Device (1)" = right).inject_hand_data()now auto-selects the wrist pose source: HMD optical tracking when available (m_xdev_available == true), falling back silently to controller aim-pose + wrist offset when not.[Manus] Initialized with wrist source: HandTrackingorControllers.initialize_xdev_hand_trackers(),cleanup_xdev_hand_trackers(),update_xdev_hand(),get_controller_wrist_pose().m_handlesis now stored as a member so it can be reused across the new helper methods without fetching it repeatedly.HandTrackeris now registered alongsideControllerTrackerin theDeviceIOSession.XR_MNDX_xdev_spaceis requested as a required extension at session creation time.std::coutlog messages prefixed with[Manus]for consistent filtering.manus_hand_tracker_printer.cpp
[Manus]to match the plugin log format.install_manus.sh (new file)
All-in-one installation script:
install-dependencies.sh (new file)
Full dependency installer for the Remote mode:
abseil-cpppatch required for GCC 11+ compatibility.ldconfig.README.md
install_manus.sh.setup_cloudxr_env.sh,run_cloudxr.sh) as a prerequisite to running the plugin.Summary by CodeRabbit
Release Notes
New Features
Documentation
Improvements