-
Notifications
You must be signed in to change notification settings - Fork 148
Description
Dear authors,
In my setup, I have a bimanual dexterous robot with:
Right arm: 6 DoFs
Left arm: 6 DoFs
Right hand: 12 DoFs
Left hand: 12 DoFs
I would like to finetune RDT-1B on this system. To embed my action/state representation into your 128-dimensional workspace, I am considering the following layout:
Right arm joint positions (6 dims): indices 0–5 (a subset of your [0, 10) block)
Left arm joint positions (6 dims): indices 50–55 (a subset of your [50, 60) block)
Right + left hand joint positions (12 + 12 dims): indices 103–126 (24 dims within the reserved [103, 128) block)
The remaining arm-related joint slots, velocities, and some reserved indices would simply be left unused or filled with zeros.
My questions are:
- From the design perspective of the unified 128-D workspace, is it reasonable to reuse the reserved range [103, 128) to store additional joint positions for dexterous hands in this way?
- During pretraining of RDT-1B, did you make any implicit assumptions about the reserved indices (e.g., that they are always zero or ignored by the model)? In other words, could populating [103, 127] with non-zero hand joint values during finetuning cause a significant distribution shift that you would be concerned about?
- Are there any recommended practices or constraints for extending the joint-angle part of the vector when moving from a single-arm setup to a bimanual dexterous setup?
Any feedback on whether this index assignment is compatible with your unified representation and pretraining assumptions would be greatly appreciated.
Thank you very much for your time and for releasing this excellent work.