Skip to content

Conversation

@deruyter92
Copy link
Collaborator

Summary

  • Extend FMPose3DInference with full animal (26-joint Animal3D) pipeline support,
    including SuperAnimalEstimator (DeepLabCut 2D), AnimalPostProcessor (limb
    regularisation), and the for_animals() convenience constructor.
  • Add a basic test suite (tests/test_fmpose3d.py) covering all public and
    internal components of the inference API with 78 tests (64 unit + 14 functional).

Changes

fmpose3d/fmpose3d.py

  • Add SuperAnimalEstimator: wraps DeepLabCut SuperAnimal, maps quadruped80K keypoints
    to the 26-joint Animal3D layout via _map_keypoints, handles missing detections
    (zero-fill fallback).
  • Add AnimalPostProcessor: applies limb regularisation (Rodrigues rotation aligning
    average limb direction to vertical) instead of root-zeroing / camera_to_world.
  • Add compute_limb_regularization_matrix and apply_limb_regularization helpers.
  • Add _default_components resolver that selects the correct estimator + postprocessor
    based on model_type.
  • Add FMPose3DInference.for_animals() class method for one-line animal pipeline setup.

tests/test_fmpose3d.py (new)

  • Unit tests (64): _map_keypoints (direct mapping, interpolation, out-of-range,
    multi-individual), limb regularisation (identity, rotation, 180-degree, degenerate,
    valid-rotation properties), both postprocessors (shape, mutation, camera_rotation
    handling, all-zero pose), _default_components, FMPose3DInference construction
    (for_animals, custom injection, _resolve_pad), _ingest_input (arrays, paths,
    directories, 8 error cases), _load_weights error paths, pose_3d input validation
    (wrong ndim, zero frames, 4D→first person, TTA path, progress callback, seed
    reproducibility), dataclasses, mocked SuperAnimalEstimator.predict (no-detection
    and valid-detection branches).
  • Functional tests (14): full human pipeline (HRNet 2D → FMPose3D 3D, shape checks,
    root-zeroed, world z-floor, finite, seed reproducibility) and full animal pipeline
    (DLC 2D → FMPose3D-animals 3D, shape checks, finite, magnitude sanity,
    reproducibility). Marked @pytest.mark.functional and auto-skipped when pretrained
    weights are not present.

pyproject.toml

  • Register functional pytest marker.

@deruyter92 deruyter92 changed the base branch from main to ti_video_demo February 11, 2026 12:31
Base automatically changed from ti_video_demo to main February 11, 2026 13:18
Copy link

@C-Achard C-Achard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good ! Should be easy to use downstream, definitely great there is some testing.

Some minor comments for future usage but nothing major.


@staticmethod
def _map_keypoints(xy: np.ndarray) -> np.ndarray:
"""Map quadruped80K keypoints to Animal3D 26-joint layout.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit of a high-level comment, would it be useful to minimally document the nomenclature somewhere ? In regards to DLC and all its functionalities "Animal3D" may be a bit ambiguous (and making it more specific could help with LLM parsing dataset structures)


@staticmethod
def _map_keypoints(xy: np.ndarray) -> np.ndarray:
"""Map quadruped80K keypoints to Animal3D 26-joint layout.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NB: I don't mean this for standards like COCO, just internal names/our own nomenclature

]


def compute_limb_regularization_matrix(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems primarily intended for visualization / canonicalization but a note is that it may not be suitable for all poses (e.g. lying down or jumping) ? As it assumes a stable “up” limb direction. It can also become unstable across frames (which could cause flickering in some cases) as there is no temporal information, only a per frame computation with no prior.
Probably worth documenting these limitations for people so they use the raw pose or come up with their own logic for their data if the assumptions are different in their case, but not critical for now.

inspected to choose pipeline components. Adding a third pipeline
means adding one branch here (or turning this into a registry).
"""
if model_cfg.model_type == "fmpose3d_animals":

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would using an Enum here make sense to avoid raw str checks ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants