Video Pose Repetition Labeller

A simple and intuitive GUI application for annotating exercise repetition segments in videos using JSON keypoint data present in CFRep.

Overview

This tool provides a streamlined workflow for manually labeling exercise repetition phases in videos. It reads JSON keypoint files generated by the RepJudge project and allows you to annotate temporal segments corresponding to different states of exercise repetitions.

Features

Core Functionality

• Simple Video Navigation: Frame-by-frame control with play/pause and slider
• JSON Integration: Reads pose keypoint data from RepJudge output files
• Batch Processing: Load and annotate multiple samples from organized directories
• Persistent Annotations: Save labels back to JSON files for further analysis

Labeling Modes

• Classic State Sequence Mode: Mark continuous state transitions (prep → rep → no-rep → finish)
• Rep Boundary Mode: Mark individual rep start/end points with automatic classification
• Manual Windowing Mode: Create custom segments for new videos without pose data
• Flexible Binary Labels: Edit and update binary labels on the fly

Advanced Features

• Annotation Editing: Edit, delete, or insert segments after creation
• New Video Support: Load and annotate videos without existing keypoint data
• Video Configuration: Build aggregated config files (video_config.json) for all labeled videos
• Automatic File Management: Rename and organize new videos into dataset structure

Exercise State Sequence

The application generates a state sequence based on a binary label (e.g., "1010"):

• prep - Preparation phase before exercise starts
• rep - Active repetition phase (when binary digit is '1')
• no-rep - Inactive/transition phase (when binary digit is '0')
• finish - Completion phase after exercise ends

For binary label "1010", the sequence would be: prep → rep → no-rep → rep → no-rep → finish

Installation

Prerequisites

• Python 3.8+
• OpenCV (for video processing)
• Pillow (for image handling)
• tkinter (usually included with Python)

Setup

1. Clone this repository:

   git clone https://github.com/lc-leonardo/RepLabeler.git
   cd RepLabeler

2. Install dependencies:

   pip install -r requirements.txt

3. Run the application:

   python video_pose_labeler.py

   Or use the launch script:

   ./launch.sh

Usage

1. Directory Structure

Ensure your data follows this structure (as generated by RepJudge):

project_root/
├── json_keypoints/          # Pose keypoint JSON files
│   ├── exercise_type_1/
│   │   ├── sample_001/
│   │   │   └── *.json
│   │   └── sample_002/
│   └── exercise_type_2/
└── videos/                  # Corresponding video files
    ├── sample_001.mp4
    └── sample_002.mp4

2. Loading Data

1. Click "Select json_keypoints folder..." and choose your json_keypoints directory
2. Select an exercise type from the left panel
3. Select a specific sample
4. Double-click or click "Load selected sample"

3. Providing Binary Labels

• If the JSON doesn't contain a binary_label field, you'll be prompted to enter one
• Enter a string of 0s and 1s representing rep/no-rep phases (e.g., "1010")
• This generates the state sequence for annotation

4. Annotating Videos

Classic State Sequence Mode (Default)

1. Use video controls to navigate:

   • Play/Pause: Start/stop video playback
   • Frame buttons: Step forward/backward by single frames
   • Slider: Jump to specific frames

2. Mark state boundaries:

   • Position the video at the end of the current state
   • Click "Mark end of current state"
   • Progress through: prep → rep/no-rep phases → finish

Rep Boundary Mode (New!)

For precise labeling when athletes rest between reps:

1. Enable Rep Mode: Check "Rep start/end mode" checkbox
2. Mark Each Rep:
   • Navigate to rep start → Click "Mark rep start"
   • Navigate to rep end → Click "Mark rep end"
   • If binary label exists: segment is automatically classified as rep/no-rep
   • If no binary label: you'll be prompted to classify as rep or no-rep
3. Repeat for all reps - rest periods are naturally excluded
4. Benefits:
   • Handles incomplete reps
   • Marks only actual rep executions
   • Automatic rep/no-rep classification based on binary label

Annotation Management

• Edit Selected: Double-click or select annotation and click "Edit Selected"
• Delete Selected: Remove specific annotation segments
• Insert Segment: Add new segments at any point
• Undo last mark: Remove the most recent annotation
• Clear annotations: Start over with a clean slate
• Binary Label Editing: Update binary labels directly in the UI

5. Loading New Videos (No Existing Data)

For adding completely new videos to your dataset:

1. Click "Load New Video..." button
2. Select video file from file browser
3. Choose annotation method:
   • Enter binary label and use state sequence mode, OR
   • Use manual windowing buttons (Mark as prep/rep/no-rep/finish), OR
   • Enable rep mode and mark individual reps
4. Save with naming: Provide exercise type, person ID, and camera angle
5. Auto-organization: Video is copied, renamed, and JSON files created automatically

6. Building Video Configuration

To create an aggregated registry of all labeled videos:

1. Click "Build video_config.json" button
2. Automatic aggregation: Scans all samples and creates unified config
3. Output: Creates video_config.json with entries for each video including:
   • Filename, exercise type, binary label
   • Rep count and full segment list
   • Sorted and formatted for easy parsing

7. Saving Results

• Classic mode: Complete all states before saving
• Rep mode: Save once you've marked all desired reps
• New videos: Provide naming info and files are auto-organized
• Click "Save annotations" to write labels back to JSON files
• Annotations are saved across all JSON files for the sample

JSON Format

Individual Frame JSON Files

Each frame's JSON file stores keypoints and segment label:

Classic State Sequence Mode:

{
    "file_name": "frame_0001.json",
    "annotations": {
        "segment": "prep",  // or "rep", "no-rep", "finish", "unlabeled"
        "keypoints": [...],
        "bbox": [...]
    }
}

Rep Boundary Mode: Individual reps marked with precise start/end frames, leaving rest periods as prep/finish:

// Frame in rest period before first rep
{
    "file_name": "frame_0010.json",
    "annotations": {
        "segment": "prep",
        "keypoints": [...],
        "bbox": [...]
    }
}

// Frame during first rep execution
{
    "file_name": "frame_0025.json",
    "annotations": {
        "segment": "rep",  // or "no-rep" if incomplete attempt
        "keypoints": [...],
        "bbox": [...]
    }
}

// Frame in rest period between reps
{
    "file_name": "frame_0040.json",
    "annotations": {
        "segment": "prep",  // automatically filled between reps
        "keypoints": [...],
        "bbox": [...]
    }
}

video_config.json Format

Aggregated registry of all labeled videos (created via "Build video_config.json" button):

{
    "squat_p1_45.mp4": {
        "exercise": "squat",
        "binary_label": "1111100",
        "rep_count": 7,
        "segments": ["prep", "prep", "rep", "rep", "no-rep", "rep", "rep", "rep", "rep", "rep", "finish"]
    },
    "pushup_p2_90.mp4": {
        "exercise": "pushup",
        "binary_label": "11111",
        "rep_count": 5,
        "segments": ["prep", "rep", "rep", "rep", "rep", "rep", "finish"]
    }
}

Integration with RepJudge

This tool is designed to work seamlessly with the RepJudge project:

1. Run RepJudge on your exercise videos to generate pose keypoints and initial JSON files
2. Use this labeller to add temporal annotations for repetition phases
3. Analyze results using the enriched JSON files with both pose data and repetition labels

Requirements

See requirements.txt for specific package versions.

Contributing

This is a simple, focused tool. Contributions are welcome for bug fixes and small improvements that maintain the tool's simplicity.

License

This project is provided as-is for research and educational purposes.

Support

For issues and questions:

1. Check this documentation
2. Review the console output for error messages
3. Open an issue in the RepJudge repository