Video Annotation Tool (UI)

A PyQt6-based GUI for analyzing and annotating OSL format datasets (OpenSportsLab).

Features

• Open and visualize OSL-style data and annotations.
• Annotate and edit events/actions with a user-friendly GUI.
• Author Streaming VQA questions at a shared timeline ask time, with multiple choices and one correct answer; see the workflow.
• Keep Localization and Dense Description times stable across multi-modality changes with authoritative UTC timestamps and projected timeline positions.
• Manage labels/categories and export results for downstream tasks.
• Easy to extend with additional viewers, overlays, and tools.

---

🔧 Environment Setup

We recommend using Anaconda or Miniconda for managing your Python environment.

Note: The GUI project lives in the annotation_tool/ subdirectory of this repository.

Step 0 – Clone the repository

git clone https://github.com/OpenSportsLab/VideoAnnotationTool.git
cd VideoAnnotationTool

Step 1 – Create a new Conda environment

conda create -n VideoAnnotationTool python=3.12 -y
conda activate VideoAnnotationTool

Step 2 – Install dependencies

pip install -r requirements.txt

---

🚀 Run the GUI

From the repository root, launch the app with:

python annotation_tool/main.py

A window will open where you can load your data and start working. The application menu bar is displayed inside the window on every platform, including macOS.

For the temporal JSON contract and multi-input UTC workflow, see the OSL JSON format and synchronized playback guide.

Inference supports the permanent Local provider plus multiple named remote OpenSportsLib servers. Jobs share one table, run one at a time per provider, and retain finished-job metadata until Clear Finished is used or the application closes. See the inference provider and jobs guide.

---

📦 Download Test Datasets

This project provides test datasets for multiple tasks, including:

• Classification
• Localization
• Description (Video Captioning)
• Dense Description (Dense Video Captioning)

More details are available at: /test_data

⚠️ Important For all tasks, the corresponding JSON annotation file must be placed in the same directory as the referenced data folders (e.g., test/, germany_bundesliga/, etc.). Otherwise, the GUI may not load the data correctly due to relative path mismatches.

Some Hugging Face datasets (including OSL datasets) are restricted / gated. Therefore you must:

1. Have access to the dataset on Hugging Face
2. Be authenticated locally using your Hugging Face account (hf auth login)

---

✅ Requirements

• Python 3.x
• huggingface_hub (install via pip install huggingface_hub)

---

🧩 Universal Downloader (recommended)

We provide a single script that downloads only the files referenced by a given JSON annotation file:

• Downloads the JSON
• Parses data[].inputs[].path (and legacy videos[].path)
• Downloads the referenced files while preserving the repo folder structure

Script:

• test_data/download_osl_hf.py

Common usage:

python test_data/download_osl_hf.py \
  --repo-id <HF_DATASET_REPO> \
  --revision <HF_REVISION> \
  --split <SPLIT> \
  --format json \
  --output-dir <LOCAL_OUTPUT_DIR>

JSON downloads fetch <split>.json and every file referenced from its inputs. Parquet downloads fetch and convert the <split>/ folder. Files are written under <output-dir>/<revision>/<split>.

Use --dry-run to preview and estimate total size:

python test_data/download_osl_hf.py \
  --repo-id <HF_DATASET_REPO> \
  --revision <HF_REVISION> \
  --split <SPLIT> \
  --format json \
  --output-dir <LOCAL_OUTPUT_DIR> \
  --dry-run

🟦 Classification – Test Data

Data location (HuggingFace):
Classification Dataset

This folder contains multiple action-category subfolders (e.g. action_0, action_1, …).

📥 Download via command line

Classification – svfouls

python test_data/download_osl_hf.py \
  --repo-id OpenSportsLab/soccernetpro-classification-vars \
  --revision svfouls \
  --split annotations_test \
  --format json \
  --output-dir Test_Data/Classification/svfouls

Classification – mvfouls

python test_data/download_osl_hf.py \
  --repo-id OpenSportsLab/soccernetpro-classification-vars \
  --revision mvfouls \
  --split annotations_test \
  --format json \
  --output-dir Test_Data/Classification/mvfouls

🟩 Localization – Test Data

Data location (HuggingFace):

• Localization Dataset (Soccer)
• Localization Dataset (Tennis)
• Localization Dataset (gymnastics)

Each folder (e.g., england efl/) contains video clips for localization testing.

📥 Download via command line

From the repository root:

python test_data/download_osl_hf.py \
  --repo-id OpenSportsLab/soccernetpro-localization-snbas \
  --revision 224p \
  --split annotations-test \
  --format json \
  --output-dir Test_Data/Localization

🟪 Description (Video Captioning) – SoccerNet-XFoul

Dataset (Hugging Face): Description Dataset

This dataset provides video captioning samples in OSL JSON format. Each split JSON references clips under its corresponding folder:

• annotations_train.json → train/
• annotations_valid.json → valid/
• annotations_test.json → test/

📥 Download Test Split

python test_data/download_osl_hf.py \
  --repo-id OpenSportsLab/soccernetpro-description-xfoul \
  --revision main \
  --split annotations_test \
  --format json \
  --output-dir Test_Data/Description/XFoul

After download, you should have a structure like:

Test_Data/Description/XFoul/
  annotations_test.json
  test/
    action_0/
      clip_0.mp4
      clip_1.mp4
    ...

---

🟧 Dense Description (Dense Video Captioning)

Dataset (Hugging Face): Dense—Description Dataset

This dataset provides dense captions aligned with timestamps (half-relative), in a unified multimodal JSON format. Each item typically references:

• half video (.../1_224p.mp4 or .../2_224p.mp4)
• raw caption file (.../Labels-caption.json)
• optional visual features (e.g., features/I3D/.../*.npy)

📥 Download Test Split

python test_data/download_osl_hf.py \
  --repo-id OpenSportsLab/soccernetpro-densedescription-sndvc \
  --revision main \
  --split annotations-test \
  --format json \
  --output-dir Test_Data/DenseDescription/SNDVC

Expected structure (example):

Test_Data/DenseDescription/SNDVC/
  annotations-test.json
  germany_bundesliga/
    2014-2015/
      <match_folder>/
        1_224p.mp4
        2_224p.mp4
        Labels-caption.json
  features/
    I3D/
      germany_bundesliga/...
        1_224p.npy
        2_224p.npy

---

🧰 Build a standalone app (PyInstaller)

This project can be packaged into a standalone desktop app using PyInstaller. The commands below assume you run them from the repository root.

Note: The app bundles runtime assets from style/, ui/, and controllers/. This matches the GitHub Actions build configuration.

---

macOS (.app)

cd annotation_tool

python -m PyInstaller --noconfirm --clean --windowed \
  --name "VideoAnnotationTool" \
  --add-data "style:style" \
  --add-data "ui:ui" \
  --add-data "controllers:controllers" \
  main.py

Output:

• annotation_tool/dist/VideoAnnotationTool.app

---

Windows / Linux (one-file binary)

Linux

cd annotation_tool

python -m PyInstaller --noconfirm --clean --windowed --onefile \
  --name "VideoAnnotationTool" \
  --add-data "style:style" \
  --add-data "ui:ui" \
  --add-data "controllers:controllers" \
  main.py

Output:

• annotation_tool/dist/VideoAnnotationTool

Windows (PowerShell)

On Windows, the --add-data separator is ; (not :).

cd annotation_tool

python -m PyInstaller --noconfirm --clean --windowed --onefile `
  --name "VideoAnnotationTool" `
  --add-data "style;style" `
  --add-data "ui;ui" `
  --add-data "controllers;controllers" `
  main.py

Output:

• annotation_tool\dist\VideoAnnotationTool.exe

---

🤖 How executables are built (CI / GitHub Releases)

In addition to manual PyInstaller builds, standalone executables are automatically built using GitHub Actions.

Release builds (GitHub Releases)

When a version tag matching v* or V* (e.g., v1.0.7) is pushed, the release workflow runs:

• Workflow: .github/workflows/release.yml
• Builds for: Windows, macOS, Linux
• Packages outputs into ZIP archives
• Uploads ZIP files as GitHub Release assets
• Generates release notes from recent commit messages

The build commands in CI mirror the manual PyInstaller commands above (including bundling style/, ui/, and controllers/). Maintainers should use the complete release checklist when preparing and verifying a release.

Manual build artifacts (workflow dispatch)

There is also a standalone build workflow that can be triggered manually:

• Workflow: .github/workflows/ci.yml
• Builds for: Windows, macOS, Linux
• On manual run (workflow_dispatch), it zips the binaries and uploads them as Actions artifacts (short retention)

CI workflows overview

• ci.yml: Multi-platform build (manual artifacts on workflow_dispatch; also runs on pushes to selected branches)
• release.yml: Multi-platform build + GitHub Release publishing (triggered by version tags)
• deploy_docs.yml: Documentation build and deployment (MkDocs)

---

📜 License

This Video Annotation Tool project offers two licensing options to suit different needs:

• AGPL-3.0 License: This open-source license is ideal for students, researchers, and the community. It supports open collaboration and sharing. See the LICENSE.txt file for full details.
• Commercial License: Designed for commercial use, this option allows you to integrate this software into proprietary products and services without the open-source obligations of GPL-3.0. If your use case involves commercial deployment, please contact the maintainers to obtain a commercial license.

Contact: OpenSportsLab / project maintainers.