Learning goal
After watching, you understand what MaskBench measures (pose accuracy + kinematics) and why those measurements are the bridge between "I masked it" and "I can defend that the masked data is still useful." You know the high-level workflow and what a MaskBench run produces.
Transcript
Cold open
You masked the video. How do you prove it's still useful? That the gesture you care about is still trackable, that the kinematic signal didn't collapse in post-processing? That's what stage two is for.
Why a separate benchmark tool
"It looks fine to me" is not a measurement. Visual inspection finds obvious failures. It does not catch subtle degradation. A masked video can look perfect while the pose estimator behind it silently produces noisier kinematics — that's a problem you only see when you measure.
MaskBench answers the quantitative version of the question. Take your masked output. Run pose estimation on it. Compare against the original or against ground truth. Report numbers — accuracy and kinematics.
It also benchmarks pose estimators against each other. Seven models in the box: YOLOv11-Pose, MediaPipePose, OpenPose, and four variants including the MaskAnyone pipeline itself. One workflow, comparable numbers.
What it measures
MaskBench measures two things, and you need both.
Accuracy. How close are the predicted keypoints to ground truth? Three core metrics. PCK, percentage of correct keypoints under a tolerance. RMSE, root-mean-square error across all frames. Euclidean distance, raw per-keypoint deviation. You need labelled ground-truth poses for these.
Kinematics. The smoothness of the signal itself. Velocity, how fast each keypoint moves between frames. Acceleration, how that velocity changes. Jerk, the rate of change of acceleration — the noise-sensitive one. You can compute these without ground truth, just from the predicted poses.
When there are multiple people, MaskBench uses the Hungarian algorithm to align person-indices across frames and between prediction and ground truth — so the metric doesn't get confused when person A and person B swap places.
Every run produces four artefacts: a per-keypoint plot for spatial patterns, a distribution plot for the full spread, an inference-time plot for performance, and a results table for the headline numbers.
The workflow — five steps
Step 1 — Dataset prep. Drop your videos in a videos folder. If you have ground-truth annotations, drop them in a labels folder, one JSON per video. That's the simple structure. Complex multi-level datasets are supported too.
Step 2 — Configuration. A YAML file declares everything: which dataset, which pose estimators to run, which metrics to compute, where outputs go. Version-control the YAML; it's the reproducibility contract.
Step 3 — Inference. Run the benchmark. MaskBench creates a timestamped checkpoint folder, runs every pose estimator on every video, and saves the JSON pose results. This is the slow step — overnight is normal.
Step 4 — Evaluation. Metrics run across all pose estimators and all videos. Person matching happens automatically. Missing or unmatched people are tracked, not silently dropped.
Step 5 — Visualisation. The framework produces the plots, the tables, and a folder of rendered videos showing each pose estimator's output overlaid on the original. The rendered videos are the qualitative companion to the numbers.
Reading the output — a worked example
A real example. A per-keypoint acceleration plot. Each column is a keypoint — nose, shoulders, hips, wrists. Each colour is a pose estimator. Tall bars mean noisy estimates.
The wrists and ankles always show more noise — they move faster, they leave the frame more often, they get occluded. That's expected. The question is whether your pose estimator's wrist noise is comparable to the others or way above the line.
And inference time — how long each estimator took. Cheap-and-fast versus slow-but-accurate is a real trade-off. The plot makes it visible. MaskBench doesn't tell you which estimator to use. It gives you the data to defend the choice you make.
When to run it
Four scenarios where running MaskBench is worth your time:
- Starting a new pilot. Benchmark a sample before you commit to processing the whole corpus.
- Writing a methods paper. Reviewers will ask why you picked this estimator and not that one. Have the numbers.
- Convincing a data steward that masked outputs preserve your analytical signal. The numbers move conversations the screenshots can't.
- After a model update. A pose estimator released a new version. Re-run MaskBench on a reference subset — confirm the upgrade didn't quietly break your downstream analysis.
Recap
MaskBench: two metric buckets — accuracy and kinematics. Five-step workflow — dataset prep, YAML config, inference, evaluation, visualisation. Output is plots, tables, and rendered overlay videos. Use it to defend your choices with numbers.
Next stage of the pipeline — MaskingOps. When you're not running one video, you're running a thousand, and you need dashboards, queues, and QA review across the whole batch.