Sports Computer Vision Datasets: 17 Public Benchmarks (2026)

If you are scouting how other teams approach sports computer vision, the datasets they benchmark on tell you more than the model papers do. A dataset fixes the task definition, the camera angle, the label schema and, quietly, what you are allowed to do with the result.

This guide covers 17 public datasets for multi-object tracking, ball tracking, athlete pose estimation, action recognition and tactical analysis. For each one it records the sport, the task, the scale and the licence, checked against the dataset's own repository or project page in September 2026.

It also answers the question most dataset lists skip: once your model works, can you ship it? For most of these datasets, not without extra permission. Only 3 of the 17 are openly licensed without depending on third-party footage.

Illustration: the kinds of tasks public sports datasets cover, from tracking and pose to ball and puck detection.

Key Takeaways

• In this list, public sports CV data is strongest for football (soccer), basketball and individual Olympic-style events. None of the 17 covers cricket or rugby, and the one cricket release we came across has 40 videos.
• Only 3 of the 17 are openly licensed without depending on third-party footage: TrackID3x3, SoccerTrack v2 and ShuttleSet, and only the first two include video. Of the rest, 6 are non-commercial only, 4 state no licence we could verify, 2 need an agreement or NDA, and 2 pair open labels with third-party footage.
• Domain-specific data matters. Fine-tuning a state-of-the-art 3D pose model on AthletePose3D cut its error on athletic motion from 214 mm to 65 mm (Yeung et al., CVPR 2025 workshop).
• Public datasets are the right starting point for benchmarking. They rarely match a production schema, camera setup or sport, which is where custom annotation takes over.

---

How we selected and checked these datasets

Every dataset here is publicly documented, built for sports-specific computer vision, and verified against a primary source: the authors' paper, project page or official repository. Survey articles and aggregator sites were used to find candidates, not to supply the numbers.

Four rules shaped the list:

• Sports-specific first. General benchmarks such as COCO and PoseTrack are widely used, but they were not built around athletic motion or match footage, so PoseTrack appears only as context in the pose section. One entry, ShuttleSet, is a stroke-record set without video, included because it ships its own tactical benchmarks.
• Scale as the authors state it. Some datasets report clips, some frames, some annotation instances. The figures below use each project's own unit rather than converting between them.
• Licence as published by the dataset, not the paper. An arXiv page shows the licence of the paper, which is often different. Every licence below links to the dataset's own repository or project page. Where none is stated, or the page could not be reached, the table says so.
• Readable, not exhaustive. Well-known sets such as Diving48, GolfDB and MTL-AQA were left out to keep the comparison manageable.

For a broader academic map of the field, A Survey of Deep Learning in Sports Applications (Zhao et al., 2023) and Deep Learning for Sports Video Event Detection (Xu et al., 2025) both catalogue datasets alongside methods.

---

The 17 datasets at a glance

---

Multi-object tracking and game state datasets

Tracking datasets label every player's bounding box and keep a consistent identity across frames, which is the foundation for positional data, distance metrics and tactical analysis. The hard part in team sports is identity: players wear identical kits, cluster tightly and leave the frame. The datasets below differ mainly in camera setup and in how far beyond boxes they go.

If you are deciding between box-level and keypoint-level labels for your own tracking work, see our comparison of skeletal tracking and bounding boxes in sports AI.

SoccerNet: football's multi-task benchmark

SoccerNet is built on 550 complete broadcast games and 12 single-camera games from major European leagues, with more than 300,000 temporal annotations and more than 1 million spatial annotations (SoccerNet). It hosts annual challenges across action spotting, ball action spotting, camera calibration, tracking, re-identification, jersey number recognition, dense video captioning, multi-view foul recognition and monocular depth estimation.

Its most complete tracking task is Game State Reconstruction (SN-GSR): 200 sequences of 30 seconds each (Somers et al., CVPR 2024 workshop). They carry 9.37 million pitch line points and over 2.36 million athlete positions, each with role, team and jersey number. Results are scored with a tracking metric called GS-HOTA.

Access: annotations download freely, but the SoccerNet data page states that downloading any video data requires signing an NDA.

SportsMOT: tracking across three sports

SportsMOT contains 240 video clips across basketball, football and volleyball, annotated with player boxes and persistent identities (SportsMOT, ICCV 2023). Test results are submitted to a CodaLab evaluation server, so scores are comparable across papers.

Licence: CC BY-NC 4.0, per the SportsMOT repository.

TeamTrack: full-pitch views from fisheye and drone cameras

TeamTrack covers football, basketball and handball in 4K to 8K full-pitch video, shot from a side-mounted fisheye camera and an overhead drone (Scott et al., CVPR 2024 workshop). It reports over 279,900 frames and, per its repository, over 4 million bounding boxes with unique IDs. Because the whole pitch stays in frame, identity has to persist without the camera cuts that broadcast footage introduces.

Licence: none stated for the data in the TeamTrack repository.

SoccerTrack v2: full matches rather than clips

SoccerTrack v2 provides 10 full-length panoramic 4K recordings of university matches, annotated in three layers (SoccerTrack v2, 2025):

• multi-object tracking;
• game state reconstruction, with 2D pitch coordinates, jersey-based IDs, roles and teams;
• ball action spotting across 12 classes such as pass, drive and shot.

Full matches matter because identity errors can compound over 90 minutes in ways a 30-second clip cannot show.

Licence: CC BY 4.0 for both videos and annotations, per the SoccerTrack v2 repository.

TrackID3x3: fixed cameras, drones and pose

TrackID3x3 targets a setup most of this list ignores: 3x3 basketball filmed by indoor fixed cameras, outdoor fixed cameras and drones, with over 277,000 frames (Yamada et al., MMSports 2025). Every frame has boxes for the six on-court players, and 6,701 frames also carry 10 two-dimensional pose keypoints per player. The authors recorded the footage themselves on a camcorder, smartphones and drones, and built the dataset because existing datasets focus on mainstream professional play and rarely include pose.

Licence: CC BY 4.0, per the TrackID3x3 repository.

DeepSportradar-v1: four basketball tasks in one release

DeepSportradar-v1 bundles four basketball datasets: ball 3D localisation, camera calibration, player instance segmentation and player re-identification, released with development kits and baselines for the ACM MMSports 2022 challenge (DeepSportradar-v1). Its paper reports 728 calibration views and 18,703 player crops for re-identification.

Licence: the instance segmentation data is CC BY-NC-ND 4.0, per its challenge repository. The re-identification repository states no data licence and credits Synergy Sports for the data. We count the release as non-commercial.

For how these tasks combine in a production pipeline, see our guide to basketball computer vision and NBA player tracking.

---

Ball and puck datasets

Ball tracking is its own problem because the target is tiny, blurred by motion and frequently invisible. Detection models tuned on players miss it, and a single occluded frame breaks naive trajectory logic.

TrackNet: tennis balls in broadcast footage

TrackNet introduced a heatmap-based network that detects small, fast, blurry balls from a single frame and learns flight patterns from consecutive frames (Huang et al., 2019). Its tennis dataset, built from 10 broadcast videos, labels each frame with ball visibility and position, which lets models learn when the ball is genuinely out of view rather than missed.

Licence: not verified. We could not reach the dataset's hosting page.

HockeyAI: players, puck and rink landmarks

HockeyAI provides around 2,100 high-resolution frames from Swedish Hockey League broadcasts with YOLO-format labels across 7 classes: player, puck, goaltender, referee, goal, face-off and centre ice (HockeyAI, ACM MMSys 2025). Companion datasets for rink keypoint mapping (HockeyRink) and player orientation (HockeyOrient) were presented at the same conference.

Licence: MIT on Hugging Face. The frames come from league broadcasts, so the MIT licence covers the released files but does not by itself settle rights in the broadcast footage. At about 2,100 frames it is a starting point for detection, not a tracking benchmark; our ice hockey computer vision guide explains why the puck is so hard to follow.

---

Pose estimation datasets for athletes

AthletePose3D's authors found that pose models trained on conventional datasets perform poorly on athletic motion, which is faster and more extreme than everyday movement, and then measured how large that gap is.

AthletePose3D: why athletic motion needs its own data

AthletePose3D covers 12 types of sports motion across roughly 1.3 million frames and 165,000 individual postures, captured specifically for high-speed, high-acceleration movement (Yeung et al., CVSports at CVPR 2025). The authors report that fine-tuning on it reduced a state-of-the-art monocular model's mean per-joint position error from 214 mm to 65 mm on athletic motion.

That is roughly a 70% reduction from changing the training data, and the clearest evidence in this list for a simple rule: fine-tune on data that looks like your deployment, not only on general benchmarks.

Licence: non-commercial and scientific research only, with a licence agreement before download, per the AthletePose3D repository. If you need openly licensed athlete keypoints, TrackID3x3 is the only dataset in this list that has them, at a far smaller scale and for one sport.

PoseTrack and other pose datasets worth checking

PoseTrack is the usual general-purpose benchmark for pose in video. Its 2017 release has 550 videos with 153,615 pose labels across 23,000 annotated frames, each person labelled with 15 keypoints in 2D and a track ID (Andriluka et al., CVPR 2018). It was not built for sport, so expect the domain gap AthletePose3D measured.

Two sport-specific 3D datasets are also public, though we did not review their licences for the tally below. SportsPose has more than 176,000 3D poses from 24 subjects across 5 sports activities (Ingwersen et al., CVSports 2023). WorldPose provides 88 sequences and more than 2.5 million 3D poses from 2022 FIFA World Cup footage, released for academic research (WorldPose, ECCV 2024).

---

Action recognition, detection and quality assessment datasets

Action datasets label what happens and when, from a whole clip's category down to the exact frame a movement starts. The granularity is what separates them: a video-level label, a timestamped event and a scored technique are three different training problems.

MultiSports: actions located in space and time

MultiSports labels 66 action classes across basketball (18), volleyball (12), football (15) and aerobic gymnastics (21), in 3,200 clips containing 37,701 action instances and 902,000 bounding boxes (Li et al., ICCV 2021). Each instance has both a time range and the box of the person performing it, which makes it a benchmark for detecting who did what, not just what happened.

Licence: CC BY-NC 4.0, per the MultiSports repository.

FineGym: a three-level action hierarchy

FineGym organises gymnastics into 10 events, 15 sets and 530 element categories, of which 354 have at least one instance, drawn from 303 competition records totalling about 708 hours (Shao et al., CVPR 2020). It contains 4,883 action instances and 32,697 sub-action instances, and covers 6 men's and 4 women's events. Its hierarchy is a useful model for anyone designing a sports event taxonomy.

Licence: annotations under CC BY-NC 4.0, per the FineGym project page; videos are referenced by YouTube ID.

FineDiving: scoring technique, not just naming it

FineDiving is built for action quality assessment: 3,000 dives from 30 competitions, including the Olympics and World Championships (Xu et al., CVPR 2022). Each dive is labelled with its action type (52 in total), sub-action types (29) and difficulty level (23), plus frame-level step boundaries. It targets models that judge how well something was done rather than whether it happened.

Access: a signed release agreement sent to the authors, per the FineDiving repository.

Volleyball: team activity from individual actions

The Volleyball dataset contains 4,830 annotated frames from 55 videos, each with player boxes, 9 individual action labels and 8 team activity labels such as left spike or right set (Ibrahim et al., CVPR 2016). It is built for group activity recognition, where the team's action has to be inferred from what each player is doing.

Licence: the code is BSD 2-Clause; no dataset licence is stated in the project repository.

Sports-1M and Olympic Sports: video-level classification

Introduced by Karpathy et al. at CVPR 2014, Sports-1M provides 1,133,158 YouTube videos across 487 sports classes, with labels generated automatically from YouTube Topics (Sports-1M repository). It is large but weakly labelled, and videos removed from YouTube are gone. Its annotations are CC BY 3.0; the footage belongs to its uploaders.

The Olympic Sports dataset (Niebles et al., ECCV 2010) covers 16 sports, from high jump and pole vault to tennis serve and basketball lay-up, sourced from YouTube and labelled through Amazon Mechanical Turk. Its dataset page states no licence.

Both suit video-level classification and pre-training rather than frame-accurate event work.

---

Tactical and language datasets

Some of the most useful sports data is not pixel-level at all: it records decisions, shot types and positions in a structure analysts can query.

ShuttleSet: stroke-by-stroke badminton

ShuttleSet records 36,492 strokes across 3,685 rallies in 44 matches from 2018 to 2021, featuring 27 top-ranked men's and women's singles players (Wang et al., KDD 2023). Each stroke carries one of 18 shot types, the hitting location and both players' positions. Its authors report that a visualisation platform built on it was used by national badminton teams during international matches.

Licence: released as CSV stroke records, not video, in an MIT-licensed repository.

NSVA: identity-aware NBA captions

NSVA contains 32,019 video clips from 132 games played by 10 teams in the 2018-19 NBA season, labelled for video captioning, fine-grained action recognition and salient player identification (Wu et al., ECCV 2022). Its captions name players and key actions such as made shots, misses and rebounds, so they can support statistics counting.

Licence: CC BY-NC for the code and data, per the NSVA repository.

Adjacent, but not vision data: the NFL Big Data Bowl publishes player tracking coordinates through annual Kaggle competitions. Like ShuttleSet, it records positions rather than footage, so it cannot train a detector or tracker. ShuttleSet is included because it ships its own tactical benchmarks; the Big Data Bowl is a competition dataset for movement prediction.

---

Can you train a commercial model on public sports datasets?

Usually not without extra permission. Of the 17 datasets in this guide, 6 are licensed for non-commercial use only and 2 require a signed agreement or NDA. Another 4 publish no licence we could verify, and 2 pair open annotations with third-party broadcast or YouTube footage. Only 3 are openly licensed without depending on third-party footage. ShuttleSet ships stroke records rather than footage, so TrackID3x3 and SoccerTrack v2 are the only openly licensed sources of training video here.

In practice this splits the list into two uses:

• Academic benchmarking. Most of these datasets allow comparison against published results under their terms. Whether evaluation inside a company counts as non-commercial depends on the licence (AthletePose3D, for example, is limited to scientific research), so check before running restricted data through a product pipeline.
• Training a model you will sell or deploy. The non-commercial and no-licence datasets are the wrong foundation. A model trained on CC BY-NC data can put a whole product in question, and "no licence stated" does not mean permission was granted.

Two cautions apply even to the open three. Check where any footage came from, because a licence on released files does not by itself settle rights in video someone else filmed. And licences change, so confirm the current terms on the project's own page before you build on it. None of this is legal advice.

---

Where public sports datasets run out

Public datasets map well onto published tasks and poorly onto production ones. Four gaps come up again and again in this list.

1. Sport coverage. Football, basketball, volleyball and individual Olympic events dominate. None of the 17 covers cricket or rugby; the one cricket release we came across, DeepSportradar's bowl-release challenge, has 40 videos and a non-commercial licence (repository). The one ice hockey dataset here is built for detection rather than tracking. If you work in cricket, our cricket computer vision guide covers what the task involves.

2. Your label schema. A public dataset encodes its authors' definitions. MultiSports has 66 action classes; your product may need a pick-and-roll split into screener and ball handler, or a foul typed as offensive. Relabelling to your taxonomy is usually more work than it looks.

3. Camera domain. Much of the video here comes from broadcasts, competition records or YouTube. TrackID3x3's authors built their dataset because fixed-camera amateur setups were missing. A model trained on broadcast angles can degrade on a single tripod camera, a drone or a phone, so test on your own camera setup early.

4. Sequence length. SN-GSR sequences run 30 seconds. Identity switches, re-identification after substitutions and minutes-played calculations only show their true error rates across full matches, and full-match tracking annotation is rare in this list; SoccerTrack v2 is the clearest example. The same 30-second limit shapes what the broadcast tracking benchmarks can show about identity across a whole game.

Coverage of women's competition is uneven too. FineGym and ShuttleSet include women's events, but we did not see women's coverage reported for most of the others; our piece on the women's sports data annotation gap looks at why.

---

When to use public data and when to build your own

Use public datasets to choose an approach, and your own annotated footage to make it work. A practical split:

If you reach the last three rows, our step-by-step guide on how to build a sports computer vision dataset covers schema design, sourcing and QA, and our in-house versus outsourced annotation comparison covers who should do the labelling.

---

Frequently asked questions

What datasets are used for sports computer vision?

Public options covered here include SoccerNet for football video understanding, SportsMOT and TeamTrack for multi-object tracking, MultiSports and FineGym for action recognition, FineDiving for action quality assessment, and AthletePose3D for 3D athlete pose. Sports-1M is a large source for video-level classification and pre-training.

Is there a public basketball computer vision dataset?

Yes, several. SportsMOT includes basketball tracking, DeepSportradar-v1 covers ball 3D localisation, calibration, segmentation and re-identification, TrackID3x3 covers 3x3 basketball tracking with 2D pose, MultiSports labels 18 basketball action classes, and NSVA provides NBA captions and action labels. TeamTrack also includes full-court basketball tracking but states no data licence. TrackID3x3 is CC BY 4.0; the others listed restrict or do not grant commercial use.

Can I use SoccerNet commercially?

Not the video without the organisers' agreement. SoccerNet's annotations download freely, but its video data requires signing an NDA. Check the current terms on the SoccerNet data page and confirm your intended use with the organisers before training a commercial model on it.

Which datasets can I use for sports pose estimation?

AthletePose3D is a large 3D pose dataset built specifically for athletic motion, with about 1.3 million frames across 12 motion types, but it is licensed for non-commercial research only. SportsPose and WorldPose are other sport-specific 3D options, and PoseTrack is the common general-purpose video benchmark. For an openly licensed option, TrackID3x3 provides 2D keypoints for 3x3 basketball under CC BY 4.0.

What is the Olympic Sports dataset?

The Olympic Sports dataset, released by Niebles et al. at ECCV 2010, contains YouTube videos of athletes across 16 sports, including high jump, discus, diving, weightlifting and gymnastics vault, labelled through Amazon Mechanical Turk. It is a classic video-level action classification benchmark.

---

Public data for benchmarks, custom data for production

Public sports computer vision datasets are excellent for learning what works. SoccerNet, SportsMOT, MultiSports and AthletePose3D define tasks and metrics the field reports against, and the two openly licensed video sets, TrackID3x3 and SoccerTrack v2, give you a legitimate place to start.

They stop short in predictable places: your sport, your schema, your cameras and your right to commercialise. That is the gap custom annotation fills.

If your model needs training data public benchmarks do not cover, whether that is tracking with persistent IDs, frame-accurate events, skeletal keypoints or a sport nobody has published, see Train Matricx pricing and the free pilot or review client results in our case studies. We annotate a free pilot clip so you can evaluate quality before committing to any volume.

Compiled by the Train Matricx annotation team. Dataset figures and licences were checked against each dataset's paper, repository or project page in September 2026. Spot an error or a licence change? Email info@trainmatricx.com.