Player Re-ID Labeling: ID Rules for Cuts and Occlusion

Player re-ID labels are only as good as the rules behind them. Before anyone draws a box, a labeling spec has to answer five questions: how long an ID must stay true, what happens when a player leaves the frame and returns, how identity is carried across a camera cut, what counts as evidence that two appearances are the same player, and what an annotator does when they cannot tell.

Player re-identification (re-ID) is the task of deciding that a player seen now is the same player seen earlier, after a cut, an exit from the frame or an occlusion. A track ID is the label that records that decision.

Public tracking datasets answer these questions differently, and the best-known ones avoid the hardest question altogether by using footage with no camera cuts. If your model has to work on broadcast video, you will be writing rules that no benchmark hands you.

This guide goes through each rule, shows how published datasets handled it, and ends with a checklist you can turn into a spec. It is the labeling companion to our overview of broadcast player tracking, court registration and re-ID, which covers the models.

Key Takeaways

• Decide the scope of an ID first: tracklet, clip, match or season. Every other rule depends on it.
• Public datasets disagree on re-entry. MOT16 gives a returning person a new ID; SoccerNet-Tracking keeps the same one (Milan et al., 2016; Cioppa et al., 2022).
• SportsMOT and SoccerNet-Tracking both use footage without camera cuts, so neither tells you how to label identity across one.
• Write down what counts as evidence for an identity link, in order of strength. Jersey number comes first, and it is often not visible.
• Give annotators an "unknown" state. A confident wrong link does more damage than an honest gap.

Why does identity need its own labeling spec?

Because a box can be perfect in every frame while the identity attached to it is wrong, and frame-level review will not catch it. Identity is a property of a sequence, so its errors only show up over time.

The tracking literature has a name for the basic failure. An identity switch happens when a tracker swaps the identities of two objects, or loses a track and restarts it under a different identity (Luiten et al., HOTA, 2020). Ground truth can contain the same mistake. If an annotator swaps two teammates as they come out of a screen, every frame after that point teaches the model the wrong player.

Sport makes this easier to get wrong than almost any other domain. Teammates wear the same kit, players cross and collide constantly, and broadcast coverage keeps cutting between cameras. A generic instruction such as "keep IDs consistent" leaves each annotator to invent their own answer to all of it.

Rule 1: How long does an ID have to stay true?

Set the identity scope before anything else. An ID that only has to hold inside one unbroken shot is a different job, at a different cost, from an ID that has to hold for a whole match.

Published datasets sit at different points on this ladder:

• Tracklet scope. MOT16, a widely used pedestrian tracking benchmark, ends a track when a person leaves the field of view (Milan et al., 2016).
• Clip or action scope. SoccerNet-Tracking holds identity for the length of a clip. It has 200 sequences of 30 seconds each plus one complete 45-minute half (Cioppa et al., 2022). SoccerNet ReID is narrower still: its identity labels are valid only within a single action, and the same player gets a different identity in each action he appears in (SoccerNet sn-reid).
• Match scope. This is what most products need, and it is where public data thins out. Our guide to sports computer vision datasets lists what exists.

Scope decides what the label promises to the model. If you train on action-scoped identities and expect match-long identity at inference, the labels never showed the model the thing you are asking for.

Our recommendation is to choose the scope your product needs, then check whether you can afford it. Match scope requires a roster and a way to anchor each player to it, which is Rule 4.

Rule 2: What happens when a player leaves the frame and comes back?

State it explicitly, because two well-known conventions are opposites.

Under MOT16's rules, a person who leaves the field of view and appears again later is assigned a new ID. The same applies after a long occlusion where the person's position could not be followed (Milan et al., 2016).

SoccerNet-Tracking does the reverse on purpose. Its authors chose to keep the same ID for an object that leaves the camera frame and comes back later in the same clip, and they present this as what separates the dataset from traditional tracking benchmarks (Cioppa et al., 2022).

Neither is wrong. MOT16's rule produces labels an annotator can be sure of. SoccerNet-Tracking's rule produces labels a sports product can use. The problem comes when a team mixes them: an annotator used to pedestrian-style tracking may follow the first while the engineer assumed the second.

For team sport we recommend the keep-the-ID rule within your chosen scope, with one condition attached. The annotator must have evidence for the link, not just a plausible guess. That evidence is defined in Rule 4.

Rule 3: How is identity carried across a camera cut?

No widely used tracking benchmark answers this, because they were built from footage without cuts. You have to write the rule yourself.

SportsMOT covers basketball, volleyball and football in 240 sequences, with over 150,000 frames and 1.6 million boxes. Each clip was manually checked to make sure it contained no abrupt viewpoint switches (Cui et al., ICCV 2023). SoccerNet-Tracking uses main-camera video, and its authors say plainly that single-camera sequences suit tracking better than broadcast video, where camera changes and replays break the continuity of tracks (Cioppa et al., 2022).

So a broadcast spec needs three things the benchmarks leave out.

Label the shot boundaries first. Every cut is a point where motion continuity is lost and identity has to be re-established from appearance. If cuts are marked as their own layer, reviewers know exactly where to look for identity errors.

Classify each shot. Live play from the main camera, a close-up, a replay, a crowd or bench shot. Replays matter most: a replay shows game time that has already been labeled, so the spec must say whether replay shots are labeled at all, and whether a player in a replay shares an ID with the live action. SoccerNet ReID is built on exactly that link between an action frame and its replay frames.

Define the link procedure. When a new shot starts, the annotator re-identifies each player against the players already known in the clip or match, using the evidence order in Rule 4. Players who cannot be linked are marked unknown under Rule 5 instead of being forced into an existing ID.

AI-generated illustration: retain the same track ID across a camera cut only when identity evidence supports the link. If occlusion hides that evidence, leave the new track unlinked and mark its identity unknown pending review. Tracking overlays are illustrative.

In our view the cut is the likeliest place for two careful annotators to disagree, which makes it the right place to spend review time.

Rule 4: What counts as evidence that two appearances are the same player?

Write an ordered list, strongest first, and require the annotator to be able to point to at least one item on it.

A workable order for team sport:

1. Jersey number plus team. The only evidence that maps directly to a roster.
2. Continuous motion. The player was followed without a break, so no re-identification was needed.
3. Position and role in a set structure. The goalkeeper, the server, the player taking the throw-in.
4. Distinctive appearance. Boot colour, sleeves, tape, hair, build. Useful within a match, unreliable across matches.
5. Elimination. Nine of ten players are accounted for, so the tenth is known.

Jersey number sits at the top and is frequently missing. SoccerNet's Game State Reconstruction dataset handles that with a tracklet-level rule: if the number is visible in at least one frame, the whole tracklet carries it, and if it is never visible the attribute is null (Somers et al., CVPRW 2024). The authors also note that role, team and jersey number together cannot always identify an athlete uniquely, for instance when two teammates have no visible number, which is why they keep a separate track ID alongside the attributes.

That is a useful pattern to copy. Keep two layers: a track ID that says "these boxes are one person", and identity attributes that say "this person is number 23 for the home team". The first can be labeled with confidence much more often than the second.

Jersey numbers are not the only possible anchor. TrackID3x3, a 3x3 basketball dataset, identifies players in its indoor footage by team and by their position at the start of the video (Yamada et al., 2025). In American football, helmets hide faces and hair, which pushes almost all the weight onto number and position, as we cover in our guide to NFL computer vision.

Rule 5: What does an annotator do when they cannot tell?

Give them a way to say so. A spec with no "unknown" state forces a guess, and the guess is recorded with the same confidence as every other label.

There are three situations to cover:

• After a pile-up. Two or more teammates enter a ruck, a scrum, a goalmouth scramble or a screen, and it is not clear who came out where. The spec should allow the annotator to end the uncertain tracks and start new ones flagged for review, instead of picking.
• After a cut. A close-up shows a player from behind with no number visible. Mark the track as unlinked until evidence appears, then link backwards.
• Never resolved. Some players stay unidentified for a whole clip. GSR's null jersey value is the published example of accepting that.

This is a judgement call about what your model can tolerate. In our view a missing link costs you some training signal, while a wrong link teaches the model something false, so the spec should make "unknown" the cheaper choice for the annotator.

Backward linking deserves a line of its own in the spec. Identity evidence often arrives late: the number becomes readable ten seconds after the cut. Annotators need permission, and tooling, to go back and attach the earlier boxes to the right player.

How should boxes be drawn through occlusion?

Decide whether an occluded player's box covers the visible part or the estimated full body, because that choice changes what a re-ID model sees in its crops.

MOT16 instructs annotators to estimate the full extent of a partially occluded person from cues such as expected size and neighbouring frames (Milan et al., 2016). SportsMOT asks annotators to predict the box under occlusion as long as some part of the body is visible, and to skip athletes with half of the torso outside the view (Cui et al., ICCV 2023). It also specifies that the box covers the athlete's limbs and torso and excludes objects such as a ball touching the body.

Full-extent boxes keep track geometry stable, which helps trackers. They also mean a re-ID crop of an occluded player contains a large share of someone else. If re-ID is the main use, consider labeling a visibility flag per box so heavily occluded crops can be filtered out of training. The broader trade-off between label types is in skeletal tracking versus bounding boxes.

Which special cases belong in the spec?

Any event that changes who is on the field or what they look like. List them by sport before labeling starts.

SoccerNet's GSR annotations show one way to handle roles: every athlete is a player, goalkeeper, referee or other, and only players and goalkeepers carry a team (Somers et al., CVPRW 2024). Role and event definitions fit naturally into the wider label schema, which we cover in sports content taxonomy and event schema design.

How do you check identity labels?

Review along the time axis. Follow one ID from its first frame to its last and confirm it is one person throughout, then repeat for the next ID.

Four checks follow directly from the rules above:

• Cut audit. Sample shot boundaries and verify every identity link made across them.
• Collision audit. Sample moments where same-team boxes overlap heavily and verify IDs before and after.
• Roster consistency. No two simultaneous tracks share a team and jersey number, and no jersey number appears that is not on the team sheet.
• Double labeling. Have two annotators label identity on the same clips independently and compare their links, concentrating on cuts and collisions.

A spot check of random frames passes all of these by accident, since each frame looks fine alone. Our guide to measuring annotation quality covers the ID-persistence audit in more detail, and how long an annotation vendor takes to become productive explains how to gate annotators on these rules before production.

The identity spec checklist

Agree it with whoever will train the model, freeze it, and version it. A rule changed mid-project means earlier labels follow a different convention from later ones. For where this sits in the wider process, see how to build a sports computer vision training dataset.

Frequently asked questions

What is the difference between a track ID and a player identity?

A track ID says a set of boxes belongs to one person within some scope. A player identity says who that person is, usually through team and jersey number. SoccerNet's GSR dataset keeps both, because attributes alone cannot always separate two athletes.

Should a player get a new ID after leaving the frame?

It depends on the convention you choose. MOT16 assigns a new ID. SoccerNet-Tracking keeps the same ID within a clip. For team sport products the same-ID rule is usually the useful one, provided the annotator has evidence for the link.

Do public tracking datasets include camera cuts?

Two widely used sports tracking benchmarks do not. SportsMOT clips were checked to exclude abrupt viewpoint switches, and SoccerNet-Tracking uses main-camera sequences. SoccerNet ReID does link players between a live action frame and its replays, but only within one action.

How do you label a player whose jersey number is never visible?

Keep the track ID and leave the jersey attribute empty. SoccerNet GSR assigns a null value in that case, and assigns the number to the whole tracklet when it is visible in at least one frame.

Identity rules are a design decision

Every re-ID dataset encodes a set of choices about scope, re-entry, cuts, evidence and uncertainty. Public benchmarks made theirs to suit a research task, and most of them stopped short of full broadcast conditions. A production dataset has to make the same choices deliberately and write them down before labeling starts, because they cannot be recovered from the labels afterwards.

If you need identity-consistent labels on your own footage, see Train Matricx pricing and the free pilot or review client results in our case studies. The pilot is a practical way to test an identity spec on your hardest clip before committing to volume.

Dataset rules and figures were checked against each paper or repository in October 2026. Recommendations are our own. Spot an error? Email info@trainmatricx.com.