Dark Horse AI: 6,000 Youth Football Matches Annotated

Executive Summary: Dark Horse AI builds automated highlight generation for youth football, which means its product stands or falls on one thing: attaching the right action to the right player. Generic annotation vendors had left it with broken player identities and a growing backlog of unprocessed footage. Train Matricx onboarded a dedicated team of 30+ sports-domain annotators inside a week, trained them on the client's own 30+ class taxonomy, and ran every match through a three-layer QA architecture. The backlog cleared in 15 days. More than 600 matches were delivered in the first 60 days, and the engagement has since passed 6,000 matches, at 98% accepted first pass on a 24-hour turnaround per match.

Case study metrics

---

Why youth football footage is harder than broadcast

Most sports AI case studies involve broadcast footage: multiple camera angles, close-ups, and a production crew following the ball. Dark Horse AI's footage is the opposite. Youth matches are recorded on Veo cameras, a single fixed unit filming the whole pitch at once.

That one difference changes every annotation decision.

Players are small in frame. With the whole pitch in view, a player occupies a fraction of the image. Shirt numbers are frequently unreadable, so an annotator cannot confirm identity by reading the back of a shirt the way they can on broadcast footage.

There are no camera cuts, but there is constant occlusion. A fixed camera never cuts away, which helps. It also never moves closer, so players overlap at distance for long stretches, and two teammates in identical kit can merge and separate several times in a single passage of play.

There is no second view. A broadcast production offers another angle when the main one is blocked. Here, if the action is obscured, there is no alternative footage to resolve it from. The annotator has to carry identity through the occlusion using context.

This is why identity, not box tightness, is the hard part of this project. The same problem shows up across the field in our guide to broadcast player tracking and re-identification, where re-identification exists precisely because trackers lose players and have to reconnect them.

---

The bottleneck: what generic annotation had produced

When Dark Horse AI engaged Train Matricx, the pipeline was stalled by three connected problems.

Catastrophic ID switching. Previous crowd-sourced vendors produced labels where a player's ID transferred to a different player after an overlap. For a highlight engine this is not a small error. A highlight is worthless, or worse than worthless, if it is filed under the wrong child's name. A parent watching their kid's reel notices immediately.

Taxonomy applied without understanding. The client's schema ran to more than 30 action classes, which demands football judgement rather than pattern spotting. Distinguishing a key pass from an ordinary touch is a decision about intent and consequence. Annotators without the domain knowledge applied those labels inconsistently, so the model learned the inconsistency.

A growing backlog. Thousands of hours of raw footage accumulated faster than it could be processed, which stopped model iteration entirely. Every week of delay pushed the product roadmap back by a week.

---

What we built

Onboarding in a week, on the client's taxonomy

We recruited, onboarded and trained a dedicated team of 30+ annotators within one week of kickoff, and trained them specifically on Dark Horse AI's 30+ class taxonomy rather than on generic football labelling. Annotators worked through the class definitions and the ambiguous cases before touching client footage.

Freezing the schema before the volume arrived is what made the turnaround possible later. Renegotiating class definitions mid-project invalidates everything labelled before the change.

A three-layer QA architecture

Accuracy comes from the steps after the first tag. Layers 2 and 3 are where identity errors surface.

Each layer looks for a different class of error.

Layer 1, primary annotator. Sport-domain annotators apply the multi-tag labels and player IDs, working to the frozen class definitions.

Layer 2, specialist reviewer. Senior annotators cross-check the complex multi-tag events and, critically, verify that player IDs survive occlusions. This is the layer that catches identity swaps, because a swap is invisible in any single frame and only appears when you follow a player across a passage of play.

Layer 3, senior auditor. Technical QA leads act as final gatekeepers, checking that the delivered data matches the client's schema exactly, so nothing reaches the pipeline in a shape the model cannot ingest.

The reason for three layers rather than two is that these are genuinely different jobs. Spotting a mislabelled event, tracing an identity through a scramble and validating schema conformance require different attention, and asking one reviewer to do all three reliably at volume does not work. Our guide to measuring annotation quality covers the audits behind this, including the ID-persistence checks that a frame-by-frame spot check will never surface.

A 24-hour SLA that held

Every newly uploaded match was annotated, reviewed through all three layers and delivered within 24 hours. A dedicated project manager owned workflow distribution, so a match that arrived overnight was in the queue before the working day started.

A turnaround like that is an operations result rather than a labelling one. It depends on the schema being settled, the reviewers being available rather than batched weekly, and first-pass acceptance being high enough that rework does not consume the next day's capacity. The arithmetic behind that last point is in our guide to annotation capacity planning.

---

Results

• Backlog cleared in 15 days. The stalled queue was processed and production returned to schedule.
• 600+ matches in the first 60 days, and more than 6,000 to date. Multi-tag classification and player ID tracking across the full volume, with the engagement still running.
• 98% accepted first pass. Deliveries passed the client's review without rework at that rate, sustained under a 24-hour SLA rather than measured on a sample.
• The partnership continues. Dark Horse AI remains an ongoing client, with matches still being annotated.

The measure that matters here is first-pass acceptance rather than a raw accuracy figure, because it is the one the client experiences. It answers a concrete question: of what we delivered, how much came back for rework?

---

Quick answers

What was the main challenge Dark Horse AI faced with generic data annotation? Crowd-sourced vendors lacked the football knowledge for a 30+ class taxonomy, which produced inconsistent event labels and, more damagingly, broken player identity after occlusions. A growing backlog of raw footage had stopped model iteration.

Why is player identity so critical for a highlight product? Because the product attributes clips to individual players. If an identity switches during an overlap, the highlight is filed under the wrong player, which users notice immediately. Position accuracy can be slightly loose without anyone noticing; identity cannot.

How did Train Matricx fix the ID tracking and taxonomy issues? By onboarding 30+ domain-expert annotators within a week, training them on the client's own taxonomy before they touched client footage, and running every match through three review layers, with the second layer specifically verifying identity across occlusions.

What does "98% accepted first pass" mean? It is the share of deliveries that passed the client's review without needing rework. It is a delivery measure rather than a claim about accuracy against an unnamed ground truth, which is why we report it.

Why is Veo footage harder to annotate than broadcast? One fixed camera covers the whole pitch, so players are small in frame, shirt numbers are often unreadable, and there is no second angle to resolve an occlusion. Identity has to be carried through contact using context rather than confirmed by reading a number.

Can this work at a smaller scale? Yes. The same schema-first, multi-layer QA process runs on a single clip. We annotate a free pilot so quality is measurable on your footage before any volume is agreed.

---

Are backlogs or broken identity data holding up your computer vision models? Train Matricx builds dedicated, human-in-the-loop annotation teams for sports tech. See pricing and the free pilot, or read how we handle football match event tagging.