Logo image
Computer Vision for Horse Racing Analysis and Adjudication
Conference paper   Open access   Peer reviewed   PUBLISHED

Computer Vision for Horse Racing Analysis and Adjudication

Anthony Bedford, Erica Mealy, Abigail Koay, Utkarsh Kumar and Shahrzad Saremi
Proceedings of the 18th Australasian Conference on Mathematics and Computers in Sport, pp.173-187
Australasian Conference on Mathematics and Computers in Sport (ANZIAM): MathSport, 18th (Gold Coast, Australia, 01-Jul-2026–03-Jul-2026)
ANZIAM Mathsport
2026
pdf
Computer Vision for Horse Racing Analysis and Adjudication1.94 MBDownloadView
Published Version Open Access
url
Proceedings of the 18th Australasian Conference on Mathematics and Computers in SportView
Published Version Open

Abstract

Computer Vision Horse Racing Multi-Object Tracking Object Detection protest adjudication artificial intelligence
Racing protests are currently adjudicated by stewards reviewing broadcast footage manually, a process that is time-intensive and inherently subjective. This paper presents a computer vision pipeline for automated horse and jockey detection, multi-object tracking, and protest adjudication support in thoroughbred racing, building on prior work (Mealy et al., 2025). We assemble a novel two-class dataset of 7,290 training images using Roboflow SAM3 automated segmentation with full manual annotation correction, a fivefold increase over our previous manually labeled baseline. We then conduct a controlled comparison of YOLOv12x (Tian et al., 2025) and RF-DETR (Robinson et al., 2025) on a held-out test set of 236 images, on which RF-DETR (ep72) achieves mAP50 of 0.920 and mAP50-95 of 0.649, outperforming YOLOv12x across 10 of 12 evaluation metrics despite training at lower resolution. We further compare three modern multi-object trackers and demonstrate that full-race identity consistency remains an unsolved problem, with our best configuration producing 1,109 unique track IDs for a 16- horse race that should yield 16. To address this in practice, we introduce a steward-centred review workflow in which the operator selects the two horses involved in a specific protest and projected proximity analysis is performed only on that pair within a short incident window. We validate this workflow against a published Racing NSW stewards inquiry (Goulburn, 2018), where the system produces a projected overlap signal beginning at t = 27.16 s, consistent with the upheld protest finding, while acknowledging that projected image-plane overlap is not equivalent to verified physical contact.

Details

Metrics

1 File views/ downloads
1 Record Views
Logo image