Technology & AI in Gymnastics judging: How FS judging could improve

[Salior]

Some excerpts:

Morinari Watanabe, the president of the International Gymnastics Federation, says this signals “the beginning of the new history of gymnastics.” The debut of such technology represented a meaningful milestone in a sport periodically marred by judging controversies, and often wracked with questions about political influence in scoring decisions.

Each of the wifi-router sized boxes, designed by the Japanese company Fujitsu, contained a set of three-dimensional laser sensors that tracked the movements of each of gymnast. That data was fed to an artificial intelligence system, accessible to the human judges, that measured and analyzed skeletal positions, speeds and angles — some of them unavailable or simply missed by the judges — as the athletes went through their movements.

At the world championships, the artificial intelligence system has served a supporting role, available to judges to confirm difficulty scores in two circumstances: in the event of inquiries (when gymnasts formally challenge the judges’ score) and blocked scores (when there is a large deviation between the sets of judges).

Thanks to all this, Watanabe explained, no longer would gymnasts — many of whom, he noted, had started gymnastics as young as age 3 and had trained competitively for more than a decade — risk having their efforts unceremoniously wasted by human error or interference.

“This is a step toward the challenge of justice through technology,” Watanabe said.

Another article explaining how the technology works, and how it is used.

Leveraging AI and 3D sensing technologies, Fujitsu’s Judging Support System conducts real-time movement capture, analyses and scoring. Human judges sometimes struggle to score complicated and quick movements using the naked eye, and it’s hoped the system can overcome these and other human limitations so that evaluation will become more fair and accurate for all athletes.

Q1. What type of sensors are used for data capture?

A1. Lidar sensors. Fujitsu’s MEMS solid-state Lidar has over 2 million pulses per second emission and a 15-meter detection range. Although such sensors are quite costly, Fujitsu predicts the system will bring in more than JP¥100 billion (~US$1 billion) profit over the next 10 years.

Q4. How does the judging system work? Any use of deep learning?

A4. The joint recognition module uses deep learning technology. The neural network model receives several multi-viewpoint depth images as input and outputs corresponding 3D joint position results.

Fujitsu worked closely with the Japanese Gymnastics Association to build up the database with a set of elements for each skill difficulty. The project team collected over 800 elements for male gymnasts and more than 500 elements for female gymnasts — where the elements comprise a series of basic skills from top gymnasts.

Q5. Does the system support Difficulty (D) or Execution (E) scoring?

A5. Both. The final score of each exercise is established by the summation of the D-Score and E-Score. Each time a gymnast completes an element with a certain skill level, he or she will be rewarded using a corresponding difficulty value (DV). Starting with a perfect 10.00 points, the E-score is lowered by 0.10 points for small faults, 0.30 points for medium faults, 0.50 points for large faults, and 1.00 point for very large faults.

Q7. Who will benefit from the system?

A7. Fujitsu says that in addition to supporting judges in making decisions, the system can also assist athletes with training, and can help the audience better understand and enjoy gymnastics.

For example, the system can accurately inform athletes regarding their stability and the exact angle between their joints, so they can make appropriate adjustments and improvements. The system can also append quantitative indicators such as height and stability to competition streaming services, to guide the audience to a better appreciation of the sport.

Third article about this, they've been working on it since 2017

 

Some discussion on reddit

 

Some issues brought up by Duhamel

 

Some possible solutions from other sports

Split tech / PCS panel for judging, More technical commentaries with statistics and analysis like the JPN TV? 

 

Some fan initiatives 

https://twitter.com/skatingscores, iirc there's a Japanese version ran by another fan but I can't find it