Crowd sourced Technical Camera project

[shanshani]

I recall some people were complaining about uncalled URs in the men's short at ACI. Does anyone have video of the specific jumps? Kind of don't want to go combing through the competition thread

[yuzuangel]

Do you think it's too much to ask if we ask people to submit fancams in (what resolution is the minimum to see takeoff and landing?) FPS of jumps ONLY (not entire videos)? Or maybe we could ask mechanical turk to cut out the videos of jumps? That way what we can do is:

 

1. get snapshots of each takeoff and landing

2. blur out the skater's face/costume

3. crowdsource whether or not it's UR

4. compare that to ISU calls

5. use mechanical turk to count number of frames to get estimate of height of jump 

6. compare that against GOE

7. maybe build a neural net over calls too see if it can predict country 

[katonice]

I've started to look through some of my files. Here's one I can contribute.

 

Loena Hendricx

GP Helsinki 2018

Short Program 

3Lz-3T

 

 

[wingman]

Great everyone - thanks for initiating this @shanshani!

I really think that a crowd-funded project is possible. And just generally looking down the road, if we want to turn this into a serious analysis that can shame the ISU and bring about REAL changes, there are a few things we should consider now:

 

Data collection

1. Where to store these videos

2. Copyrights. (Any lawyers on the planet? Who own these videos? What about events that say "no fan cams allowed"?)

And what about broadcasted videos? (Do you think there's a chance that we can get permission to use them? Would be happy to write and ask -- I suspect that short clips from those are OK but not the whole performance)

 

3. Splicing, tagging and sorting: ideally each video is split into multiple clips, each clip just have 1 second before the jump, the jump, and the second after (say).

Then we need to sort the video by *jumps*, with naming convention like jumptype_skater_program_competition, so we will get

4Lo_Hanyu_SP_ACI2019

for example

and within that folder we have multiple clips of the same jump. 

 

I think step 3 should be done by the video submitter, to minimize our burden. 

 

After we collect a giant database, we can think of the second step: methodologies (I still lean towards crowd-funding a Kaggle -- I think it gives great visibility to the problem as well)

 

[YesWay]

I like this... I think it's a very good idea to gather high-quality hard evidence, of ISU breaking its own rules - in an organised/coordinated way.

 

Splurging such evidence everywhere possible, could then shame them into reform.

Or maybe pressure the Olympics into launching an official inquiry, etc.

(I think they are possibly the only organisation that might have some kind of influence over the ISU)

 

Otherwise, ISU gets to dismiss all complaints as just crazy/ignorant fans having a rant because they don't like the scores. (For the same reason, I'm am slightly concerned at ideas I've seen elsewhere, to have banners/protests at actual competitions. Personally I think that plays into ISU's hands, and might also distract or intimidate the skaters - the very people we want to protect?)

 

Anyway, good luck!

 

 

[yuzuangel]

Crowdfunding a computer vision Kaggle sounds like fun  

 

I do think it's a bit of a hard problem to solve with computer vision because knowing when a blade touches the ice is not easy even for a human sometimes. The ice is white and there's usually not much shadow that would let you tell if a boot is on the ice or not. I'm just trying to think -- what features would we even be able to use? Biggest one I can think of is the angle of the blade but ;; it depends on the camera angle as well. Well, I guess that's the beauty of Kaggle and getting other people to think about the problem.

[shanshani]

8 hours ago, yuzuangel said:

Crowdfunding a computer vision Kaggle sounds like fun  

 

I do think it's a bit of a hard problem to solve with computer vision because knowing when a blade touches the ice is not easy even for a human sometimes. The ice is white and there's usually not much shadow that would let you tell if a boot is on the ice or not. I'm just trying to think -- what features would we even be able to use? Biggest one I can think of is the angle of the blade but ;; it depends on the camera angle as well. Well, I guess that's the beauty of Kaggle and getting other people to think about the problem.

With 30fps or other low frame rates it's hard to tell, but I've found when looking at 240 fps it isn't actually that hard to determine the frame where it happens. Multiple angles helps too

[yuzuangel]

If/when we get enough "smoking gun" data, do you think we could take this to a journalist or some online news site? I think NBC did a piece about biased judges once (lol), but I wonder if something like Vice would take on such a story. Or any other popular online magazine.

[shanshani]

On 9/20/2019 at 11:32 PM, yuzuangel said:

If/when we get enough "smoking gun" data, do you think we could take this to a journalist or some online news site? I think NBC did a piece about biased judges once (lol), but I wonder if something like Vice would take on such a story. Or any other popular online magazine.

I think we have to be careful about how we characterize such footage. We don't know what the cause of a bad tech call could be--bad angles, judges skipping reviewing the element, something more nefarious, etc. But sending it to figure skating media would be helpful, yes, as well as tweeting it at the ISU with the relevant hashtags. Though I don't think it would be enough interest for general sports media to bite, not when it isn't Olympics season.

[wingman]

On 9/19/2019 at 3:25 PM, yuzuangel said:

Crowdfunding a computer vision Kaggle sounds like fun  

 

I do think it's a bit of a hard problem to solve with computer vision because knowing when a blade touches the ice is not easy even for a human sometimes. The ice is white and there's usually not much shadow that would let you tell if a boot is on the ice or not. I'm just trying to think -- what features would we even be able to use? Biggest one I can think of is the angle of the blade but ;; it depends on the camera angle as well. Well, I guess that's the beauty of Kaggle and getting other people to think about the problem.

 

I thought about this a bit -- don't they use computer vision to track and predict ball trajectories in baseball and cricket quite a bit? Granted t that the ball has well-defined boundaries, but I think it's not a trivial problem in baseball at all, and they have managed for ages now. 

 

Another option is to put a small tracking device on the skaters' boots. Two chips for each boot is enough (one in front and one below) -- and this way we don't need to deal with cameras *at all* They have chips in soccer balls, for example, so clearly the technology is there. Obviously it won't get adapted at competitions immediately, but I think that it can get taken up by amateurs, training, etc, and eventually make it to official competitions? In the long run, this sort of tracking beats cameras + algorithms *for sure*. 

 

We really need to think in the long run and have an indisputable technology. (Getting the ISU to adopt it may take as long as getting FIFA to adopt their goal line technology, but it's totally worth it)

 

Btw, athletes might want this to dispute ISU's phantom call, for example. There's totally a market for it.

 

 

 

[yuzuangel]

12 minutes ago, wingman said:

Another option is to put a small tracking device on the skaters' boots. Two chips for each boot is enough (one in front and one below) -- and this way we don't need to deal with cameras *at all* They have chips in soccer balls, for example, so clearly the technology is there. Obviously it won't get adapted at competitions immediately, but I think that it can get taken up by amateurs, training, etc, and eventually make it to official competitions? In the long run, this sort of tracking beats cameras + algorithms *for sure*. 

I have been thinking about tracking devices for AGES. But I don't have much electrical engineering or mechanical engineering background. Do you know anyone who knows anything about devices that can measure when a boot leaves and meets the ice? I'm guessing the g force on a body in the air is different from a body on the ground right? But maybe the takeoff curve is smooth enough that you can't tell exactly when the liftoff/landing happens. Does anyone have an engineering background and can help? I would totally love to help design and prototype something like this. I know a lot of skaters who can do doubles and triples that we can test on. We can even give these tools to skaters learning their jumps to know if their jumps are UR or not.

[shanshani]

On 9/21/2019 at 10:15 PM, sallycinnamon said:

 

 

 

hm, Samarin's 4Lz is somewhat PR, looks to be around the 120-150 degrees mark, but it's not as bad as we've seen from others. He does clearly land all the way around, so I'll give him that.

 

Rizzo though you're allowed a bit more PR on loops, but even then that was a bit much. Plus it's clearly under. Did it get called?

[Paskud]

50 minutes ago, shanshani said:

hm, Samarin's 4Lz is somewhat PR, looks to be around the 120-150 degrees mark, but it's not as bad as we've seen from others. He does clearly land all the way around, so I'll give him that.

 

Rizzo though you're allowed a bit more PR on loops, but even then that was a bit much. Plus it's clearly under. Did it get called?

Matteo 4lo? Yes, was called UR. Tho for me it should be downgrade.

[yuzuangel]

3 hours ago, shanshani said:

Rizzo though you're allowed a bit more PR on loops, but even then that was a bit much. Plus it's clearly under. Did it get called?

I keep watching the clip and it looks like he takes off forward to me, so I've been very confused at the criticism. How much PR do you see?

 

Takeoff:

 

Landing curve (yes it was UR so he has been on the ice for a while but this is where he fully checks his rotations)

 

In general I feel like it's not very easy to excessively PR a loop jump. Also skaters often change the curve of their blade on the ice in order to jump (i.e. for the loop you want to get really deep on your outside edge). Unlike toe jumps where the PR is very obvious on the pick, I think for edge jumps you can only compare the PR to the trajectory of the skater throughout the jump which is harder to determine.

[shanshani]

29 minutes ago, yuzuangel said:

I keep watching the clip and it looks like he takes off forward to me, so I've been very confused at the criticism. How much PR do you see?

 

Takeoff:

 

Landing curve (yes it was UR so he has been on the ice for a while but this is where he fully checks his rotations)

 

In general I feel like it's not very easy to excessively PR a loop jump. Also skaters often change the curve of their blade on the ice in order to jump (i.e. for the loop you want to get really deep on your outside edge). Unlike toe jumps where the PR is very obvious on the pick, I think for edge jumps you can only compare the PR to the trajectory of the skater throughout the jump which is harder to determine.

I'm just counting where the pick leaves the ice, which was over 180 from the point where he started the takeoff/the starting point implied by the trajectory of the jump. I guess you could argue that loop jumps are allowed 180 PR, and Matteo wasn't that far over 180, so maybe it's not as bad of a case as we see routinely on lutzes these days.