Mock judging panel

[Nuitsuki]

21 minutes ago, robin said:

 

Do you want to be a side-judge as well?

Count me in!

[robin]

Oh god... I tried to judge an SP for a test run and it took me so long (I replayed each element at least 3-5 times)

 

but it's fun

 

eta: I've done Yuzu's SP just now and I swear I'm not biased, these boxes were checking themselves LOL

[yuzupon]

1 hour ago, Neenah said:

Checking points is useful for us here because we want this to be a discussion and a learning experience but for the judges doing it on the spot it really is not possible or realistic. The GOE are subjective by nature and that is not a bad thing per say but it can be misused easily. The only way to solve this problem is to give the judges time to rewatch everything they want and to have a second judge review the scores (which is what we do with students marks). The ISU is trying to that by having 9 judges who are scoring the same thing and taking the mean but the fact remains that the judges have to make quick decisions without consulting the rules every time, so they rely on their own interpretation of the bullet points. 

 

I will check the app when I have time and I think it would also be useful to us to discuss how we interpret the bullet points and weigh them against each other some time in the future

 

 

3

Reading your reply kinda makes me think that I might have misunderstood the purpose of this mock judging?

I think it is already established that making judges check bullet points in real competitions takes time and is still highly inefficient (technology still not permitting, etc); I wasn't disputing that.

 

My understanding of why we're doing this is that we want to see how "a relatively ideal" judging, where time is of no relevance and we can be as close to the guidelines by ISU, would measure up against the actual judging. And then, perhaps we can see truly just how viable this "relatively ideal judging (plus minus fans' effort to be unbiased)" is, how much different the results would be, and kinda take that into informing us on how we can view figure skating judging in the future.

--> That, as I tend to make everything to be, the "we are taking this seriously" view on this mock judging.

 

We can also just do it absolutely for fun, in which case, everything goes, and it absolutely doesn't mean this is what we demand judges do in real competition.

...

no?

 

13 minutes ago, robin said:

Oh god... I tried to judge an SP for a test run and it took me so long

I only did 1 skater's SP, you know who, and it took me 20ish minute.

[Nuitsuki]

I think I'll  need to watch one program at least twice to give out the score. 1st time to decide GOE and 2nd for PCS 

 

[kaeryth]

A suggestion: How about doing half and half? Maybe judge the LP in real time like the judges do and with the SP you guys can take your time...

[Xen]

Wow, as TP I might end up going faster. o_O

[robin]

@Raomina Is it possible to see the scores of individual skates before I am done scoring the whole event? And is it possible to see protocol tables?

[Neenah]

1 hour ago, yuzupon said:

Reading your reply kinda makes me think that I might have misunderstood the purpose of this mock judging?

I think it is already established that making judges check bullet points in real competitions takes time and is still highly inefficient (technology still not permitting, etc); I wasn't disputing that.

 

My understanding of why we're doing this is that we want to see how "a relatively ideal" judging, where time is of no relevance and we can be as close to the guidelines by ISU, would measure up against the actual judging. And then, perhaps we can see truly just how viable this "relatively ideal judging (plus minus fans' effort to be unbiased)" is, how much different the results would be, and kinda take that into informing us on how we can view figure skating judging in the future.

--> That, as I tend to make everything to be, the "we are taking this seriously" view on this mock judging.

 

We can also just do it absolutely for fun, in which case, everything goes, and it absolutely doesn't mean this is what we demand judges do in real competition.

...

no?

 

I believe we are both thinking the same thing but coming at it from different directions.  I was just trying to say that it is a useful exercise as long as we remember not to judge the judges or base any conclusion on this.

 

Sorry if I caused confusion, I sometimes have problem expressing myself clearly in English. 

 

[Raomina]

2 hours ago, robin said:

@Raomina Is it possible to see the scores of individual skates before I am done scoring the whole event? And is it possible to see protocol tables?

Technically, you could 'finalize' before you finish and go to the end to see it and then click the back button on top (or click on skater names on the results page) to go back do the others since everything is saved anyway, but the idea was that you could not see the scores before the final rankings so that you do not try to adjust your scores based on what the total is and just let things end up where they are based on how you judge each element. 

 

About protocols,  you can see the final rankings as a table similar to that of the ISU results by clicking on view as table in the results page but for detailed individual protocols, I've thought about it but I'm not sure how they should be implemented. Should it be a print out of the checkboxes? As a checkbox table? Just numbers? Should it be optimized for mobile viewing or should a file be generated? 

 

Also, how should I integrate scoring from multiple people? Maybe I could try to create a page where someone can upload everyone's files after they export to 'add' each judge's scores to be averaged. 

 

I'm glad people are finding it usable! It's my first time coding an app since I was just learning so i know it's not the most optimized implementation. 

 

Just a note that it's a single page html app so DO NOT PRESS REFRESH on the browser. Press the back buttons on top or go back to the main page through the url to restart the navigation if you did press refresh. 

 

[Raomina]

Also, I initially implemented the GOEs as numericals so that you could score more like the judges but based on some initial feedback I changed it to checkboxes since the events are not live anyway, but you can tell how much longer it would take if judges had to check individual boxes during the event, which was what some people proposed elsewhere that I thought was not a practical option.  

 

And at some point, I think you will know roughly what the boxes are and how many GOEs each element is worth more instinctively, so it's a good learning experience. 

[robin]

1 minute ago, Raomina said:

Technically, you could 'finalize' before you finish and go to the end to see it and then click the back button on top (or click on skater names on the results page) to go back do the others since everything is saved anyway, but the idea was that you could not see the scores before the final rankings so that you do not try to adjust your scores based on what the total is and just let things end up where they are based on how you judge each element. 

 

About protocols,  you can see the final rankings as a table similar to that of the ISU results by clicking on view as table in the results page but for detailed individual protocols, I've thought about it but I'm not sure how they should be implemented. Should it be a print out of the checkboxes? As a checkbox table? Just numbers? Should it be optimized for mobile viewing or should a file be generated? 

 

Also, how should I integrate scoring from multiple people? Maybe I could try to create a page where someone can upload everyone's files after they export to 'add' each judge's scores to be averaged. 

 

I'm glad people are finding it usable! It's my first time coding an app since I was just learning so i know it's not the most optimized implementation. 

 

Justa note that it's a single page html app so DO NOT PRESS REFRESH on the browser. Press the back buttons on top or go back to the main page through the url to restart the navigation if you did press refresh. 

 

 

It's very easy to use and practical! Thank you (And I appreciate the checkboxes for bullets)

 

Hm what would be really nice is having a judges scores table for each skater but with only one column (bc there's just one judge). Is that too much work?

 

it would be even nicer if after everyone's results are uploaded a full judges scores table were created for all the skaters (how hard would it be to generate pdf files?)

[Raomina]

23 minutes ago, robin said:

 

It's very easy to use and practical! Thank you (And I appreciate the checkboxes for bullets)

 

Hm what would be really nice is having a judges scores table for each skater but with only one column (bc there's just one judge). Is that too much work?

 

it would be even nicer if after everyone's results are uploaded a full judges scores table were created for all the skaters (how hard would it be to generate pdf files?)

 

Ok, I'd have to look it up and learn how to do that but I'm sure it would be doable! About single column for each skater, I'd have to think about how to present that and what you should click to lead to that. Scrolling list for all skaters or individual pages etc. 

[Murieleirum]

Sorry if I come in late my friends :3' I would love to be side-judge, if there's still the possibility... I'm confident with GOEs regarding jumps and spins, not so confident regarding Step Sequences and levels on step sequences and spins (levels are called by the TP, right? But I should be able to understand it anyways), but I am a fast learner and I was waiting for something to motivate me to study even more the technique behind figure skating c: 

As for PCS, I think I'm slowly starting to pay attention to knees and edges more, although I'd be a little tentative in judging SS as a whole; TR should be a little easier because it's a 'it's there or it isn't there' kind of thing; and with CO and PE, and especially in IN I might end up being a little devil. (If the Interpretation is flat from start to finish, for example, and there isn't a journey to go along with musical crescendo-decrescendo and such, you're not gonna get pretty numbers as far as I'm concerned).

 

Please let me join and I'll try and save up time whenever you guys decide!!  

[robin]

7 minutes ago, Murieleirum said:

Sorry if I come in late my friends :3' I would love to be side-judge, if there's still the possibility... I'm confident with GOEs regarding jumps and spins, not so confident regarding Step Sequences and levels on step sequences and spins (levels are called by the TP, right? But I should be able to understand it anyways), but I am a fast learner and I was waiting for something to motivate me to study even more the technique behind figure skating c: 

As for PCS, I think I'm slowly starting to pay attention to knees and edges more, although I'd be a little tentative in judging SS as a whole; TR should be a little easier because it's a 'it's there or it isn't there' kind of thing; and with CO and PE, and especially in IN I might end up being a little devil. (If the Interpretation is flat from start to finish, for example, and there isn't a journey to go along with musical crescendo-decrescendo and such, you're not gonna get pretty numbers as far as I'm concerned).

 

Please let me join and I'll try and save up time whenever you guys decide!!  

 

You are very very welcome to join

 

I have to say, after doing the test round today, what's most difficult is judgung StSqs and PCS

 

16 minutes ago, Raomina said:

 

Ok, I'd have to look it up and learn how to do that but I'm sure it would be doable! About single column for each skater, I'd have to think about how to present that and what you should click to lead to that. Scrolling list for all skaters or individual pages etc. 

 

I think it's also enough as long as it's possible to see the uncurved GOEs in the tables page eg. like this:

 

Yuzuru Hanyu Elements	4Lo	4S+3T	FCSp4	3A	...
GOE	3	3	3	3	...
Patrick Chan Elements	4T+3T	...
GOE	...

 

 

[Xen]

18 minutes ago, Murieleirum said:

Sorry if I come in late my friends :3' I would love to be side-judge, if there's still the possibility... I'm confident with GOEs regarding jumps and spins, not so confident regarding Step Sequences and levels on step sequences and spins (levels are called by the TP, right? But I should be able to understand it anyways), but I am a fast learner and I was waiting for something to motivate me to study even more the technique behind figure skating c: 

As for PCS, I think I'm slowly starting to pay attention to knees and edges more, although I'd be a little tentative in judging SS as a whole; TR should be a little easier because it's a 'it's there or it isn't there' kind of thing; and with CO and PE, and especially in IN I might end up being a little devil. (If the Interpretation is flat from start to finish, for example, and there isn't a journey to go along with musical crescendo-decrescendo and such, you're not gonna get pretty numbers as far as I'm concerned).

 

Please let me join and I'll try and save up time whenever you guys decide!!  

Hmm....SS. Well-go with the standard that figure skating should look "easy". This easy is defined by aspects such as does the skater look balanced (ease of movement), does this skater have good speed and coverage of ice, does this skater seem to gain speed out of nowhere. Then you figure in stuff such as whether the skater used harder movements, such as rockers/counters/choctaws as part of their linking movements instead of normal 2 foot skating and gliding, and whether the skating still looks smooth and "easy" despite the difficulty levels thrown in. 

 

There is a reason why difficulty of TR movements and the quality of the TR movements can be somewhat correlated with SS, and why people joked about quality crossovers. The harder TR movements usually are 1-foot skating, with complex turns that require you to go counter-intuitive on your balance. You should be able to gain speed off of turns and deep edges alone due to knee and ankle action, but to do it consistently, maintain speed/gain speed, and hold your balance is hard.  And now you know why I tend to get pissed when JPN skaters get worse SS scores.