General Yuzuru Chat

[shanshani]

11 minutes ago, Salior said:

 

Try this website? http://skatingscores.com They have a lot of entries 

 

You can contact the person on twitter too, they'll probably be happy to share their data https://twitter.com/SkatingScores 

that's where I'm getting my data from, but I need it in an excel/google spreadsheets format

[rockstaryuzu]

9 minutes ago, shanshani said:

that's where I'm getting my data from, but I need it in an excel/google spreadsheets format

If you know coding, you could probably write a script to parse the data and input it for you. Might be faster. Unfortunately I am not a coder or I would be happy to help. 

 

Also, I think this project is an interesting idea, and I would like to remind you that correlation does not prove causation, when it comes to data. You might think you have a smoking gun, however, try and confirm it by a second method before coming to definitive conclusions. said the scientist

[yuzuangel]

17 minutes ago, rockstaryuzu said:

If you know coding, you could probably write a script to parse the data and input it for you. Might be faster. Unfortunately I am not a coder or I would be happy to help. 

 

Also, I think this project is an interesting idea, and I would like to remind you that correlation does not prove causation, when it comes to data. You might think you have a smoking gun, however, try and confirm it by a second method before coming to definitive conclusions. said the scientist

I wrote a script to parse their data at one point and they yelled at me. So be careful

[rockstaryuzu]

1 minute ago, yuzuangel said:

I wrote a script to parse their data at one point and they yelled at me. So be careful

Hmph, I wonder why? It's not going to harm them...anyway I only know scripts can be used, I haven't a clue how to make one. 

[shanshani]

23 minutes ago, rockstaryuzu said:

If you know coding, you could probably write a script to parse the data and input it for you. Might be faster. Unfortunately I am not a coder or I would be happy to help. 

 

Also, I think this is a good idea, and I would like to remind you that correlation does not prove causation, when it comes to data. You might think you have a smoking gun, however, try and confirm it by a second method before coming to definitive conclusions. said the scientist

Alas, I don't know how to code, and moreover I've heard skatingscores has been a bit touchy about having their data scraped in the past. I might be able to get my boyfriend to do it though. "What do you want for Christmas?" "A website scraper that will import figure skating scores into an excel document for me" lol

 

I'm aware that correlation doesn't equal causation, but at the same time there is such thing as inference to the best explanation. If a judge consistently scores skaters of the same nationality higher than skaters of other nationalities, across multiple competitions and skaters, it is reasonable to conclude that they are likely biased in some way in favor of home country skaters, especially given figure skating's history of dodgy judging and the existence of fair judges to compare the "biased" judges to. As I believe I have amply demonstrated in similar past analyses, I am amply able to consider alternative hypotheses and think carefully about results and honestly I find it kind of annoying to be talked to as though I'm undertaking baby's first statistics project, especially since this is like version 3 of the project and I've already dealt with some of these criticisms in previous analyses. I know you're well intentioned and I'm just being touchy, but the amount of hours I have put into this makes it incredibly irritating. I am speaking loosely in these posts, but rest assured the serious analysis will not take the shortcuts in reasoning I feel at liberty to take when I'm just talking casually. I am happy to respond to substantive criticisms when that serious analysis is posted.

[yuzuangel]

4 minutes ago, rockstaryuzu said:

Hmph, I wonder why? It's not going to harm them...anyway I only know scripts can be used, I haven't a clue how to make one. 

Maybe they think they did the work of scraping ISU documents so other people need to do the same too. Well either way, @shanshani I'll try to help (I know I say this a lot BUT this weekend lol) 

What competitions do you have already?

[shanshani]

sorry you go a bit crazy when you've been staring at spreadsheets for hours on end while being immensely jetlagged

[shanshani]

13 minutes ago, yuzuangel said:

Maybe they think they did the work of scraping ISU documents so other people need to do the same too. Well either way, @shanshani I'll try to help (I know I say this a lot BUT this weekend lol) 

What competitions do you have already?

I have all of the grand prix from last season + ACI, Lombardia, and Ondrej Nepala, and ACI and Lombardia from this year. So basically I need to rest of the senior Bs from last year as well as 4CC, Euros, and Worlds. I've changed the format of the competition sheet significantly from the last iteration of this project because I had major problems with how it handled judging from competitions where the the judging panel changed between the SP and FS, not to mention there was simply too much data to input so it was a nightmare from a workload perspective. Plus there were theoretical issues about comparability between different disciplines. So it's revised and simpler this go around.

 

If you're interested, the template is here and a filled out example is here. Basically all you need to do is write down the list of skaters, their nationality code, the judges, their nationality code, and then the score each judge gave each skater (easiest to do this from skatingscores.com). Separate sheet for each segment of the competition.

[rockstaryuzu]

9 minutes ago, shanshani said:

Alas, I don't know how to code, and moreover I've heard skatingscores has been a bit touchy about having their data scraped in the past. I might be able to get my boyfriend to do it though. "What do you want for Christmas?" "A website scraper that will import figure skating scores into an excel document for me" lol

 

I'm aware that correlation doesn't equal causation, but at the same time there is such thing as inference to the best explanation. If a judge consistently scores skaters of the same nationality higher than skaters of other nationalities, across multiple competitions and skaters, it is reasonable to conclude that they are likely biased in some way in favor of home country skaters, especially given figure skating's history of dodgy judging and the existence of fair judges to compare the "biased" judges to. As I believe I have amply demonstrated in similar past analyses, I am amply able to consider alternative hypotheses and think carefully about results and honestly I find it kind of annoying to be talked to as though I'm undertaking baby's first statistics project, especially since this is like version 3 of the project and I've already dealt with some of these criticisms in previous analyses. I know you're well intentioned and I'm just being touchy, but the amount of hours I have put into this makes it incredibly irritating. I am speaking loosely in these posts, but rest assured the serious analysis will not take the shortcuts in reasoning I feel at liberty to take when I'm just talking casually. I am happy to respond to substantive criticisms when that serious analysis is posted.

Right. I didn't know any of that. 

 

Where my criticism is coming from is exactly this: no matter how much effort you put into the data analysis, it's not going to be definitive proof, and if it's presented to some organization like the ISU as definitive proof, the potential is very high that it won't have the desired result and could in fact backfire in unforeseen ways. 

 

So what I'm saying is, be careful. What if your data indicates that a judge is biased and that judge not only denies it, but then turns around and sues you for libel? I'm not saying don't do it. I'm saying, understand the risks. 

[shanshani]

16 minutes ago, rockstaryuzu said:

Right. I didn't know any of that. 

 

Where my criticism is coming from is exactly this: no matter how much effort you put into the data analysis, it's not going to be definitive proof, and if it's presented to some organization like the ISU as definitive proof, the potential is very high that it won't have the desired result and could in fact backfire in unforeseen ways. 

 

So what I'm saying is, be careful. What if your data indicates that a judge is biased and that judge not only denies it, but then turns around and sues you for libel? I'm not saying don't do it. I'm saying, understand the risks. 

I'm going to couch my actual claims in probabilistic, careful language like any good statistician when I actually write up the project. It's just too exhausting to do that all the time/there's kind of a tradeoff between communicating clearly and communicating carefully. Plus my definition of biased is "scores own skaters higher than other skaters" and I'm going to be somewhat agnostic about the cause and nature of the bias, eg. whether it's intentional or not. I'm not going to make any claims about people colluding to fix scores or whatever. I know I don't have evidence for that, and my project is not designed to be able to find collusion between judges anyway.

 

But at the same time, I think showing significant statistical bias is enough to throw your judging into question, regardless of whether you're deliberately trying to take down opposing skaters or whether it's just because your own skaters give you the warm fuzzies so you throw them some extra points. A significant minority of judges don't show evidence of national bias, so it's not like it's impossible for judges to be fair.

 

I'm pretty sure "judge A scores their own skaters two standard deviations above how they score other skaters" isn't really cause for a libel suit, especially when backed up with data. It would be ridiculous. Plus it's really hard to sue for that kind of thing in the US, which is the only place I have any money. I just don't think there are really serious risks here, other than for the ISU if this somehow catches more attention than it did last time. The biggest risk is I sink a bunch of time into a project that will be ignored

[yuzuangel]

Lol there's nothing to sue for libel for if the statistics are calculated directly from the data. And while correlation does not equal causation, you can control for a lot of confounding factors. But the only way to get "true" causation would be to run some double blind or A/B test

[rockstaryuzu]

25 minutes ago, shanshani said:

But at the same time, I think showing significant statistical bias is enough to throw your judging into question, regardless of whether you're deliberately trying to take down opposing skaters or whether it's just because your own skaters give you the warm fuzzies so you throw them some extra points. A significant minority of judges don't show evidence of national bias, so it's not like it's impossible for judges to be fair.

This answers my concern pretty well. Because this is something the ISU should already be looking at - i.e. trying to improve their impartiality by understanding the current state of affairs.

 

Sorry if I touched a nerve earlier. I've seen a lot of heated discussion about this issue on Twitter today, along with a lot of wild ideas for how to address it, and it's got me a little mis-calibrated myself. I'm all for a rational approach though. 

[shanshani]

2 minutes ago, rockstaryuzu said:

This answers my concern pretty well. Because this is something the ISU should already be looking at - i.e. trying to improve their impartiality by understanding the current state of affairs.

 

Sorry if I touched a nerve earlier. 

It's okay, I've been kind of irritable the past couple of days because of jetlag haha. And yeah I think the kind of project I'm doing is exactly the kind of thing ISU needs to be doing to create some kind of accountability mechanism for judges and some way of critically examining judges' records, but alas the ISU is not going to do this kind of thing themselves.

[rockstaryuzu]

1 minute ago, shanshani said:

It's okay, I've been kind of irritable the past couple of days because of jetlag haha. And yeah I think the kind of project I'm doing is exactly the kind of thing ISU needs to be doing to create some kind of accountability mechanism for judges and some way of critically examining judges' records, but alas the ISU is not going to do this kind of thing themselves.

Even if they just did it as basic quality improvement - sort of a 'how are we doing at this, anyway? are our methods working?' kind of thing - that would be better than nothing. 

[Old Cat Lady]

1 hour ago, shanshani said:

would anyone want to help me with a tedious data entry project? basically I would need you to input judges' scores into a spreadsheet. it's pretty easy, it's just time consuming for me to do alone. the point is to create a way to track individual judges' biases by keeping a record of how they scored individual skaters across multiple competitions.

 

Does google sheets work? I'm pretty busy over the next couple months so I'm not sure how prompt I'll be with it, but I'm also doing a lot of testing at work so I'm sure I can help somewhat.

 

Do you have a template for it?  We should make a group to keep each other updated on progress.