In his recent article Mr. Channon
commented on dressage judging accuracy and referred to other Olympic sports. In
number of earlier publications gymnastics or ice skating judging systems were
called out. Let's have a look at figure ice skating judging system as one of
examples which dressage could follow. Please find below key elements comparison
of judging systems of these two sport disciplines which clearly demonstrates
how much the argumentation is missing the point, and actually, how similar (or
far worse?) situation in ice skating judging is.
1. Scale of judges marks
In dressage judge operate marks from 0 to 10 with 0.5 interval. In skating, the judge can assign the Grade of Execution mark from -3 to +3 range with 1,0 interval. Mark ‘0’ indicates that figure performance was executed without mistakes and a base mark will be assigned. That would be the equivalent of base mark in dressage of 7,0 points for expected quality performance.
2. Guidelines
The ice skating judge must follow so called guidelines. These guidelines define:
A) athlete mistakes which trigger judge to use negative mark with penalty ranging from -1 to -3, and
B) 8 criteria to be used by judge to inflate the base score: 2 out of 8 criteria must be met by athlete performance to increase score by 1 point, 4 out of 8 criteria to move to +2 points and 6 out of 8 criteria to move to +3 points.
Finally, the judges scores are converted based on conversion table per each figure into actual points which has no standardized relationship to the base value (1 score change can vary per figure from +10% to even 25%). That would be a bit analogic to dressage coefficient system, where some of dressage figures are twice more important then others. In figure ice skating portfolio of jumps, steps, pirouettes, etc. is large, so each figure has different difficulty which is translated into different base value, so to say - different coefficient.
Alike in skating, judges have the FEI criteria defined for dressage scoring:
A) guidelines which are compulsory for dressage judges for fundamental mistakes which define performance mistakes and scores for such mistakes ranging from zero up to 7,0 (base score), and
B) dressage Handbook which should help judges with definitions if 8 criteria (as in skating!) to be evaluated in order to provide proper scoring of performance.
3. Single move score determination
In both dressage and ice skating, the judge starts with base score and either decreases the score of the athlete following the defined guidelines for mistakes or moves up following predefined 8 criteria of performance evaluation.
4. Collective marks
In dressage we had collectives, however in 2018 only rider collective mark remains. The other collectives, which were dropped starting January 1st 2018, would be equivalent of ice skating component marks - they reflect overall performance of skaters and are scored from 1 to 10 points with 0,25 interval. These scores are called components: skating skills, transitions/foot work, performance/execution, choreography/composition, interpretation/timing.
5. Final score determination
In ice skating there are max 9 judges. Out of 9 judges marks 2 judges marks are randomly dropped. From remaining 7 judges marks Hi and Lo are dropped. Average of 5 marks is a final score for each figure. All figure skating judges sit together along the wall.
In dressage we have up to 7 judges sitting around arena observing performance from different point of view. The nature of dressage figures determines that some judges may not observe some mistakes which not a case in ice skating is.
6. Judging accuracy
We are facing major discussion in media related to judging accuracy. The magical 5% difference is a topic which grows to crisis in judging. Let’s see how it is in ice skating …
A) There is debate about judging in skating as such, however it is limited discussion about judging accuracy since … there are no publicly available scores from individual judge, media and organizers presents totals scores only! Official score sheets present anonymized (but not anonymous) individual item marks. These sheets are not presenting final score or ranking per judge. Only final score of athlete is presented. As a result the stakeholders cannot discuss and compare judges accuracy, athletes or audience can debate the judges performance however, there is no direct visibility to individual scores as such!
B) There are 21 scores available with 0,5 point interval to be assigned by judge in dressage. For ice skating there is a range of 7 scores with interval at 1,0 point. So by design (less granular scale), ice skating judging is less accurate then dressage system! And as a result, ice skating federation sensitivity in the process of judges evaluation is at 15% and the controversy starts at 20% differences!
1. Scale of judges marks
In dressage judge operate marks from 0 to 10 with 0.5 interval. In skating, the judge can assign the Grade of Execution mark from -3 to +3 range with 1,0 interval. Mark ‘0’ indicates that figure performance was executed without mistakes and a base mark will be assigned. That would be the equivalent of base mark in dressage of 7,0 points for expected quality performance.
2. Guidelines
The ice skating judge must follow so called guidelines. These guidelines define:
A) athlete mistakes which trigger judge to use negative mark with penalty ranging from -1 to -3, and
B) 8 criteria to be used by judge to inflate the base score: 2 out of 8 criteria must be met by athlete performance to increase score by 1 point, 4 out of 8 criteria to move to +2 points and 6 out of 8 criteria to move to +3 points.
Finally, the judges scores are converted based on conversion table per each figure into actual points which has no standardized relationship to the base value (1 score change can vary per figure from +10% to even 25%). That would be a bit analogic to dressage coefficient system, where some of dressage figures are twice more important then others. In figure ice skating portfolio of jumps, steps, pirouettes, etc. is large, so each figure has different difficulty which is translated into different base value, so to say - different coefficient.
Alike in skating, judges have the FEI criteria defined for dressage scoring:
A) guidelines which are compulsory for dressage judges for fundamental mistakes which define performance mistakes and scores for such mistakes ranging from zero up to 7,0 (base score), and
B) dressage Handbook which should help judges with definitions if 8 criteria (as in skating!) to be evaluated in order to provide proper scoring of performance.
3. Single move score determination
In both dressage and ice skating, the judge starts with base score and either decreases the score of the athlete following the defined guidelines for mistakes or moves up following predefined 8 criteria of performance evaluation.
4. Collective marks
In dressage we had collectives, however in 2018 only rider collective mark remains. The other collectives, which were dropped starting January 1st 2018, would be equivalent of ice skating component marks - they reflect overall performance of skaters and are scored from 1 to 10 points with 0,25 interval. These scores are called components: skating skills, transitions/foot work, performance/execution, choreography/composition, interpretation/timing.
5. Final score determination
In ice skating there are max 9 judges. Out of 9 judges marks 2 judges marks are randomly dropped. From remaining 7 judges marks Hi and Lo are dropped. Average of 5 marks is a final score for each figure. All figure skating judges sit together along the wall.
In dressage we have up to 7 judges sitting around arena observing performance from different point of view. The nature of dressage figures determines that some judges may not observe some mistakes which not a case in ice skating is.
6. Judging accuracy
We are facing major discussion in media related to judging accuracy. The magical 5% difference is a topic which grows to crisis in judging. Let’s see how it is in ice skating …
A) There is debate about judging in skating as such, however it is limited discussion about judging accuracy since … there are no publicly available scores from individual judge, media and organizers presents totals scores only! Official score sheets present anonymized (but not anonymous) individual item marks. These sheets are not presenting final score or ranking per judge. Only final score of athlete is presented. As a result the stakeholders cannot discuss and compare judges accuracy, athletes or audience can debate the judges performance however, there is no direct visibility to individual scores as such!
B) There are 21 scores available with 0,5 point interval to be assigned by judge in dressage. For ice skating there is a range of 7 scores with interval at 1,0 point. So by design (less granular scale), ice skating judging is less accurate then dressage system! And as a result, ice skating federation sensitivity in the process of judges evaluation is at 15% and the controversy starts at 20% differences!
Anyway, let’s have a look at the recent examples of ice skating judging. I have taken a look at the score sheets of the most recent skating international competitions - senior man competition in Budapest in Dec 2017. The program consists of 13 figures, test was judged by 6 judges:
The winner:
Judges marks for 13 figures - dressage equivalent: 69,3% - 75,3%
Collective marks equivalent – points difference between judges: 0,5 – 0,75
Figure scores %points differences (in dressage equivalent): 3 moves with 20% difference, 7 moves with 10% difference.
Another randomly selected competitor from middle of the ranking:
Judges marks for 13 figures - dressage equivalent: 63,1% - 67,4%
Collective marks equivalent – points difference between judges: 0,75 – 1,5
Figure scores %points differences (in dressage equivalent): 2 moves with 20% difference, 8 moves with 10% difference.
I can only assure that very much the same differences are observed across the whole score board. Could not you agree, that these marks ranges are not different to dressage results boards, Mr. Channon?
Let's have a look at another, more prominent example - here are 2017 European Championships scores, 9 judges, 7 figures.
The winner:
7 figures score average - dressage equivalent: 85,7% - 92,8%
Collective marks equivalent – points difference between judges: 0,5 - 1,0
Figures scores %points differences (in dressage equivalent): 3 moves with 20% difference, 4 moves with 10% difference, all moves score has a difference!
Vice champion:
7 figures score average - dressage equivalent: 74,2% - 84,2%!
Collective marks equivalent – points difference between judges: 0,75 – 1,25
Figure scores %points differences (in dressage equivalent): 1 move with 30% difference, 2 moves with 20% difference, 4 moves with 10% difference, all moves score has a difference!
The same (or higher) level of
differences between judges I calculated for the last Olympic games. I think
that these differences would be immediately contested and commented as
outrageous if happening during EC in dressage ...
Three months ago ISU, international
skating federation decided to increasing the number of scores (GOE) from seven
(+3 through -3, including 0) to 11 (+5 through -5, also including 0). The interval
between the scores would be set at 10 percent of the base value, as opposed to
the current system. So new figure skating systems looks very much similar to
our dressage judging system!
So let me summarize this short high level analysis. As one can see from the above comparison:
A. Both judging systems are pretty similar,
B. Ice skating judging differences are the same or bigger comparing to dressage,
C. In dressage, judges scores differences above 6% are under scrutiny, in ice skating 15% is acceptable threshold,
D. In ice skating scores should be the same as judges sit together and enjoy the same point of view, in dressage judges evaluate different view with the aim to detect mistakes hidden for other judges,
E. Ice skating system is not presenting each of the judges final scores and ranking, just compound final scoring to avoid discussions.
F. New figure ice skating judging
system moves to dressage like scale ranging 11 scores with 10% interval.
G. In figure ice skating judging system each
move/figure has different base value, so to say in dressage language,
coefficient. It could be interesting proposition to drop presentation of separate scores/rankings of each dressage judge and move towards combined score presentation in dressage. That would end for good all discussions about the system! In this way judges can move to detailed discussion about:
a) updating the dressage handbook and turning it into more practical and compulsory code of points,
b) training judges in application of the new code.
Would not you agree?
Brak komentarzy:
Prześlij komentarz