The Ranking Methodology
The NRSL has two independent measures of every athlete we track. In both cases we look at a subset of the regattas based on consistency of the racing over years and level of competition.
- Power Score™ is a proprietary algorithm specifically designed to measure the quality of the rowing of each athlete against their entire racing category in the United States.
- Points- We have created a system to reward participation and speed in a series of the most competitive (inter)national level regattas.
The following events were used in this BETA Ranking.
- NSR I
- NSR II
- Trials I
- Trials II
- East Coast Speed Order - 6k erg
- East Coast Speed Order - Row
- Pan Am Trials
The Power Score™’s statistical model looks at all racers, their event class, where they finished in the event and where the winner finished in that event in order to standardize each race. We then isolate the positive and negative standard deviations for all events on a day and standardize the regattas to one another. As the data is collected over the course of a season, a year, and a career it is more mathematically sound to make quantitative assessments of individuals. This then allows us to attempt to differentiate that individual from a random rower. When tested and graphed, rowing race results are well approximated by 0.63 positively skewed normal distribution. This means that we are able to apply several statistical tests to the population, and more importantly to individuals.
The first step is determining whether the athlete is above or below the average, this is done with a 3-case sign test which also checks for overlap in the results, indicating that the rower is not differentiable from the mean. Where there is insufficient data we use a percentile of the population to more accurately place rowers. Once it is know whether the individual is in the top or bottom 50% we can run a series appropriate 1-tail T-test to further narrow down where in the population the athlete belongs. This test contains more error as the individual moves toward the top 33% so the most important element to keep in mind is the confidence interval that comes along with our final rating. The main number seen has a α=0.9-0.95, which is a standard interval for statisticians using this test, is accompanied by a color coded Confidence Interval (CI) which shows our best guess, but acknowledges how much uncertainty is inherent to the Power Score™.
We represent the confidence interval with the number by displaying it in three colors. Red has a wide range that is 15 or more. This happens when there is only a single observation or results are disparant. Yellow is an intermediate (±6-14) confidence indicator and identifies relatively consistent racers with a few results and extremely consistent racers with only 2 or 3 results. Green is our highest degree of certainty(≤5) and only racers with multiple, extremely consistent, results will show up with green indicators.
The biggest question in all of this is the sample size, or the number of times athletes race. For an individual to have an accurate representation the math is made more robust by having as many results as possible. Just like the individuals the population as a whole is made more accurate by having more data. One of the keys to comparing individuals in different boat classes accurately comes from our ability to unify results from every event into a single, fair, data pool. The process keeps all of the original race data but transforms it so that a lightweight 4+ can be compared to a openweight 8+ and an openweight 1x without biasing in either direction. This requires all elements of the race to be transformed including winning time mean time and slowest time reported. These results stay in the system for future reference and point acquisition, but the program evaluates them such that they show more information about the rowing population and less about their event population. Each result informs the tests better, and narrows the Confidence Interval. It is this narrowing of CIs that will truly identify how different one rower is from another, not just the best guess.
An academic paper, presently in preparation, will provide a more complete explanation of the statistical analysis that creates Power Score.
The ranking by point system that is employed is much simpler than, and perhaps just as important as, all of the statistical work being done above. This system exists to promote those who compete regularly and display great speed. Each race that the NRSL looks at has a certain number of points designated as its maximum for winners based on whether it is an erg, head, or non-Olympic/ Olympic sprint race. The number of points for the winner will be diminished if there are fewer than 50, 35, 12, or 8 respectively; however there is a minimum of 10points awarded to every participant who completes the race. This means that a single bad race does not take a front runner out of the picture it just encourages them to attend an extra event in the year and they will be back on even footing. The one limitation that is put on point earning, is that only the top result from each day of a regatta is used. This means that rowers cannot enter 6 events, paddle down the course, and get 60 points.