UDisc Improves Accuracy Of In-App Round Ratings System

More than 12.5 million rounds of disc golf have been rated on UDisc since October 2023 based on play across 28,007 layouts at 11,951 courses. That's provided plenty of inputs and opportunities for learning, all of which have resulted in wholesale improvements to our round ratings algorithm that were just released to the app. 

Quick takeaways

• Ratings now more accurately reward strong play at tougher courses.

• The model also corrects for a previous disparity between rounds with varying hole counts.

• The changes enable better comparison of ratings across different hole counts, as well as styles and difficulties of courses.

As a refresher, UDisc launched round ratings last fall to help players answer an age-old question: “How good was my round?” Powered by a regression model that can produce a rating for any round played on a smart layout with a difficulty classification, UDisc round ratings allow players to compare their performances across different courses with varying styles and difficulties. For more information about the 1-300+ scale, as well as a deeper technical dive, see the article we released in conjunction with the round ratings launch.

While minor updates to our model have happened as more people score rounds and courses change with time, we identified some issues that called for a more significant adjustment to how ratings are calculated. The result of this change is that the model will now produce more accurate ratings across the board, specifically by better accounting for varying course difficulties and number of holes played during a round. Let’s dive into the updates.

Course difficulties

Our updated model is better at understanding how much better a round on a hard course or layout is than one at an easier track. Let's take a look at some numbers to explain the changes, using three examples from the #1 course in the United States, Maple Hill, thanks to its wide variety of layouts. 

As shown above, the updated model now appropriately rewards performances on harder layouts while correcting for previous ratings that were too high on easier layouts. Anyone who plays at Maple Hill regularly can likely attest that shooting even par (E) on the Golds is a much more difficult feat than shooting 9-under par on the Reds, a truth which the new round ratings models reflect (256 → 229, a difference of 27 points), whereas the old models had similar results for those 2 cases (246 → 239, only a difference of 7 points). These new ratings, then, shake out to be more accurate and more reflective of how good a round really is. 

One other point to notice here is the total spread in ratings. The difference between the Golds and Reds according to the old model is 52 points (see the column “Old round rating: -9”) while the new model reports a difference of 79 (“New round rating: -9”). This extra spread is an indication that the updated model is doing a better job at representing the difference in difficulty between these layouts.

What does this mean for players? Well, if you find yourself playing easy courses, you may find round ratings are a few points lower than they were before. If you play difficult courses, you may find that round ratings are a few points higher. In general, we expect round ratings to “feel” more accurate and more aligned with how well you played when looking between easier and more difficult courses.

Round lengths

The updated model also does a much better job comparing rounds of different hole counts. 

One of the issues with the previous model was a miscalibration for courses with fewer or more than 18 holes. What this means is that ratings for a 9-hole course tended to stay closer to 150, which is an “average” round; if you scored higher than a 150, your round was under-rewarded. On the flip side, for courses with more than 18 holes, ratings above 150 were overinflated. We have improved this behavior for the current model and it will produce fairer ratings across courses of all lengths.

Let’s again look at an example to get an idea for how the updated model will perform. We can use the fourth best course in Arkansas, JBeast Junior, which is an easy 9-hole course. It also has an 18-hole layout, which is just twice through the main 9-hole layout. Let’s compare shooting a perfect round on both of these options:

Looking at the last row, we see that both the old and new models apply a penalty for playing 9 holes instead of 18. A small penalty is justified: it’s a lot easier to go on a hot streak for a few holes than it is to maintain a consistent high level of play for 18 holes. However, the old model deducted a whopping 39 points while the updated model only deducts 9.

By making these changes to the regression model, it levels the playing field. The end result is that performances for 9-hole rounds will now be more comparable to 18- or even 27-hole courses.

The 'Half the Holes' test

Speaking of courses with more holes, ratings on that side have also changed, as we can see by examining ratings from the world-renowned DeLaveaga's 29-hole layout.

Getting birdies on half the holes at DeLaveaga is certainly impressive, but we did not think it was 313 impressive. Now, that comes in at a still great, but more reasonable, 279.

Going one step further, we can compare this -14 at DeLa with a -9 at Maple Hill Golds, which represents getting birdies on half the holes on those hallowed grounds. In the old version, the DeLa round was rewarded more than the one at Maple Hill Golds (291). That seemed…off.

With the new updates to the model, that -9 at Maple Hill Golds now earns a 308. Compare that to the new 279 at DeLa, and we see a much clearer story about which was the more impressive round.

We’ve only shown examples where a player shoots better than 150, but a mirrored scenario exists for lower round ratings as well. If a player shot a round below 150 on a 27-hole layout, their rating previously would have been over-penalized. On the flip side, if a player shot less than 150 on a 9-hole layout, their rating would've been over-inflated. To summarize the entire picture, the previous range of round ratings was compressed for low hole counts and stretched for high hole counts. The re-calibration in this update reduces this compression and stretching.

What does this all mean for you and your everyday rounds? Well, your performances across different numbers of holes played can now be fairly compared. While you will get a boost for proving that you are consistent over a larger number of holes, it will now be much smaller and should feel fair.

With these changes, we think this model will do a much better job at capturing round ratings between all courses and enabling you to compare your ratings better. Now, a 9-hole layout should be comparable to any other courses that you play. Personal bests and amazing rounds will better stand the test of time. These changes should even allow you to compare your rounds among your friends or even with pros (assuming you play by the same ruleset). We hope now that when finishing a round, the round rating that appears will feel like an excellent approximation of how well you played.

The formulaic changes behind the updated model

OK, this is where we get into the nitty gritty – the stuff for the real stats nerds. (One of us! One of us!)

These improved calculations are the result of two underlying changes in our overall method for determining how difficult a layout is. If you need a refresher on how we calculated round ratings in the past, refer to this section of our round ratings article; the TLDR is that using a so-called “standard average player” we would calculate a distribution of possible scores on each hole and roll these distributions up to a scoring distribution on the full layout. 

The first change in the updated model moves away from “standard average player” to something more representative of disc golfers who record rounds on UDisc. The second is a statistical subtlety with how we roll up scoring distributions from holes to layouts.
Before going into detail on these two changes, let’s review one thing that has not changed: Round ratings are still based on the same core score projection regression model. This model is capable of determining the distribution of scores that any player in the world would likely record on any hole in the world – even holes and courses that that player has never encountered before! It’s these projected distributions that serve as the basis for round ratings calculations in both old and new versions of the model.

For the first change, we found that even though the “standard average player” (a hypothetical player whose skill is average among all UDisc players) is “average” by definition, it is not a good representation of the wide variety of skills present among UDisc players. Instead, the updated model pivots to using a large random sample of actual UDisc players, weighted by playing frequency. What does this mean exactly? Let’s walk through the process to see. 

Consider hole 1 at Maple Hill Golds. Supposing our random sample consists of only two players (in reality we use many many more players), the table below shows hypothetical projected scoring distributions for the two players

Note: In practice, probabilities are generated for all possible scores from ace through 12, but we’re keeping it clean here for the sake of explanation.

The combined distribution in the last row is the average of the two players’ distributions on this hole.

Finally, we roll these average distributions up across all the holes in the layout and compare it to your actual round score to generate your round rating.

For maximum consistency, the players present in the random sample are the same for every hole in the world. However, we do periodically refresh both the score projection regression model and the random sample of players, as both the UDisc player base and courses evolve (e.g., a new mandatory is added, a tree falls down, the rough becomes more or less rough, etc.).

The exact way we roll up the scoring distributions for each hole to the layout level is the subject of the second change to our model. In both the old and new versions of the model, your round rating is a remapping of the statistical notion of a z-score measured against the projected scoring distribution on the layout you play. In order to calculate a z-score, we need three numbers: 

1. Your actual round score

2. The mean of the layout’s projected scoring distribution

3. The variance of the layout’s projected scoring distribution. 

Two of these (your actual round score and the mean of the layout’s projected scoring distribution) are straightforward to calculate once we have each hole’s projected scoring distribution. The third (the variance of the layout’s scoring distribution) is quite a bit more complex.

In the old round ratings model we made a bold simplifying assumption that playing any two holes in the same round constitutes two statistically independent events. In essence, this is saying that your score on one hole has no correlation with your score on the other hole. This makes the calculations easier because variance becomes additive – the variance of the layout’s projected scoring distribution is just the sum of all the variances of all of the holes’ projected scoring distributions.

Unfortunately, this is a poor assumption. Hole scores in the same round can be correlated for a plethora of reasons: weather conditions, mental state, hunger levels, whether or not your buddy is razzing you, etc. This means that we also needed to add covariance terms to the layout variance calculation. We have taken these covariance terms into account in the new version of the round ratings model, and since there is a covariance term for each pair of holes on the layout, the layout variance grows with the square of the hole count. This is the cause for the substantial round rating changes for rounds with many more or fewer than 18 holes.

What are the main differences between the distributions produced by this new method when compared to the previous method? In short, a sampling method that better represents the wide variety of skills present in the UDisc player base combined with more robust statistical modeling results in round ratings that more accurately reflect the quality of your rounds across all the different courses you encounter in your disc golf journey. Now get out there and try them for yourself – the updated ratings model is in the app and ready for your next round.