I have blogged about the Universal Tennis Rating (UTR) System before (Ratings vs. Rankings) and I believe it is an unmatched tool for coaches, players, and parents to accurately see just how good they really are in comparison with everyone from professional players across the world to anyone in your backyard. I know many college coaches are using UTR to evaluate how a player would fit into their line-up, top collegiate players are using it to see if pro tennis is a dream they should attempt to pursue, and junior players are using it as the measuring stick to see if they really are improving at a faster rate than their peers.
With the all the undeniable valuable surrounding UTR people still ask, “How accurate can the Universal Tennis Rating System really be?”
Well just this week I did a case study of my own to answer that very question using the results of the Pennsylvania AAA State High School Singles Qualifying Tournament here in Western, PA. Once the tournaments concluded and all finals results were published I logged into UTR and looked up the ratings of every single player who attempted to qualify. I put each player’s rating number next to their name on the draw sheets and looked to see if the actual results mirrored what each player’s respective UTRs were. What I discovered in doing this was nothing short of amazing!
In AAA High School Tennis in Western, PA there are 4 conferences or sections. Each section holds a singles qualifying tournament in which the top 4 players in each (16 total) move onto the next round where they square off to complete and qualify for the state tournament.
In looking at each player’s UTRs in the 4 section tournaments the outcomes mirrored the UTR ratings perfectly. In all of the section tournaments the player with the highest UTR finished 1st, second highest UTR 2nd and so on. The correlation between actual results and UTRs were beyond coincidence!
Section 1
Section 2
Section 3
Section 4
I was intrigued so I continued my case study on to the next qualifying round. The second qualifying rounded consisted of a tournament where the 16 best high school tennis players in Western, PA square off to qualify for the State Championship Tournament. I thought if anywhere the Universal Rating System would not predict results would be here where players are so closely matched. Out of all 16 matches played in the entire tournament 14 of them mirrored results the UTR ratings would have predicted.
Qualifying Championships
So what about the 2 matches where the lesser rated player won? This must mean UTR is not accurate, right?
Well, what I discovered in these two matches supported the accuracy of the Universal Rating System even more. The first match in which the lesser rated player won had a margin of 0.04 difference between the two competitors. The second match had a difference of 0.27. I think It is pretty safe to say these players were very evenly matched and neither was the favorite. What was even more interesting was the same player was involved in both of those matches, signifying that perhaps this was a breakthrough tournament for her.
After closely looking at the UTRs of players in a competitive tournament setting it just reaffirms to me how accurate it is. I’ll be continuing this experiment in a few weeks by looking at the Universal Ratings of all the players who will be competing in the Pennsylvania State Singles Tournament in a few weeks. Stay tuned for the findings…
Outstanding. Great Share!
Bill
I have also had a chance to assess the accuracy of the Universal Tennis Rating system. I have a child who plays national US junior tournaments, junior ITF tournaments around the world and professional events. It is accurate for US junior tournaments. It is completely inaccurate for world junior ITF tournaments and professional tournaments. The foreign players are totally under ranked as are the professional players. The college players are really under ranked. The US players playing only world ITF junior events and pro events are under ranked when they are actually playing a harder schedule than the over ranked US players only playing US junior events. The rating system is a good idea but if you are competing regularly in events against under ranked players – say a season of college tennis – you can’t get your ranking up. Same if you move to pros or only play ITF junior events in other countries. Your ranking actually falls when you are really playing a harder schedule. The system rewards playing over ranked US junior players. The foreign players are way better across the board. I can’t tell you how many times I have looked up the ranking of a foreign opponent of my child and laughed at how low it was considering how good the player actually was compared to a similarly ranked US player. Countless times this year!!!
Thanks a ton for your input, you obviously have tremendous wisdom from your experiences and I give you tremendous credit and respect for the work you do with your daughter. You are the exact kind of person we should be listening the most to in order to develop a competitive pathway to championship tennis.
I could absolutely see how ratings could be skewed at those higher levels in UTR, especially in college when players get pigeon holed to certain spots in the line-up or when not all players in the ITF and pros are rated and/or rated accurately. The concept of the system in and of itself does make a lot more sense to me because players are chasing getting tangibly better versus just accumulating meaningless points. Players compete for two big reasons, to win titles and defeat rivals and we have to get back to basics of just getting better at tennis.
Adam Chodur my son had a utr of 8.97 last week and after finally winning a tournament and beating a player 10 utr one of his bigger wins last week his utr this week dropped to 8.94. Adams coach who plays 50 and under has the same utr as my son who is 12 because age does not matter. Adam serves at 85mph the coach at 115mph. A 12 year old playing up which Adam does if Adam plays another utr 9, 16 year old that is not the same age does matter in the juniors. Location does matter. UTR 10 in Nevada is different than 10 UTR in California. So I agree the UTR has some serious flaws and reaching out to UTR regarding this they have ignored and not responded. Definetly not the most accurate system in the world. Universal Tennis has been the most accurate.
Sorry I am only seeing this comment now it got buried in the list. Your son’s UTR could have gone down because the algorithm takes the last 30 competitive matches in a calendar year. Its actually quite complex. So although he may have had a win over a UTR 10 some of his other matches may have expired in the calendar year thus his UTR could have gone down. However, do not get caught up on 8.97 and 8.94 they are the exact same thing. UTR is a rating and not a ranking its gives you an accurate barometer of where your skill level is at. There is also other factors that go into competition and I can tell you a 16 year old is favored over a 12 year old in almost all cases there are psychological element to a match like that. I have no seen UTR vary from area to area but it may be easier in some areas to attain a higher UTR because the level of competition is high where if you live in a rural area where there are not many tournaments or opponents of that level its hard to attain a higher rating without results. UTR is not perfect nothing is and we cannot try to make it. However I will tell you it is much better than a points system because it encourages players to work hard to improve and to play the best competition they can find.
I agree with you wholeheartedly! Even my child was interested in seeing her rating increase each month. It’s always nice to have just another little motivator. And, the system is dead on for players playing in the US against opponents who only play in the US. It’s a terrific benchmark!!
i am not surprised by Lori Bellis’s comment. I experimented with developing a rating system for a group of players. I wanted to schedule competitive matches for everyone. It soon became apparent to me that scheduling the matches so they would be competitive, skewed the ratings. If I wanted the “best” ratings, everyone must play everyone else. The difference for under or over-rated players should be expected if there is an insufficient amount of mixing.
Totally agree. I am not really sure about this but I think the ITF scores are not entered into the Universal System. Being able to input scores is a founding principle for UTR to actually work. And in college, players just get stuck always playing players at their level, unless they break out on their own and play futures they are just stuck there. That is why their rating is stagnant and also why players fail to jump from college to the pros… 1/3 of their matches should be against better players and they just are not getting that.
Alex, thank you for posting your article on the accuracy of the UTR system.
Your high school tennis results are consistent with those recently reported by Jeff Menaker, after the New York State Tournaments. See http://goo.gl/iWolND
I’m am surprised to see the comment posted that suggests UTR is “completely inaccurate for world junior ITF tournaments and professional tournaments.” Speaking from my own experience, I have found the UTR system to be as/more accurate with pro / international events, than domestic junior events (i.e. I’ve submitted about 2,000 junior match results, have two daughters that played ITF junior events, and host organization for ITF Junior Grade 5, Men’s F8 Futures, and Women’s $25K Challenger events). Testing process (where possible) included complete draw review. Reviews for some ITF junior tournaments can be a challenge, where international players have provisional ratings (i.e. less than 100% UTR Reliability due to having only a few match results in the system).
Keep in mind, a Universal Tennis Rating indicates a player’s average level of play over his or her last thirty matches played during the past twelve months. What the system is not intended to do, is predict the outcome for any given player, on any given day, under a multitude of playing conditions, which may vary from one match environment to another.
With this in mind, the UTR system should provide adequate rating details to promote level-based competition, which fosters player development and also helps players find a good game.
– Bruce
I have child successfully playing D1 tennis. Per this rating system might have been able to play DIII. Also have another child playing junior tennis on a national level and per this rating system should be able to enjoy playing recreational tennis. I’m confused.
The system is by no means perfect and the pyramid graph is more of a guideline than something set in stone however in my experiences it is fairly accurate. I’d have to know the specifics of your situation a little more but the range of playing levels in D-I varies greatly. For example the University of Virginia on the men’s side ranges from players with a UTR of 13-15 while a school like Duquense University which is also D-I ranges in players with a UTR from 9-12. In regards to your other son or daughter playing national tournaments it is obvious they can play at the level however their UTR may not reflect that because they are not getting enough competitive matches throughout the year into the UTR system. The only way to raise your UTR is to have wins or competitive matches against those with a UTR higher than your own and it has to get entered into the system. There could be a kid out there playing competitive practice matches with college or pro players everyday and it would certainly reflect in their level of play but not their UTR. Again I would need to know more specifics but it could be a good indicator its time to look at their competition schedule. The year should be structured in 1/3’s in regards to competitive play and perhaps its time to evaluate because the skill level is growing quickly. Thanks for reading and any way I can help you out let me know. Here is a link to scheduling the year out in 1/3s.
Hi Alex,
How does one find out what their UTF rating is?
Amy
Just go to http://universaltennis.com and create a free basic account and if you have played in enough matches that have been entered into the system you’ll have one. It will also tell you how reliable your rating is.
The problem is the secretive nature of the ratings system. I’ve noticed significant issues in the lower rankings and there is said to be some issues with high school before college as well. With the actual formula remaining a secrete, it lends itself to problems in my opinion. Although the idea is good, I am against using secrete formulas.
I don’t know about secretive… The algorithm UTR uses is proprietary so making it public after all the work they put into it would not make a lot of sense for them. They want to grow tennis but they are providing an amazing service and deserve to be rewarded for their work.
Accurate? Serena Williams has a UTR of 13. A certain group from a D1 college team, which finished last season somewhere ranked 160 in the country, each have UTRs of 13.
Can it really be accurate if it rates players from a college that is not on anyone’s RADAR the equal of the #1 WTA player in the world?
This is was an easy example to really question the system, sources, and algorithm, rather than simply drinking the popular coolaid of the day. I know of a number of other cases in local juniors where UTRs for 14 years olds with very little experience are equal or very close to other juniors with long and successful histories, and which are visibly much better (when actively analyzed playing).
Also I just looked it up… the D1 NCAA womens singles champion from Virginia has a UTR of 12.35. I’d love to know the team you are referring to in comparison to Serena. Maybe a Men’s D1 team?
Posted a reply a while ago that is not showing yet. It was nothing diragatory, simply a comparison of current UTR levels of some players to demonstrate problems in the algorithm used. Seems that comments on the article above are being censored.
I am sorry I have been away from the blog for a while…my wife and I just had a baby boy 🙂
In response to your other comment by no means is the system perfect. The system itself works but in order for rating to truly reflect a player’s highest possible competitive level they must be playing higher levels of competition. I do not know specifics of Serena’s UTR but I would say she is one of the best women in the game and her actual level of play is probably higher than is reflected in her UTR simply because she doesn’t play anyone with a UTR higher than hers therefore she cannot improve her rating in the UTR system. This happens in college as well as players get stuck in spots in the lineup and never play higher levels of competition or their school does not have a competitive enough schedule. Happens in junior tennis as well and its a darn good indication for that child’s parents or coach that if their UTR is the same as their opponents and they are easily winning they are not being stretched enough in competition and its time to look at their yearly scheduling. Again by no means is the UTR system perfect but in my experiences is that it is very accurate in the majority of matches. There will always be outliers. I work with a girl who really does not play lots of tournaments just yet and when she starts her UTR will be grossly underrated compared to her skill set. My main point with the article is that it is accurate and probably enough so that it should be used to seed tournaments like this high school state championship. It also is an excellent tool for parents and coaches to assess how their child is growing over time and in planning out their yearly tournament schedule.
Its a far better system then the NTRP. For example calling someone or yourself a 4.5 or 5.0 player doesn’t tell you very much. I decided to look up the UTR of players on my daughter’s 4.5 league and they ranged for a high of UTR-12 to a low of UTR-5. That’s a good D1 player playing on the same 4.5 team as a low level D3 player. However It’s not perfect, Jamie Lobe, the former NCAA champ is a UTR-13.. but her friend, a man, ( my daughter’s hitting partner) is a 11.5. A match between them should not be competitive. Lobe should easily win but the truth is the Lobe loses every time they play.
The NTRP is a subjective system. You need someone to subjectively rate you and give you a score. UTR is calculated on actual match performance and there is nothing subjective. If you win hitting and underhand serve you still won.
There is a difference between men and women physically on the tennis court that can’t be denied. The system is also not perfect. There are some ex-players who could have a UTR that might be higher but they simply are not playing tournaments or people who accurately reflect the level of play they are capable of. I myself don’t even have a UTR but I also have not played a tennis tournament in years. There are also different ways specific players matchup that make a difference.
On a final note I believe UTR is most accurate when traditional scoring is used. No-Ad scoring only requires players to win 1 critical point to win an entire game. Winning by two heavily favors the better player. No-Ad scoring brings a lot a randomness and parity and makes upsets in terms of UTR much more likely.
What is the UTR corporate structure and how do they make money? Also, the club players’ results are at least a month behind the pros. Serena’s results from Saturday are there but my results from last month are not…..any thoughts?
I don’t know much about their corporate structure. I know they just hired a new CEO and thus the push on the tennis channel. The only way they make money as far as I know is through website subscriptions. The premium side of the website is fairly valuable for the serious competitive junior player and it is essential for college recruiting in and out of the United States at this point.
As far as results being entered I am fairly certain that is in your club’s end. All the pro, college and junior scores are automatic. I have personally submitted high school match scores and they were up within 24 hours. I am guessing the person entering your club scores is only doing so once a month.
Hope that helps you out.
Thanks for the info on corporate…..as far as scores, these are USTA League matches that are put up the same day the match is played. Who is the CEO of UTR?
Mark Leschly is the new CEO. I honestly do not know him or even communicated with him. Your USTA league matches should be in there right away. Tennislink is completely automatic.