When I set up the first Living Room Times NCAA Pool in 1996, I was using a primitive piece of DOS software (yes, DOS!) that assigned, by default, point values through the tournament’s six rounds in a 5-7-10-15-20-25 fashion. I’m not sure if this was customizable, but even if it was, I gave it zero thought. I’d never run an NCAA Pool before, after all, so I had no basis on which to doubt the wisdom of the proposed point totals. I just went with the software’s built-in scoring system.
Over the years, however, it became clear that this scoring system was wildly imperfect. Picking the national champion shouldn’t be worth only 5 times more than picking each first-round winner. With 32 first-round games, such a system makes the first round (as a whole) worth more than the Elite Eight, Final Four and national title game combined! Put another way, the first round is worth almost six-and-a-half-times more than the “sixth round” (the title game). Put yet another way, more than 55% of the total points available in the entire pool are given out on the tournament’s first weekend! Basically, the system doesn’t sufficiently reward picking the right teams at the end.
But while I recognized this, I never wanted to change the scoring system, because I liked the ability to historically compare scores over the pools’ 15-year life span. It emphasized how long I’ve been doing the pools, and added drama to late rounds in pools where the champion is already known (e.g., “will Mike Tran break the all-time points record”?).
The NCAA’s addition of the “First Four,” however, has forced my hand. While it was possible to ignore the #PIG (play-in game) because the winner was a #16 seed which was just going to lose its next game anyway, the #quadPIG cannot be ignored because two of the winners will be at-large teams, #11 or #12 seeds with serious hopes of reaching the second weekend and beyond. The play-in games now have to be included in the pool. This not only shortens the pool signup window, it also means that it’s no longer possible to maintain the historical 5-7-10-15-20-25, maximum-477-points-total scoring model that I’ve been using since Bill Clinton’s first term as president.
Even a small tweak, like, say, making the First Four games worth 3 apiece, then leaving everything else the same (or, say, reducing the 15 to 13 and making each First Four game worth 2, thus maintaining the 477-point max with a 2-5-7-10-13-20-25 system), would throw off and ultimately invalidate the historical comparisons, which were the only reason I’ve kept using the stupid 5-7-10-15-20-25 system all these years. So I’ve decided to completely throw it out and start from scratch.
But what to replace it with?
Many pools use a 1-2-4-8-16-32 system, making each round worth the same amount (32 points), but the individual games worth twice as much with each successive round. Although it has a great internal symmetry and logic, I’ve never liked that system, because I think it makes the later rounds too important, and thus the first and second rounds too unimportant. In other words, it’s the opposite extreme from the 5-7-10-15-20-25 system. I want a happy medium.
At the same time, I’m fond of scoring systems that allow for “upset points,” in which you get extra credit for picking the lower seed to win, thus discouraging overly “chalky” brackets. But upset points shouldn’t completely overwhelm the scoring — for instance, you shouldn’t get three or four times as many points for predicting a 12-over-5 upset as you do for picking the #5 — lest wild brackets that pick an inordinate number of upset picks — many of them wrong, but a handful right — get an unfair advantage over more reasonable brackets. (Rule of thumb: if picking all four #12s, and going 1-for-4, gets you anywhere near as many points than picking all four #5s, and going 3-for-4, that’s no good.) Also, the relative impact of upset points should fade somewhat in the tournament’s later rounds, lest the person who had George Mason plus a bunch of incorrect Final Four picks get hopelessly far ahead of the person who missed Mason but got LSU, UCLA and Florida right, and picked Florida over UCLA in the final (for instance). The Mason picker deserves major bonus points, but not a virtual win-the-pool-free card.
Keeping these principles in mind, I am considering the following scoring system for this year’s men’s pool:
FIRST FOUR (quad-PIG): 8 points per correct pick
ROUND OF 64: 15 points per correct pick + 1 times the seed of the winning team
ROUND OF 32: 25 points per correct pick + 2 times the seed of the winning team
SWEET SIXTEEN: 40 points per correct pick + 3 times the seed of the winning team
ELITE EIGHT: 70 points per correct pick + 4 times the seed of the winning team
FINAL FOUR: 105 points per correct pick + 5 times the seed of the winning team
TITLE GAME: 160 points per correct pick + 6 times the seed of the winning team
That means the total number of points available (not including upset picks) in each round (as a whole) are: 32 for the First Four, and then for the six full rounds, 480-400-320-280-210-160. That’s a much more gradually declining slope than the 160-112-80-60-40-25 of my old system (in percentage terms, the proposed system is 26%-22%-17%-15%-11%-9%, versus the old 34%-23%-17%-13%-8%-5%). Obviously, it’s not the symmetrical 16.66%-per-round distribution of a 1-2-4-8-16-32 system, but as I said, I don’t like that system’s overemphasis on later rounds. I think my system is a pretty decent happy medium.
To play out a couple of scenarios, if one #12 seed pulls a first-round upset, our hypothetical 3-for-4 picker of all #5s over #12s would get 60 points for his trouble, while the 1-for-4 picker of all #12s over #5s would get 27 points. Meanwhile, someone who went 3-for-4 because they picked two #12s, the one that won and another that lost, would get 67 points (7 more than the “chalky” 3-for-4 contestant). Finally, someone who went 2-for-4 because they picked the “wrong” #12 seed, and thus got one upset pick wrong and one non-upset pick wrong — this is what always seems to happen to me — would get 40 points, while somebody who went 2-for-4, but had the upset as one of their 2 correct picks, would get 47 points. It’s hard to be scientific about this, but that strikes me as roughly the right balance.
As for my aforementioned 2006 Final Four scenario, the person who picked George Mason to reach the Final Four, but got all the other Final Four teams wrong, would get 114 points, while the person who picked #2 UCLA, #3 Florida and #4 LSU, but not #11 Mason, would get 246 points. That’s barely twice as much as the Mason guy, which is actually a nice bonus for Mason guy, considering he got fully 3x fewer Final Four teams right. I think that’s fine, but I don’t want him to get too much of a bonus, considering he still only got one Final Four team right. Meanwhile, somebody who got both Mason and LSU, but not the others, would get 200 points — well ahead of somebody who got just UCLA and Florida (180), but well behind our 3-for-4 UCLA-Florida-LSU guy (246). However, if somebody got Mason, Florida and UCLA, they’d be sitting pretty at 274, well ahead of the chalkier UCLA-Florida-LSU 3-for-4 contestant (246). Basically, correctly picking the upset gives you a strong “tiebreaker”-like advantage over somebody who got roughly the same number of games right in a given round, but it won’t generally vault you ahead of somebody who got a much higher overall percentage of picks right. Again, I think that’s just about how it should be.
Anyway, I’m just wondering if anyone has any thoughts. Ultimately, this is my call as the Living Room Times Pool Dictator :), but I value any feedback that readers and contestants might have.
On the other hand…
1-4-7-10-16-24-41 would add up to 477, and approximate the “happy medium” percentage values I’m talking about, although without upset points, allowing me to sorta kinda maintain historical comparisons…
Personally I don’t think you need to provide bonus points for upset picks. People who choose those are ALLREADY getting a bonus, they are getting points that those who picked the obvious choices aren’t.
Upset picks are a gamble, but they pay off for people who are willing to take some guesses on possible/probable upsets.
In addition it means comparing scores between years will be harder. A theoretical perfect bracket in 2012 might be worth less (or more) than a theoretical perfect bracket in 2013, you are at the mercy of which upsets happen where, I think perfect should be perfect. Obviously thats less likely to occur but it does mean you can reasonable compare performance between years. People who got similar scores were likely similarly accurate in picking, where as with an upset system, a person who picks more innacurately but gets some upsets right could end up with a similar score to someone who is more accurate.
I agree with David, upsets and seeding should have nothing to do with the point system — you either picked the winner correctly or you didn’t. Also, the main problem giving weight to specific upsets or seedings is that its not commensurate with the difficulty. Consider: In the first round you have two key statistical problems you’re dealing with: first, how likely is it that the lower seed will win (e.g. a 16 over a 1, a 15 over a 2, etc.), and then second, how likely is it that the upset will be pulled off by a particular team. For example, a 12 seed knocks off a 5 seed roughly 25% of the time, meaning in all likelihood at least one of the four 5 seeds is going to lose, but how do you know which 12 seed to pick out of the four? The problem with giving weight to upsets is, these two statistical challenges intersect at different points of the axes depending on the seed. For example, on one end of the scale, the odds of a 16 or a 15 seed winning are so remote, it’s not even worth trying to guess which one of them might be the one to pull it off. On the opposite end, 7-10 and 8-9 games are so 50-50, it’s just about a total crapshoot, and there’s literally no point in giving someone extra credit for picking the lower seed (if you did, you’d be incentivized to pick all four 10 seeds and all four 9 seeds, which is highly irrational and unlikely, but points-wise, it gives you the highest, most likely payoff, vs. trying to guess which two 9 seeds and which two 10 seeds will lose and which will win). The sweet spot is, unsurprisingly, the 5-12 matchups. Since probabilities are that one 12 seed will win, the question becomes, is it worth trying to guess which one? I say yes. Why? Because typically, if you do homework on the teams, there typically is one 12 seed that looks more likely to pull off the feat than the others (maybe it has a senior-laden squad, maybe it has the lowest kenpom differential vis-a-vis the 5 seed, or maybe it just matches up really well based on style of play). So what you have then is a situation where it makes no sense in rewarding people for picking the 16 seed upset because it’s, basically, total irrational luck if they get it right, and there’s no sense in rewarding people for picking the right 9 or 10 seeds because, again, it’s a 50-50 crapshoot so what’s the point? (Across a large enough sample, some will hit the craps and others won’t in roughly equal numbers). The only place where rational thought and empirical research is truly rewarded by weighting seeded upset is in the 12 seed game (13 and 14 seeds require more luck, respectively, and 11 seeds are tough because though the same methodology works for 11 seeds as does with 12 seeds as far as picking the most likely underdog winner is concerned, you’re occasionally going to see two 11 seeds win, which again adds to the challenge and luck aspect). But, since everyone already knows that 5-12 games are where the upsets occur, and people disproportionately pick the same underdogs to pull off the upset, rewarding people for picking the right 12 seed upset is BORING, so really, why bother?
Also, I think you’re failing to consider another issue, which is that the rounds do not each occur in a vacuum unrelated to each other. For example, in a 1-2-4-8-16-32 system, each round might have the same number of points, but the probability of, say, getting 75% of the points in Round 4 is astronomically more difficult than getting 75% of the points in Round 1 (not factoring in that we seed teams for a reason), because each successive round builds off of how well you did in the previous round. Thus, if you are in a 1-2-4-8-16-32 system, and you manage to get 32 points in the last round, that rightly ought to be rewarded because the statistical difficulty of that is off the charts.
Now, because there are clear inequalities among tourney entrants (e.g., picking Duke to win the final is far more likely than if you picked, say, the Sun Belt champ coming in with a 17-14 record and an RPI of 145), the scoring has to be adjusted to account for that. However, compensating for that by putting higher weight on upsets only encourages irrational and unjustifiably risky picks. You’re far better off aggregating the influence of win probability (as evidenced in seeding) by round than by game. A good mathematician or statistician could probably calculate something more accurate for you, but my approach would ballpark your scoring system as less exponential than the 1-2-4-8-16-32 model, but it’d still have to be an exponential model nonetheless (as opposed to, say, a 5-10-15-20-25-30 model, which is an additive model of n+5). Let’s assume, for hypothesis sake, that the factor is 1.5 (and round up to the next whole numeral), and you start with a high enough value that the rounding doesn’t skew the results significantly. You would get, for instance, 100-150-225-338-506-759. Maybe that’s too many points, so you go down to 10-15-23-34-51-76. The bottom line is you need a solid algorithm to tell you want the factor should be: 1.1, 1.34, 1.5356, 1.720572, or whatever. Then you just need to translate that into nearest whole numbers without skewing the factor math too much.
All that said, I’m less hung up on how you score the play-in games. Because it’s not a full round and the teams will be fairly evenly matched, I’d be inclined to score them value-wise as close to the first round per-game value. In my example above, they could be worth, say, 9 points (versus 10/1.5 which is 6.67, or rounded up, 7).
I seriously can’t believe I just spent an HOUR debating something as relatively meaningless as NCAA TOURNAMENT SCORING WEIGHTING. Bah.
Wow! I am so impressed with the statistical analysis by Brendan, David and AML. Everything makes perfect sense, but isn’t it intrinsically more “fun” to be rewarded for picking an upset? AML, it is perfectly logical the way you present the case for not rewarding upsets, but I still feel like it makes the contest more interesting, especially for the participant who only “follows” college hoops from Selection Sunday to the first Monday in April. If they feel like they may have a chance picking against an expert like Brendan, they probably enjoy participating more, hence driving up the number of people in the pool.
How about a Fibonacci based scoring system?
1-2-3-5-8-13-21
Its less dramatic than an exponential curve, but more so than a linear scale. If you wanted more point differentiation in each group you could slide where it starts too. Instead of starting at 1, you could start at 3:
3-5-8-13-21-34-55
I largely agree with David and Andrew that upset bonuses will make it essentially impossible for you to compare pools going forward, not just against pools in the past but against other pools with upset bonuses.
However, I do disagree with Andrew significantly that getting all the points in a later round is statistically way more difficult than getting all the points in an earlier round. For one, as he noted, teams are seeded for a reason. For another, let’s consider the case of needing to get all the points in the fourth round:
There are 4 games in the 4th round. Getting all 4 of them correct requires picking those 4 games, plus picking those 4 teams to have gotten there in the first place (3 previous games each), for a total of 16 games picked correctly. Getting all the points in the first round requires picking 32 games correctly. Even if each game’s outcome was a coin flip, it would be statistically easier to get all the points in later rounds than in earlier rounds. This is one of the main reasons I dislike the exponential scoring systems — they place too much emphasis on getting the final winner correct. (For those who care, the minimum number of games which must be picked correctly to get all the points per round: 32, 32, 24, 16, 10, 6).
Such things are inherently more difficult to calculate when trying to see probabilities of getting only a particular percentage of the games right, as there are significantly more combinations which will give the desired percentages, but we can still do minimums. The minimum number of games which much be picked correctly to get 75% of the points per round: 24, 24, 18, 12, NA, NA — NA because there are fewer than 4 games in certain rounds and thus you can’t get exactly 75% of those points.
Anyway, I like the basic numbers Brendan suggested, I’d just drop the bonus points for the upset picks. They’re good if you really want to give people an incentive to pick unusual choices, but they’ll prevent any sort of direct comparison between years other than the dominance of a given bracket metric that Brendan’s used — and even if that is highly dependent on what other people happen to have chosen.
I appreciate all the feedback. I think I’ll go with 1-4-7-10-16-24-41 and continue allowing comparisons as “points out of 477,” albeit with an asterisk thanks to the admittedly different scoring system.
One last comment, your scoring system does basically relegate the first four games to meaninglessness. Getting all four right is the same as getting ONE first round game right. There is also a significant jump in later rounds. The differential between rounds is as follows:
3-3-3-6-8-17
If you go with my proposed system its:
2-3-5-8-13-21
Not that mine is the RIGHT one per se, just that I think your totals are a little weird.
In particular if you look at the percentage of total points in the two systems (mine vs yours) each is decreasing for each round BUT there is a huge jump between the third and fourth rounds in your system (23.48% to 16.77%) vs. my system (20.95% vs 17.02%)
Actually with some parameters you could play around with the numbers to get a system that works with your intent of maintaining a 477 point total but has a better curve/slope to it.
For example: 3-4-7-11-16-22-29 gets you the same 477 point total but shifts emphasis around a little. Or if you wanted the last round to be worth more you could do that too, basing it on percent totals for each round and coming up with numbers based on that.
Oh, one more thing while I’m thinking about it. I think the 477 points should be the total four the normal tournament part and the Play-In-Games should be an additional number on top of that. In the old pool you could get 477 points for picking 63 correct games. Now you are picking 67 correct games. The number of points shouldn’t be equal for picking different amounts of games correctly.
The more I think about it, the more I think you should pick the percentage of total points you want for each round and back fill the points per round based on that.
If you’re going for a 477-point model, I vote for David’s 3-4-7-11-16-22-29 setup. I think that is far closer to maintaining the operational or comparative integrity to the old pool point model of 5-7-10-15-20-25.
Mike, good analysis, I completely overlooked the fact that picking the games correctly in Round 3 only requires getting 4 games right in Round 1, and then 4 games right in Round 2 — all other picks in the first two rounds can be wrong — so getting the same amount of points in Round 3 games is not “astronomically” more difficult than in earlier rounds.
Hmm…. how about 2-4-7-11-16-22-33? Or 3-4-7-11-15-22-33? Or 3-4-7-10-16-24-33? (All of those add up to 477.) Remember, part of my original purpose was to recalibrate things so the later rounds are worth more compared to the early rounds than they have been. I’m using the necessary addition of the First Four as an excuse to finally do that, which I’ve half-wanted to do for the better part of 15 years…
In that case, I prefer 3-4-7-11-15-22-33. I think the play-in games should be worth almost as much as the first round games, so the 1-point increase is good. It’d be nice if there was a greater difference between Rounds 3 and 4 than between Rounds 2 and 3, but that’s a nit more than anything else.
You could also have a second contest and scoring method that may encourage more upset-picking, around the concept of who picks the most games correctly (IOW, each correct pick is worth 1 point, for a total possible of 65). You can then re-score previous years along that basis, and while the previous years’ total possible points is only 63, you can still compare year-to-year by looking at correct pick percentage.
Personal preference, I like the 2-4-7-11-16-22-33. As a second choice, I’d go with 3-4-7-10-16-24-33 or 1-4-7-10-16-24-41. The 11-15 jump feels worse to me than does a 10-16 (16.8 to 13.4% of the points in a 10-16, v 18.4 to 12.6% of the points in an 11-15).
Pingback: Bowtrol Reviews
Pingback: Online casino
I know I’m years late but I’m running a pool, was looking to optimize my scoring, and came upon this old post. Don’t know if anyone’s going to read this, but here it goes:
I agree with much of the analysis posted above: particularly that the 32 pts/round scoring system puts too much emphasis on the later rounds and that awarding points based on seeds and/or upsets is counterproductive.
There was an interesting note about each round not occurring in a vacuum and how “each successive round builds off each other.” With that in mind I’ve done some quick calculations and found that it takes the following number of total correct picks to get 100% of the points in each round: 32-32-24-16-10-6. That is it takes 32 correct picks to get all the 2nd round picks correct, but only 6 correct picks to get the eventual winner correct.
Obviously, as each round goes by it becomes a harder to correctly pick the next winner because any errors you make are compounded. However, if we assume the seeds are a reasonable approximation of the relative strengths of each team then the task of picking the later winners is somewhat simplified. Based on this I’ve decided that the points for each round should be based on the number of correct picks required to get 100% of the points for that round (Pr) and the actual round the game takes place in (Rd).
A quick multiplication of (Pr)*(Rd) divided by the number of games in each round gives the following point values per round: 1-4-9-16-25-36 with total points per round of 32-64-72-64-50-36 and a total of 318 points available. Coincidentally this is the same number of points as (Rd)*(Rd). With this scoring system the knowledgeable player is rewarded by being able to correctly pick rounds 2-3-4 which have typically have the highest possibility of surprises but casual players who pick a lot of chalk can still be rewarded with (seemingly?) significant points in the later rounds.
To expand a bit I’ve also gone through and normalized this system to coincide with a more traditional 32 pts/round scoring system. By adjusting the points to 1-5-11-19-30-48 the total available points is 384 or twice 1-2-4-8-16-32 scoring system but maintains the emphasis on rounds 2-3-4. This adjustment does reduce the importance of round-1 in favor of round-6 but many may see that as a positive.
Of course this ignores the First Four games but most pools I’ve seen give everyone a bye in that round anyway.
If anyone’s out there I’d love to hear you thoughts!