NerdRank - Inside the rankings of the Fantasy Football Nerd (FFN)
How do you compile your rankings and determine where a player is ranked?
I get this question a lot, so I'm going to provide an overview of how the Nerd works.
Beginning in the 2009 season, the rankings system within the FantasyFootballNerd.com became both more complex and simplified at the same time. If you see similarities to the process and terms below, that's because much of it was patterned after Google. They simply have incredible technology and I felt that their logical approach would benefit the Nerd's internal ranking system. Here's a peek under the hood of the FFN.
Stages
There are three main components to the Nerd's ranking system.
- Nerd Crawler: the actual search engine that does the aggregating of rankings from all the websites.
- Quality Score: a dynamic, internal assessment of each source within the Crawler's index.
- NerdRank: the calculation for each player based upon the rankings from the Crawler and the Quality Score achieved by that source.
Inside the Crawler
The Nerd Crawler is quite fast and very efficient. It's job is to simply visit each site and report back on the rankings that each site has applied to all of the players. The details from every crawl are recorded for use down the line. When you search on FFN, you're searching our cached results so that the Crawler can remain bandwidth-friendly to all of our sources.
Inside the Quality Score
Here's where things got more complicated from the original Nerd engine. There are more than a dozen factors that go into assigning a Quality Score for each source. The primary factor, however, is accuracy. Based on past performance, how accurate was this source? How did this source trend during the season (ie, draft rankings vs season performance, weekly projections vs that week's performance, etc.)?
The higher the Quality Score, the more "votes" that site gets during the final stage of calculation. The Quality Score is the secret sauce to the Nerd. I'll open up more details regarding the rest of the process below, but the Quality Score remains central to NerdRank.
The Quality score is very important because it allows the Nerd to normalize the data a bit. Imagine that you had a basket full of apples. If you saw one or two bad apples, you might conclude that the basket is bad. If however, the bad apples were pushed to the bottom (or taken out of the basket altogether), you would most likely conclude that the basket is good. One or two bad apples cannot alter the final NerdRank in such a manner to be classified as significant as long as there are enough quality apples in the basket. Too vague? (I'm not very good at analogies). Basically, if a source site performs poorly and their projections are either radical or detrimental to the consensus, a Quality Score limits their impact on the NerdRank.
Inside the NerdRank
The NerdRank is the process that combines all the data from the previous steps. The NerdRank formula in the current version of the Nerd is much simpler than the original version released in 2008. Essentially, it multiplies each source rank (r) and the associated Quality Score (QS). It then takes the sum of the weighted rankings and divides it by the sum of the Quality Scores (QS).
What isn't shown in this basic formula (image above) are the other variables, but they may have an influence on the final outcome. The standard deviation is very important here. From Wikipedia A low standard deviation indicates that the data points tend to be very close to the same value (the mean), while high standard deviation indicates that the data are "spread out" over a large range of values. If Tom Brady's standard deviation is low, this means that the sources all tend to agree that he belongs at that ranking. If, however, we see a higher standard deviation, we can conclude that there isn't much consensus on where he fits.
Trending is also very important. Remember earlier when we mentioned how the Nerd Crawler keeps a record of each player from each source from each visit? Well, we can use that data to see how the sources feel about a certain player over a period of time and we can compare that to other players in the same tier.
Conclusion
Let's be honest. 99% of the people who use the Nerd on a daily/weekly basis don't want or care to know how it works. That's why the Nerd converts the "nerdy-things" into easy to use visual displays. Rankings are from 1 (good) to 5000 (bad). Charts are great for showing the NerdRank in relation to the highs and lows from the various sources. A color coded standard deviation icon (green is good, red is bad) is much simpler than knowing that the standard deviation is 2.86 and the standard error of the mean is 0.904.
If you've made it this far, congratulations. While I haven't provided you with a complete blueprint, I hope you gained a better understanding of how the Nerd works. There will continue to be enhancements made along the way, and I wouldn't be surprised to see a different alogrithm in place in the future.
(If you're a competitor, good luck. Perhaps one day I'll be using your technology!)
Good luck this season!
J. Joseph Dyken
FantasyFootballNerd.com