How do you compile your rankings and determine where a player is ranked?
I get this question a lot, so I'm going to provide an overview of how the Nerd works.
Beginning in the 2009 season, the rankings system within the FantasyFootballNerd.com became both more complex and simplified at the same time. NerdRank borrows not only the name but the same basic tenets of Google's PageRank in that some sources are considered more authoritative than others. Here's a peek under the hood of FFN.
There are three main components to the Nerd's ranking system.
The Nerd Crawler is quite fast and very efficient. It's job is to simply visit each site and report back on the rankings that each site has applied to all of the players. The details from every crawl are recorded for use down the line. When you search on FFN, you're searching our cached results so that the Crawler can remain bandwidth-friendly to all of our sources.
There are more than a dozen factors that go into assigning a Quality Score for each source. The primary factor, however, is accuracy. Based on past performance, how accurate was this source? How did this source trend during the season (ie, draft rankings vs season performance, weekly projections vs that week's performance, etc.) and how did each expert do overall across all positions as well as by individual position? Our research shows that some experts are consistently better by specific player position. That's important when calculating their Quality Score and applying that QS to their projections.
The higher the Quality Score, the more "votes" that site gets during the final stage of calculation and the more influence it has over the final outcome. Every site gets a chance to "vote", but the sites that are accurate more often get more "votes" in the final tally. NerdRank considers them to be more authoritative.
Additionally, if a source site performs poorly and their projections are either radical or detrimental to the consensus, a Quality Score limits their impact on the final outcome.
The NerdRank is the process that combines all the data from the previous steps. Essentially, it multiplies each source rank (r) and the associated Quality Score (QS). It then takes the sum of the weighted rankings and divides it by the sum of the Quality Scores (QS). The most accurate experts get "juice" points as well, which is an homage to both the concept (power and respect) of the 90's film with the same name as well as my fraternity nickname while I was in college. The additional votes are attained by being the most accurate of the group.
What isn't shown in this basic formula (image above) are the other variables, but they will have an influence on the final outcome. The standard deviation is very important here. From Wikipedia A low standard deviation indicates that the data points tend to be very close to the same value (the mean), while high standard deviation indicates that the data are "spread out" over a large range of values. If Tom Brady's standard deviation is low, this means that the sources all tend to agree that he belongs at that ranking. If, however, we see a higher standard deviation, we can conclude that there isn't much consensus on where he fits.
Trending is also very important. Remember earlier when we mentioned how the Nerd Crawler keeps a record of each player from each source from each visit? Well, we can use that data to see how the sources feel about a certain player over a period of time and we can compare that to other players in the same tier.
Let's be honest. 99% of the people who use the Nerd on a daily/weekly basis don't want or care to know how it works. That's why the Nerd converts the "nerdy-things" into easy to use visual displays. Rankings are from low (good) to high (bad). Charts are great for showing the NerdRank in relation to the highs and lows from the various sources. A color coded standard deviation icon (green is good, red is bad) is much simpler than knowing that the standard deviation is 2.86 and the standard error of the mean is 0.904.
I hope you gained a better understanding of how the Nerd works. There will continue to be enhancements made along the way, and I'm always interested in hearing how it's working for you.
Good luck this season!
J. Joseph Dyken
Founder of FantasyFootballNerd.com