Get the ranks that have dominated
the fantasy basketball industry for over a decade.

The Bruski 150 Evaluation Process

In a truly grueling and inane exercise, we look at the Bruski 150 to find out how we did in a few different lights but ultimately come up with some easy head-to-head analysis of the B150 ranks vs. Rotoworld's ranks (*Note: As mentioned in the open, they only had general ranks without a differentiation between 8- and 9-cat, we're just living with this for this season and reviewing how to assess next season). We also attempt to add some deeper meaning and value to the analysis by weighting ranking wins and losses by the impact they have, which has both qualitative and quantitative evaluations. In each of these ranking analysis I welcome anybody to offer a counterpoint, but the key for making this fair and worthwhile is to be brutally honest with the assessments (which also keeps me well on the side of fair so I can just let this fly and know it's legit).

What we’re looking to determine is which rank was the smarter rank, accounting for the totality of the situation.

The first and broadest analysis is a rank-by-rank assessment which determines an overall record between the two sites. After all, each of the ranks matter in some context so zooming out to see what the aggregate win-loss records are is a good way to show an overall strength of ranks. It also keeps a few good or bad ranks from swinging the analysis.

Then we look at the ranks while accounting for how important a given prediction was. First we do this by assigning an impact rating. Then when looking at the ranks in relation to ADP, we’re looking for how likely or unlikely was a site’s followers to get the good (or bad) pick, which we end displaying on the graphic below by color (it's just easier to read that way). Therefore, we call that the color rating.

We multiply the impact rating with the color rating to create a spectrum of outcomes that are a dart throw at emulating year-long profit and loss scenarios, which are the essence of preseason rankings.

So in summary:

• Head-to-Head win/loss totals for the aggregate picture
• Impact Ratings give a more quantitative weighting for predictions and Color Ratings allow for ADP, rank differences and common sense to create a qualitative rating for predictions.
• To create this Impact Score we have a simple scoring system that multiplies these two ratings together and then aggregates the data set

Diving Deeper Into the Evaluation

COLOR RATINGS

The color schemes are:

• Dark Green (massive win involving a player that performed very well relative to ADP and/or the other site, easily had opportunity to draft/avoid that player)
• Green (a general rankings win, better positioned to draft or miss a player who over/under performed, when evaluating both sites and ADP)
• Yellow (painful loss, rankings prediction put drafters in likely position to move the needle backward with their team)
• Red (brutal loss, rankings prediction hurt drafters in significant ways, missing the mark badly when chance wasn’t a factor)

Another way to look at it might be:

• Dark Green (you did a real good thing)
• Green (you beat the other site/ADP in a way that wasn’t totally negligible)
• Yellow (your rank moved the needle backwards for squads)
• Red (you did a real bad thing)

How they get scored:

• Dark Green (+4)
• Green (+2)
• No Color (1)
• Yellow (-2)
• Red (-4)

Not all prediction wins are created equally. Some are dumb luck and have massive impact, which isn’t the sign of a good prediction, and other great predictions have smaller impacts but deserve more credit. If there was an uncontrollable event not tied to obvious injury risk, that may not get an assessment. In cases where a player without known injury risk is performing as a win and then bad luck hits, nine times out of 10 that's going to result in a win for the better predictor there (rather than strict allegiance to final rank).

On this front we want to look at the nature of injuries. Were they something that we could have known about? Were they factored into the draft situation as a risk-reward play? If a player got extremely lucky due to unforeseen injuries ahead of him, we’re not trying to reward or punish predictions as much as we would a prediction that’s based on known variables — one that reflects greater understanding of stat sets, usage rates and the like.

Mix that all up and then everything gets weighed out in context, but at its core these color ranks are measuring how much gain and loss occurred in terms of pure value.

Each rank and evaluation is given the type of scrutiny you’d want to have if you could turn back time and do it all over again.

As we go further down in the draft, when player values start to bunch up, the grading loosens up a tiny bit and color grades won’t reward mild differences. At the same time a sleeper that can crawl up into early round value would get rated as a high impact.

Again, the key to this is to be brutally harsh with myself and give my competition benefit of the doubt when evaluating these predictions.

It’s entirely possible I have screwed up on a piece of logic in an example in an attempt to be expedient. I’m pretty sure any shifting results will be within a reasonable margin of error and not take away from the findings.

If you see anything hugely off, just let me know and I'm happy to make adjustments.

IMPACT ANALYSIS

The impact analysis seeks to determine whether the prediction put the drafter in the position for a gain, avoid a loss and to what degree — and then it aggregates that for the entire prediction set.

As for the impact analysis itself, it is also qualitative to some degree but it does trend toward ‘just the facts.’ It's qualitative in the sense that if a prediction win didn't beat ADP, that's not a very impactful play, which happens when ADP actually wins out (it happens!). From there, we’re measuring how much distance was there between the predictions and the results.

That scale from 1-5 — it’s really just 1-4 as a grade of 5 is for Hall of Fame level needle-movers that occur maybe once in a season if they occur at all. Let's look at the scale:

5: Historical result
4: Prediction leads to extremely important high-end production, or a large number of rounds in the ballpark of a half-draft or greater of increased (or decreased) value for their fantasy GM
3: Prediction leads (or loses) very important high-end production, or results in gaining or losing a significant number of rounds of value (in the 4-8 range depending on how early or late the player in question was drafted)
2: Prediction secures substantial increased (or decreased) value for their fantasy GM (2-4 rounds worth of value dependent upon the facts)
1: A basic head-to-head win that doesn't meet the criteria above is a 1

No players from the last two years received the fabled ‘5.’ Only one player got a 5 a few seasons ago and that was James Harden who nearly lapped the entire field in 8-cat.

Finally, Here are the Receipts

***CLICK THE IMAGE TO CHECK IT OUT***