Confirmation Bias, Moneyball, and Next Year
I've been thinking about this for quite a while before putting it into words... so bear with me and take the whole journey before you react.
"Confirmation bias" is a well-known and oft-cited phenomenon in the experimental sciences. It identifies the predisposition of the experimenter to prove his hypothesis with the data prematurely or inaccurately. It hits hardest those softer sciences that deal with surveys or psychological studies. For example, a devoted feminist might find preliminary data leading to suggest violence against women spikes on superbowl sunday (this is famously in error, btw). It might equally lead a fan who likes Sergio to see a Jack turnover as more evidence of a decision already arrived at.
Confirmation bias seems particularly relevant at this juncture in the season when we're starting to look harder at next year than this year. I see confirmation bias in every post that wonders why Jack was playing in the 4th quarter last night (who the heck else?). I see confirmation bias in every defense of Jack in the 4th quarter (for example, in Dave's recap he cites 3 assists--well, I watched the game and know at least one of those assists was a desperation pass after 20 seconds of dribbling against the clock where Outlaw bailed him out with a quick jumper; not the offense I think we want).
It exists in other places as well. Not to inject to much racism in the argument where it's not wanted but I often see someone suggest that we don't "need another white guy." The movie "white man can't jump" and Larry Bird famously suggesting that no white guy can guard him are other examples of a kind of confirmation bias: racial discrimination and prejudice.
So in summary for Part I, I think we--as fans--often come across confirmation bias based upon players who look like they can play, do things we like to watch, or confirm some kind pet theory we have of the way the game ought to be played (a fast pace or a half-court or heavy defense or whatever). We see that all the time here, and certainly saw it last night's press conference where someone with a confirmation bias against Jack and for Surge asked Nate the dreaded question, which only confirmed his confirmation bias about fans.
But that leads us to a more pervasive second kind of confirmation bias and what I really wanted to talk about: expert confirmation bias. Where the most brilliant minds in science suffer from this phenomenon (can anybody say Ptolemaic Epicycles?), certainly we should expect coaches, sportswriters, and even GMs to suffer from this as well. What does Nate see when he watches a game? How about Stephen A Smith or Bill Walton or Charles Barkley? Certainly, they know more about the game and their judgments are more considered. But it would be extremely naive to think that Nate doesn't see a defensive lapse from Surge differently from Jack's; or a failed free throw from Joel a little differently than Webster. Players playing for coaches have to bring more evidence to bear to defeat a predisposition than brand new player out of the draft. This is why I think Surge will probably always be a poor fit for Nate and why Joel and Raef might get limited minutes even should they badly outplay Oden. I think when we here "development" and "time" these terms are used to defend decisions made against evidence contrary to confirmation bias. Not that these aren't very relevant issues but, really, how much time and development is needed? Many times, I suggest, this time is longer for those at the positive end of the confirmation bias: players who look like they can play.
This brings us to Moneyball. For those of you who've read the work you know that it's about statistics overcoming confirmation bias. The book, chronicling the Oakland A's, is filled with examples of scouts and coaches and GMs who fail to assess players properly because they do or do not have that look." "He just *looks like a gifted athlete." Meanwhile, capable players are being passed because they look awkward or have an unimpressive physique. Granted, baseball doesn't command the kind of athletes that basketball does but the example is still extremely relevant.
So, we use statistical data to overcome confirmation bias, as any good fantasy owner knows. In this objective world, we can overlook glitzy performance and get at the brass tacks, no?
The basketball box score is nice because it's ubiquitous and universal. After that, I think we can all agree that it's a very poor assessment of a player's contributions. Animart1 has introduced us to the "fouls drawn" stat in Europe. We have steals that don't credit the deflection, rebounds that make no distinction between contested and uncontested balls, shots that are credited equally if wide open or not, judgment calls around assists, and no good measure of defense. Any time we use the box score to credit an argument, we're using a flawed system that can do nothing to contradict ingrained confirmation bias. That's why Oakland in Moneyball was successful: they used a more credible statistical system.
For all the controversey, the +/- point system is an attempt to get a more credible measure. I know people have lots of trouble with it, but I like very much the gesture behind it and what it's trying to do. Is there a better way?
That's why I'm so fascinated with KP's algorithm: presumably, he has a better way of assessing players. I often wonder what his system is. I often wonder about what his system has to say about Surge, Jack, Web and Joel. I wonder if his algorithm is in conflict with Nate, who seems a sure-fire kind of guy who trusts his instincts and gut while watching a player and would have little time for statistial nonsense. What does the system have to say about development and future potential? What is the score that "culture" brings to the argument?
And what do we do as fans? Well, to be good fans, I think we need to do what any good scientist does: look specifically for evidence contrary to your hypothesis. I try to watch for the good that Jack does and the defensive lapses of Surge. I think Dave and other Jack defenders need to look hard at not just the number but the kind of turnovers that Jack coughs up. (In reality, I'm in a place now where I don't think the Blazers have a good PG and it's pointles to speculate on one or the other.)
More to the point, we need to agree as fans what sort of evidence is required to recommend a firm conclusion. That's the conversation I'd like to have... can Blazer's Edge create a whole new Algorithm of assessing talent? It has worked in the baseball world and we have enough talent (and obsessive slavish devotion) to make it work. I think of the following:
- Unassisted baskets. Good morning, Travis Outlaw! I want to know what players can command a basket when they wish and, more to the point, who operates outside of the flow of the offense.
- Offense creation. What passes lead to open looks at the basket or free throws? Right now, assists is sadly lacking in generating these kinds of issues.
- Careless / Dumbass turnovers. Sometimes, a TO is under pressure, sometimes the results of boneheadedness (I saw this last night a few times).
- Pressured TOs. Who can handle the pressure?
- Deflections. Something we already keep in some circles.
- Fouls drawn (I love that... I'd also like to know if they're assisted or producede fully formed from the player himself.)
- Box outs. That's at least as important as the rebound itself.
- Help defense. Can we track that somehow?
- Contested shots. Certainly this matters a great deal.
- Contribution to an offensive set. Is it obvious when a basket is made as part of a play and can that be quantified?
So those are my thoughts. I'd like to see us in the BE community argue less about what we think we saw, less about the reported box score statistics, and more about what we should be looking for to come to firm conclusions. This is already a very intelligent site as blogs go... and I think it can get a lot better. In fact, if we can figure out a way to bring stats-keeping together on this website, we could change how the game itself is ultimately reported. It has happened in other places.
0 recs |
25 comments
Comments
Actually I think he said
by bubba on Feb 25, 2008 2:38 PM PST reply actions 0 recs
Optimist in a room full of poop...
<kidding>
by DonkeyShins on Feb 25, 2008 2:41 PM PST reply actions 0 recs
It's a good post, though
I have a neural net program in which you can plug in lots of numbers (sport stats, stock quote etc) which are precursors to a result. You can then train the program to use the stats to replicate the result.
You might then be able to take a look back and find out which stats ended up most heavily weighted - most predictive of the results - after the program finished training.
This would take way more free time than I have, but it's doable.
by bubba on Feb 25, 2008 2:54 PM PST reply actions 0 recs
If it could do that
by bubba on Feb 25, 2008 3:23 PM PST up reply actions 0 recs
Neural net
by PoliSam on Feb 26, 2008 7:54 AM PST up reply actions 0 recs
They're used in a lot of places
by bubba on Feb 26, 2008 9:54 AM PST up reply actions 0 recs
Toll-Eh-May-Ick Epp-E-Sigh-Culls
First off, I'd like to direct the rest of you statheads to 82games.com. They track all kinds of unusual statistics there that you won't find in the box score. Most interestingly they track on court / off court stats in addition to + / -. For example, you can see how the team's rebounds fluctuate when a certain player is on the floor, this takes into account tipped balls and box outs not just controlled balls. There are certainly still some confounding factors in the measures but they provide interesting color to the existing statistical metrics.
Any statistic in a vacuum is meaningless, you need the variety of stats to get an idea of what is going on. Furthermore, many statistics give you different information about a player than you would think. For example, FG% isn't actually a measure of a player's shooting ability, because it also measures a players shot selection.
http://www.82games.com/0708/0708POR.HTM
Some of the stats you want to see can be found over there (at least those that can be objectively quantified), like assisted buckets and drawn fouls. Others can be gleaned from some more statistical comparisons of various primary statistics.
By the way, 64% of Travis' buckets are assisted, which is low for a forward, as I'm sure you expected. For comparison's sake:
Player / Assisted FG%
Sergio / 33%
Brandon / 36%
Jack / 44%
Blake / 59%
Travis / 64%
Joel / 67%
Lamarcus / 68%
Martell / 78%
Frye / 79%
Jones / 90%
by Bretski on Feb 25, 2008 3:40 PM PST reply actions 0 recs
That's a very interesting post
I'm pretty sure the teams havestatistics about deflections and fouls drawn. I think it would be easy for the NBA to have at least the fouls available to everyone.
Some of the other statistics you mentioned have the problem that they are "too much" open to interpretation. That's why scouting is important.
I think that stats help but scouting and coaching is more of an Art than Science. Stats and data help you make an educated guess.
by Bruno on Feb 25, 2008 3:54 PM PST reply actions 0 recs
good post
you touch on this, but my first thought is that coaches and general managers are definitely the guiltiest when it comes to confirmation biases, self-fulfilling prophecies, etc, not only in the NBA, but college, high school and the youth levels as well. ultimately, they have the most riding on their evaluations of players and they can easily become the desperate gambler: with the most information about the players and with the most to lose, they are willing to push the limits of reason to attempt to obtain the result they envision.
using your example, i think Nate genuinely wants Jack to be his point guard. barring a trade, i think that regardless of how he plays, Jack will continue to get more minutes than Blake and Sergio.
this creates a tough situation for all involved, including fans.
all things considered, I wonder if fans are less guilty of the confirmation bias that you describe than we might think. generally, I think fans are most likely to determine their opinions of individual players based on 1) are they helping the team win? or 2) are they exciting? or 3) are they full of potential? I also think that most fans, especially those that don't follow a team 24-7, are more apt to change their opinions of a player than coach/management are, especially over the course of a season. for professional reasons (monetary, reputation, etc.), I don't think coaches/management have this luxury or tendency.
will an increased statistical look at individual performance help overcome this? i think the jury is still out. but i think your call for a new look at how we view team/individual success is warranted. aside from the stats you mentioned, is there anything Dave or myself can add to our coverage of the team that you think would speak to that point and better focus the discussion?
as an aside, you are right to point to race as a factor for creating confirmation bias; I would add weight, condition, height, and others to the list as well. not just in basketball, but in life.
just a few pennies from me.
by Ben. on Feb 25, 2008 4:04 PM PST reply actions 0 recs
Bias
Especially given relatively short college careers, I suspect NBA teams quantitatively analyse each NCAA game and code plays for value. i.e. hard baseline screen, 12 points. Indecicive defensive rotation, -5 points, calling out a screen on D, 3 points, rushed shot, -4 points, etc. Obviously, some of this is highly subjective, but I wouldn't be surprised if each game is gone through and the plays coded to give a per-minute feel of each thing a player does.
I actually pretty amazed that folks read as much as they do into the limited box scores we get as fans. After all, KP saw something in James Jones' awful stats with the suns...
by Engineering Problem on Feb 25, 2008 4:12 PM PST reply actions 0 recs
Great post Fizbin!
by drawingjeremy on Feb 25, 2008 4:16 PM PST reply actions 0 recs
Phistophicles and Amlmart1
http://www.youtube.com/watch?v=mnodPfR1X2U
by amlmart1 on Feb 25, 2008 7:34 PM PST reply actions 0 recs
Amazing post fizbin!
by LaMarvelous on Feb 25, 2008 10:32 PM PST reply actions 0 recs
I just realized the the time...
Rod Serling has come back from the grave to haunt me. Rod? Rod?
by LaMarvelous on Feb 25, 2008 10:36 PM PST up reply actions 0 recs
Exactly right
by dp8039 on Feb 26, 2008 7:44 AM PST reply actions 0 recs
An excellent post.
Then we have the problem of player development. Blake is the best point guard of the three but is closer to his final development point.
I am a ridiculous Rodriguez fan, his is the first jersey I have ever bought for myself, but Jack is presently somewhat better as a player than Rodriguez. However, he's also three years older with enormously more time on the floor.
So the algorithm on who to play, and how much is even more complicated than who is the most effective. Right now we are trying to work out who to lay now, for two years from now. Add in the problem for coaches that losing games gets them fired, while developing young players makes GM's reputations and it becomes very complicated.
by EnglandDan on Feb 26, 2008 7:55 AM PST reply actions 0 recs
Interesting
I've mentioned before that I've seen turnovers called "bad pass" when really the receiver fumbled. OTOH someone with "great hands" might have been able to corral it. To me (and here is the subjective part) it looked like an error on the part of the receiver. But is it still a bad pass or a fumble? Or a bad pass because the passer should have known that the receiver might not be able to handle it? Category for "bad decision"?
I think that the coaches (and KP) look beyond play when valuing a player. By all reports, Jack is a real winner in the locker room. He's definitely a leader on the floor - he talks constantly to his teammates. He was one that the Mikes said would probably become a coach. These things may weigh heavily with the coach when he's setting his lineup. Maybe Jack has lots of these transparent plusses. (Transparent to fans, highly visible to the coaching staff.)
I also see a need to remind people that Allen was the one who wanted Sergio. When it came time to buy the pick during the draft everyone (including KP) said "No". Allen said "yes" so the deal was done. (His money after all.) Remember Allen vetoed a Miles deal. He may be behind other roster decisions too.
by jorga on Feb 26, 2008 8:19 AM PST reply actions 0 recs
That's a good point
I think Allen needs to stay out of that process and let KP come to him when he needs the cash. I want that championship and I strongly feel that KP is the way.
by ratbastird on Feb 26, 2008 9:03 AM PST up reply actions 0 recs
A lot of work...
However, "We need to agree as fans what sort of evidence is required to recommend a firm conclusion". That's not going to happen. I think within your own diary you outlined pretty well several reasons why fans are NEVER going to agree on what sort of evidence.
Confirmation Bias potential aside, most fans are reactionary and empirical in forming opinions. Not all, but many fans don't go past the last game or the last series of plays they remember to form an opinion on a coach or player. I'm afraid if you are looking for agreement among fans you will never find it. Fantasy leagues have probably brought more fans to start looking at stats more closely but overall a fan is still going to react more to the "facts" that he "thinks" he see when watching a game then numbers on a spread sheet.
Very interesting diary, well done.
by Krang on Feb 26, 2008 8:55 AM PST reply actions 0 recs
Two parts
I see two parts:
Part one is that you state people see what they expect to see and want to see and that's not always accurate.
I agree with this and this has been going on since our first ancestors decided it'd be a good idea to get out of the water, dry off, get a tan, and go climb some trees to see the world better and start tea parties where they socialized with others.
Part Two is a better need for stat measurements in general but with a secondary purpose of helping to counterbalance the bias.
I think KP has this, so I'm not so concerned about the counterbalance. I agree that I, as the average fan, would LOVE those stats you meantioned.
Hey dave, maybe you should tell that advisory board that we'd love us some serious blazer stats! Then again that would be feeding info to the enemy, and I like them in the dark as we feast upon the rotted corpse of their stupidity.
If other teams are down, I say wave hello from up high and much on some popcorn and then point and laugh.
But MAN would i love those stats.
by ratbastird on Feb 26, 2008 9:15 AM PST reply actions 0 recs
Most Important Stat is OBP.
The most comparative stat in basketball is turnovers. Keep the turnovers low and shoot more. Unfortunately it doesn't work if the players can't shoot or get to the free throw line. For each player, a plays created should be a stat. Tipped balls tap outs, swing pass, penetration dish, good outlet pass etc. Plays created must lead to a good shot attempt or FTs.
BRoy may only 7 assists, but his plays created could be upwards to 20.
So I'm guessing some combination of the 4 stats, turnovers, shooting percentage, FTs, and plays created together for an offensive rating. Scary thought, just thinking about this, Jarrett Jack may be our second best player on the team right now. No wonder we aren't very good.
by blzrfan on Feb 26, 2008 9:31 AM PST reply actions 0 recs
The stat in basketball is points per possession.
I think turnovers get over emphasized. Is a pass that has a 25% chance of getting picked off but if completed results in a dunk 90% of the time a good pass? The answer is that statistically (.75x.9=.665, a great shooting percentage) it's a fantastic play but most coaches hate it.
Early in the season with Sergio Rodriguez playing we were giving up a ton of points at a high percentage, because Sergio was taking lots of risks. Nate benched him and he has not played that style since. However, during that period Sergio had by far the best +/- on the team because we scored so efficiently when he was playing that style. Just an example.
by EnglandDan on Feb 26, 2008 11:19 AM PST up reply actions 0 recs
I don't thinks points per possession works
by blzrfan on Feb 26, 2008 2:52 PM PST up reply actions 0 recs
I guess I should forgive you for suggesting
by tweener on Feb 26, 2008 9:44 AM PST reply actions 0 recs
Funny you should mention...
Thus the need for better objective measures.
by Fizbin on Feb 26, 2008 10:16 AM PST up reply actions 0 recs

by 



















