Blazersedge: An SB Nation Community

Navigation: Jump to content areas:


Sports blogs for fans, by fans.
Around SBN: Cal RB Jahvid Best Seriously Injured, Carted Off Field

Confirmation Bias, Moneyball, and Next Year

I've been thinking about this for quite a while before putting it into words... so bear with me and take the whole journey before you react.

"Confirmation bias" is a well-known and oft-cited phenomenon in the experimental sciences. It identifies the predisposition of the experimenter to prove his hypothesis with the data prematurely or inaccurately. It hits hardest those softer sciences that deal with surveys or psychological studies. For example, a devoted feminist might find preliminary data leading to suggest violence against women spikes on superbowl sunday (this is famously in error, btw). It might equally lead a fan who likes Sergio to see a Jack turnover as more evidence of a decision already arrived at.

Confirmation bias seems particularly relevant at this juncture in the season when we're starting to look harder at next year than this year. I see confirmation bias in every post that wonders why Jack was playing in the 4th quarter last night (who the heck else?). I see confirmation bias in every defense of Jack in the 4th quarter (for example, in Dave's recap he cites 3 assists--well, I watched the game and know at least one of those assists was a desperation pass after 20 seconds of dribbling against the clock where Outlaw bailed him out with a quick jumper; not the offense I think we want).

It exists in other places as well. Not to inject to much racism in the argument where it's not wanted but I often see someone suggest that we don't "need another white guy." The movie "white man can't jump" and Larry Bird famously suggesting that no white guy can guard him are other examples of a kind of confirmation bias: racial discrimination and prejudice.

So in summary for Part I, I think we--as fans--often come across confirmation bias based upon players who look like they can play, do things we like to watch, or confirm some kind pet theory we have of the way the game ought to be played (a fast pace or a half-court or heavy defense or whatever). We see that all the time here, and certainly saw it last night's press conference where someone with a confirmation bias against Jack and for Surge asked Nate the dreaded question, which only confirmed his confirmation bias about fans.

But that leads us to a more pervasive second kind of confirmation bias and what I really wanted to talk about: expert confirmation bias. Where the most brilliant minds in science suffer from this phenomenon (can anybody say Ptolemaic Epicycles?), certainly we should expect coaches, sportswriters, and even GMs to suffer from this as well. What does Nate see when he watches a game? How about Stephen A Smith or Bill Walton or Charles Barkley? Certainly, they know more about the game and their judgments are more considered. But it would be extremely naive to think that Nate doesn't see a defensive lapse from Surge differently from Jack's; or a failed free throw from Joel a little differently than Webster. Players playing for coaches have to bring more evidence to bear to defeat a predisposition than brand new player out of the draft. This is why I think Surge will probably always be a poor fit for Nate and why Joel and Raef might get limited minutes even should they badly outplay Oden. I think when we here "development" and "time" these terms are used to defend decisions made against evidence contrary to confirmation bias. Not that these aren't very relevant issues but, really, how much time and development is needed? Many times, I suggest, this time is longer for those at the positive end of the confirmation bias: players who look like they can play.

This brings us to Moneyball. For those of you who've read the work you know that it's about statistics overcoming confirmation bias. The book, chronicling the Oakland A's, is filled with examples of scouts and coaches and GMs who fail to assess players properly because they do or do not have that look." "He just *looks like a gifted athlete." Meanwhile, capable players are being passed because they look awkward or have an unimpressive physique. Granted, baseball doesn't command the kind of athletes that basketball does but the example is still extremely relevant.

So, we use statistical data to overcome confirmation bias, as any good fantasy owner knows. In this objective world, we can overlook glitzy performance and get at the brass tacks, no?

The basketball box score is nice because it's ubiquitous and universal. After that, I think we can all agree that it's a very poor assessment of a player's contributions. Animart1 has introduced us to the "fouls drawn" stat in Europe. We have steals that don't credit the deflection, rebounds that make no distinction between contested and uncontested balls, shots that are credited equally if wide open or not, judgment calls around assists, and no good measure of defense. Any time we use the box score to credit an argument, we're using a flawed system that can do nothing to contradict ingrained confirmation bias. That's why Oakland in Moneyball was successful: they used a more credible statistical system.

For all the controversey, the +/- point system is an attempt to get a more credible measure. I know people have lots of trouble with it, but I like very much the gesture behind it and what it's trying to do. Is there a better way?

That's why I'm so fascinated with KP's algorithm: presumably, he has a better way of assessing players. I often wonder what his system is. I often wonder about what his system has to say about Surge, Jack, Web and Joel. I wonder if his algorithm is in conflict with Nate, who seems a sure-fire kind of guy who trusts his instincts and gut while watching a player and would have little time for statistial nonsense. What does the system have to say about development and future potential? What is the score that "culture" brings to the argument?

And what do we do as fans? Well, to be good fans, I think we need to do what any good scientist does: look specifically for evidence contrary to your hypothesis. I try to watch for the good that Jack does and the defensive lapses of Surge. I think Dave and other Jack defenders need to look hard at not just the number but the kind of turnovers that Jack coughs up. (In reality, I'm in a place now where I don't think the Blazers have a good PG and it's pointles to speculate on one or the other.)

More to the point, we need to agree as fans what sort of evidence is required to recommend a firm conclusion. That's the conversation I'd like to have... can Blazer's Edge create a whole new Algorithm of assessing talent? It has worked in the baseball world and we have enough talent (and obsessive slavish devotion) to make it work. I think of the following:

  1. Unassisted baskets. Good morning, Travis Outlaw! I want to know what players can command a basket when they wish and, more to the point, who operates outside of the flow of the offense.
  2. Offense creation. What passes lead to open looks at the basket or free throws? Right now, assists is sadly lacking in generating these kinds of issues.
  3. Careless / Dumbass turnovers. Sometimes, a TO is under pressure, sometimes the results of boneheadedness (I saw this last night a few times).
  4. Pressured TOs. Who can handle the pressure?
  5. Deflections. Something we already keep in some circles.
  6. Fouls drawn (I love that... I'd also like to know if they're assisted or producede fully formed from the player himself.)
  7. Box outs. That's at least as important as the rebound itself.
  8. Help defense. Can we track that somehow?
  9. Contested shots. Certainly this matters a great deal.
  10. Contribution to an offensive set. Is it obvious when a basket is made as part of a play and can that be quantified?
These are just thoughts I have. If I could command a genie to give me the perfect box score, it would contain statistics like this.

So those are my thoughts. I'd like to see us in the BE community argue less about what we think we saw, less about the reported box score statistics, and more about what we should be looking for to come to firm conclusions. This is already a very intelligent site as blogs go... and I think it can get a lot better. In fact, if we can figure out a way to bring stats-keeping together on this website, we could change how the game itself is ultimately reported. It has happened in other places.

0 recs  |  Comment 25 comments

Story-email Email Printer Print

Comments

Display:

Actually I think he said
that you can mistake a turd for a gold nugget if you believe that gold is brown and squishy.

by bubba on Feb 25, 2008 2:38 PM PST reply actions   0 recs

Optimist in a room full of poop...
...there's gotta be a pony in here somewhere!

<kidding>

by DonkeyShins on Feb 25, 2008 2:41 PM PST reply actions   0 recs

It's a good post, though
I think it's worth looking at.

I have a neural net program in which you can plug in lots of numbers (sport stats, stock quote etc) which are precursors to a result. You can then train the program to use the stats to replicate the result.

You might then be able to take a look back and find out which stats ended up most heavily weighted - most predictive of the results - after the program finished training.

This would take way more free time than I have, but it's doable.

by bubba on Feb 25, 2008 2:54 PM PST reply actions   0 recs

If it could do that
I can think of measurments I'd entering that had nothing to do with basketball or the stock market.

by bubba on Feb 25, 2008 3:23 PM PST up reply actions   0 recs

Neural net
For what profession do do you use neural networks? I'm curious because they are starting to get some use in political science and one of the faculty at my department is a big proponent of them, but she hasn't really sold  me on their use.  

by PoliSam on Feb 26, 2008 7:54 AM PST up reply actions   0 recs

They're used in a lot of places
where pattern recognition is important. I'm not an expert on their real world applications, I have a neural net program I use for fun (trying to predict point spreads for football and basletball games, actually) But it's been a few years since I've messed with it much.

by bubba on Feb 26, 2008 9:54 AM PST up reply actions   0 recs

Toll-Eh-May-Ick Epp-E-Sigh-Culls
Interesting diary Fizbin.

First off, I'd like to direct the rest of you statheads to 82games.com. They track all kinds of unusual statistics there that you won't find in the box score. Most interestingly they track on court / off court stats in addition to + / -. For example, you can see how the team's rebounds fluctuate when a certain player is on the floor, this takes into account tipped balls and box outs not just controlled balls. There are certainly still some confounding factors in the measures but they provide interesting color to the existing statistical metrics.

Any statistic in a vacuum is meaningless, you need the variety of stats to get an idea of what is going on. Furthermore, many statistics give you different information about a player than you would think. For example, FG% isn't actually a measure of a player's shooting ability, because it also measures a players shot selection.

http://www.82games.com/0708/0708POR.HTM

Some of the stats you want to see can be found over there (at least those that can be objectively quantified), like assisted buckets and drawn fouls. Others can be gleaned from some more statistical comparisons of various primary statistics.

By the way, 64% of Travis' buckets are assisted, which is low for a forward, as I'm sure you expected. For comparison's sake:

Player  / Assisted FG%

Sergio   / 33%
Brandon  / 36%
Jack     / 44%
Blake    / 59%
Travis   / 64%
Joel     / 67%
Lamarcus / 68%
Martell  / 78%
Frye     / 79%
Jones    / 90%

by Bretski on Feb 25, 2008 3:40 PM PST reply actions   0 recs

That's a very interesting post

I'm pretty sure the teams havestatistics about deflections and fouls drawn. I think it would be easy for the NBA to have at least the fouls available to everyone.
 Some of the other statistics you mentioned have the problem that they are "too much" open to interpretation. That's why scouting is important.

  I think that stats help but scouting and coaching is more of an Art than Science. Stats and data help you make an educated guess.

by Bruno on Feb 25, 2008 3:54 PM PST reply actions   0 recs

good post
thanks for taking the time to write that up. really interesting read.

you touch on this, but my first thought is that coaches and general managers are definitely the guiltiest when it comes to confirmation biases, self-fulfilling prophecies, etc, not only in the NBA, but college, high school and the youth levels as well.  ultimately, they have the most riding on their evaluations of players and they can easily become the desperate gambler: with the most information about the players and with the most to lose, they are willing to push the limits of reason to attempt to obtain the result they envision.

using your example, i think Nate genuinely wants Jack to be his point guard. barring a trade, i think that regardless of how he plays, Jack will continue to get more minutes than Blake and Sergio.

this creates a tough situation for all involved, including fans.

all things considered, I wonder if fans are less guilty of the confirmation bias that you describe than we might think.  generally, I think fans are most likely to determine their opinions of individual players based on 1) are they helping the team win? or 2) are they exciting? or 3) are they full of potential? I also think that most fans, especially those that don't follow a team 24-7, are more apt to change their opinions of a player than coach/management are, especially over the course of a season.  for professional reasons (monetary, reputation, etc.), I don't think coaches/management have this luxury or tendency.

will an increased statistical look at individual performance help overcome this?  i think the jury is still out.  but i think your call for a new look at how we view team/individual success is warranted. aside from the stats you mentioned, is there anything Dave or myself can add to our coverage of the team that you think would speak to that point and better focus the discussion?

as an aside, you are right to point to race as a factor for creating confirmation bias; I would add weight, condition, height, and others to the list as well. not just in basketball, but in life.

just a few pennies from me.

"I want Portland to be the model franchise." ~ Bobby Medina

by Ben. on Feb 25, 2008 4:04 PM PST reply actions   0 recs

Bias
Outstanding post.  If we're honest with ourselves, we can identify our own biases.  For instance, I have trouble thinking of Roy as a great defender, with his short arms and upright court posture.  Another bias is toward an uptempo style, despite evidence that pace doesn't correlate well with playoff success.

Especially given relatively short college careers, I suspect NBA teams quantitatively analyse each NCAA game and code plays for value.  i.e. hard baseline screen, 12 points.  Indecicive defensive rotation, -5 points, calling out a screen on D, 3 points, rushed shot, -4 points, etc.  Obviously, some of this is highly subjective, but I wouldn't be surprised if each game is gone through and the plays coded to give a per-minute feel of each thing a player does.

I actually pretty amazed that folks read as much as they do into the limited box scores we get as fans.  After all, KP saw something in James Jones' awful stats with the suns...

by Engineering Problem on Feb 25, 2008 4:12 PM PST reply actions   0 recs

Great post Fizbin!
Really interesting as a number of our fellow BEers stated.  Confirmation bias is also a big player in modern day politics, but we won't go there on this site.... :)

by drawingjeremy on Feb 25, 2008 4:16 PM PST reply actions   0 recs

Phistophicles and Amlmart1
I like to see my name in a post together with great old philosopher´s one.

http://www.youtube.com/watch?v=mnodPfR1X2U

"You, my friend, are an enigma" (einman77)

by amlmart1 on Feb 25, 2008 7:34 PM PST reply actions   0 recs

Amazing post fizbin!
Pretty heady stuff, here it is 10:30 pm and you have me thinking hard - which can be painful. Seriously though I can several hardcore analytic types here on BE, who if they could set aside their "confirmation biases" and formed a proper brain trust with you, that brain trust just might be able to come up with something more trustworthy than our emotions and opinions for evaluating players. I for one would enjoy seeing that happen. Maybe we could all stop freaking out then when one persons opinion steps on another persons opinion. Peace out!
18 Points, 9 Rebounds, 4 Assists, 3 Steals, 2 Blocks and 0 TOs - LMA's Stats Rookie/Sophmore Game

by LaMarvelous on Feb 25, 2008 10:32 PM PST reply actions   0 recs

I just realized the the time...
on my first post here is 1 hour off from real PST. Either that or I just entered "The Twilight Zone".

Rod Serling has come back from the grave to haunt me. Rod? Rod?

18 Points, 9 Rebounds, 4 Assists, 3 Steals, 2 Blocks and 0 TOs - LMA's Stats Rookie/Sophmore Game

by LaMarvelous on Feb 25, 2008 10:36 PM PST up reply actions   0 recs

Exactly right
I'm interested in sabermetrics in baseball as well.  Sabermetrics is the study of statisitical analysis in baseball.  I've always wondered why similar tings are not available in basketball, or other sports for that matter.  For instance, in baseball, we can look at a player's career thusfar and grade his career against all other players in the history of the game.  Then, after adjusting for different styles of play and ballparks, we can provide similarity scores to get a rough career trajectory.  I bring this up because I've been thinking about Martell Webster.  When he came out of high school, the comparison given was Glen Rice, with the shooting ability and size.  I wonder how far off is he from that?  As for the stats posited in your diary entry, those stats of course do not exist.  But, in the early 80s, when sabermetrics was developing, a lot of the stats people wanted to use, like where a ball was put into play by a batter, did not exist either.  People actually watched all the games they could and manually created those stats.  This process took years, but now is a science.  I wonder if basketball would ever develop that way too.

by dp8039 on Feb 26, 2008 7:44 AM PST reply actions   0 recs

An excellent post.
I will back up the recommendation of 82games.com as a highly useful site.  If Nate was using efficiency/defensive efficiency stats, and stats for who plays best with whom, then Steve Blake would be playing every minute he could take.  Statistically Blake is much better than either Jack or Rodriguez.

Then we have the problem of player development. Blake is the best point guard of the three but is closer to his final development point.  

I am a ridiculous Rodriguez fan, his is the first jersey I have ever bought for myself, but Jack is presently somewhat better as a player than Rodriguez.  However, he's also three years older with enormously more time on the floor.

So the algorithm on who to play, and how much is even more complicated than who is the most effective.  Right now we are trying to work out who to lay now, for two years from now.  Add in the problem for coaches that losing games gets them fired, while developing young players makes GM's reputations and it becomes very complicated.

by EnglandDan on Feb 26, 2008 7:55 AM PST reply actions   0 recs

Interesting
I agree that the statistics used now can be very inexact and that we need more, but I also can see that the more precise they become, the more they become subjective.

I've mentioned before that I've seen turnovers called "bad pass" when really the receiver fumbled. OTOH someone with "great hands" might have been able to corral it. To me (and here is the subjective part) it looked like an error on the part of the receiver.  But is it still a bad pass or a fumble?  Or a bad pass because the passer should have known that the receiver might not be able to handle it?  Category for "bad decision"?

I think that the coaches (and KP) look beyond play when valuing a player.  By all reports, Jack is a real winner in the locker room.  He's definitely a leader on the floor - he talks constantly to his teammates.  He was one that the Mikes said would probably become a coach.  These things may weigh heavily with the coach when he's setting his lineup.  Maybe Jack has lots of these transparent plusses.  (Transparent to fans, highly visible to the coaching staff.)

I also see a need to remind people that Allen was the one who wanted Sergio.  When it came time to buy the pick during the draft everyone (including KP) said "No".  Allen said "yes" so the deal was done. (His money after all.)  Remember Allen vetoed a Miles deal. He may be behind other roster decisions too.  

"...people are young until they aren't. And even then they're usually young a little more yet." - Dave, February 21.

by jorga on Feb 26, 2008 8:19 AM PST reply actions   0 recs

That's a good point
I'd forgotten about that.  

I think Allen needs to stay out of that process and let KP come to him when he needs the cash.  I want that championship and I strongly feel that KP is the way.

"No way. I love this town. We're gonna bring back the glory years. These boys are my boys. This is my team and my town." - Greg Oden when asked about leaving.

by ratbastird on Feb 26, 2008 9:03 AM PST up reply actions   0 recs

A lot of work...
  ..went into that diary. Interesting thoughts about Confirmation Bias. Your 10 areas to track would be very revealing over a period of time. Good work.

  However, "We need to agree as fans what sort of evidence is required to recommend a firm conclusion". That's not going to happen. I think within your own diary you outlined pretty well several reasons why fans are NEVER going to agree on what sort of evidence.
   Confirmation Bias potential aside, most fans are reactionary and empirical in forming opinions. Not all, but many fans don't go past the last game or the last series of plays they remember to form an opinion on a coach or player. I'm afraid if you are looking for agreement among fans you will never find it. Fantasy leagues have probably brought more fans to start looking at stats more closely but overall a fan is still going to react more to the "facts" that he "thinks" he see when watching a game then numbers on a spread sheet.

   Very interesting diary, well done.  

"Mother Nature started this fight, I think it's about time we ended it!"

by Krang on Feb 26, 2008 8:55 AM PST reply actions   0 recs

Two parts
Excellent post.

I see two parts:

Part one is that you state people see what they expect to see and want to see and that's not always accurate.

I agree with this and this has been going on since our first ancestors decided it'd be a good idea to get out of the water, dry off, get a tan, and go climb some trees to see the world better and start tea parties where they socialized with others.

Part Two is a better need for stat measurements in general but with a secondary purpose of helping to counterbalance the bias.

I think KP has this, so I'm not so concerned about the counterbalance.   I agree that I, as the average fan, would LOVE those stats you meantioned.

Hey dave, maybe you should tell that advisory board that we'd love us some serious blazer stats!  Then again that would be feeding info to the enemy, and I like them in the dark as we feast upon the rotted corpse of their stupidity.

If other teams are down, I say wave hello from up high and much on some popcorn and then point and laugh.  

But MAN would i love those stats.

"No way. I love this town. We're gonna bring back the glory years. These boys are my boys. This is my team and my town." - Greg Oden when asked about leaving.

by ratbastird on Feb 26, 2008 9:15 AM PST reply actions   0 recs

Most Important Stat is OBP.
In baseball, the point of an at-ball is to avoid an out especially double plays.  To keep the inning going, move baserunners, and tire out the starting pitcher with pitch count.

The most comparative stat in basketball is turnovers.  Keep the turnovers low and shoot more.  Unfortunately it doesn't work if the players can't shoot or get to the free throw line.  For each player, a plays created should be a stat.  Tipped balls tap outs, swing pass, penetration dish, good outlet pass etc.  Plays created must lead to a good shot attempt or FTs.  

BRoy may only 7 assists, but his plays created could be upwards to 20.

So I'm guessing some combination of the 4 stats, turnovers, shooting percentage, FTs, and plays created together for an offensive rating.  Scary thought, just thinking about this, Jarrett Jack may be our second best player on the team right now.  No wonder we aren't very good.

by blzrfan on Feb 26, 2008 9:31 AM PST reply actions   0 recs

The stat in basketball is points per possession.
This takes into account all the problems with turnovers, ability to shoot, whether free throws are counted or not.  What really matters is how many points you score when you have the ball compared with how well the other team does that.

I think turnovers get over emphasized.  Is a pass that has a 25% chance of getting picked off but if completed results in a dunk 90% of the time a good pass?  The answer is that statistically (.75x.9=.665, a great shooting percentage) it's a fantastic play but most coaches hate it.

Early in the season with Sergio Rodriguez playing we were giving up a ton of points at a high percentage, because Sergio was taking lots of risks.  Nate benched him and he has not played that style since.  However, during that period Sergio had by far the best +/- on the team because we scored so efficiently when he was playing that style.  Just an example.

by EnglandDan on Feb 26, 2008 11:19 AM PST up reply actions   0 recs

I don't thinks points per possession works
It measures the team efficiency, not the player.  Of course basketball is a team sport, but individual play dominate the ebb and flow in the half court offense.  

by blzrfan on Feb 26, 2008 2:52 PM PST up reply actions   0 recs

I guess I should forgive you for suggesting
that we trade LMA.  I no longer want to trade you Fizbin..... and you are smarter than I am.  It's a refreshing & thought provoking diary.

by tweener on Feb 26, 2008 9:44 AM PST reply actions   0 recs

Funny you should mention...
That post I made was about as reactionary and filled with bias as they come. You were probably right to react against it.

Thus the need for better objective measures.

by Fizbin on Feb 26, 2008 10:16 AM PST up reply actions   0 recs

Comments For This Post Are Closed


User Tools

A site by Blazer fans, for Blazer fans
Start posting about the Trail Blazers »

Join SB Nation and dive into communities focused on all your favorite teams.

FanPosts

Community blog posts and discussion.

Recommended FanPosts

Small
New B-Roy is the Old Kobe
The_dude_small
Stats 101: Whats The Deal? Let Me Explain...
Broyhope_small
First Rose Garden Game Redux...
Troll_stone_cropped_small
Nate in the land of the giants
Jesus_icon_i_small
OT: Petitio Principii, Worldview, and Sports Related Conversations

Recent FanPosts

Small
Help! Can someone explain Blake/Miller/Roy?
100_0017_small
Junk 11/8
Small
Nate comfortable with Miller calling Blazers' plays??
Imported_photos_00004_small
The Blazers NEED to make a move at or before the trade deadline...
The_dude_small
Stats 101: Averaging The Turnover
Copy_of_dcp_2180_small
Homer junk 11/7/8
Small
Showcasing Blake for Trade?
Small
En Garde -- Questions about RAMBO II
Small
We need the triangle offense
Small
Great win but still..

+ New FanPost All FanPosts >

FanShots

Quick hits of video, photos, quotes, chats, links and lists that you find around the web.

Recommended FanShots

marty's buzzer beating dunk
Brandon Roy Wallpaper
Chad Ford (Insider): Travis Outlaw "Probably Won't Be Re-Signed"
Manu catches a bat during game. live coverage

ps
check out blair at the begining of video he freaks like a girl
Start Miller

Recent FanShots

Club Level tickets for sale tonight
Prince has a ruptured disc
Anyone converting stream?
A statistical look at last year's referees
Iverson takes indefinite leave from Grizzlies
I was going for the Sin City look, cause that is what I see when I look at this picture. Turned out more like last seasons cartoon intro.
Great Night
Roy is a combo guard.
Claver's team wins again.
Blazercentric SS&R Post

+ New FanShot All FanShots >


Editors

Kitten_small Dave

Ben_small Ben.

Moderators

Pict1126_small -ken

Polar_bear_small jorga

Terryporter_small prezofdeath

Small usmcr3049

Jesus_icon_i_small T Darkstar

Wallpaper_small geoffm