seekingferret (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
seekingferret) wrote2015-07-21 08:58 am
seekingferret) wrote2015-07-21 08:58 am
Entry tags:
(no subject)
I've been reading Michael Lewis's Moneyball the past few days. As I've been a studious baseball stat nerd since I was eight years old, you might have thought I would have leapt on Moneyball from its publication, but the thing was, I never heard anyone say anything they got from Moneyball that I didn't already know, so it never seemed that important to read. I knew sabermetrics long before Michael Lewis or Billy Beane did. (I won a research award from SABR the year before Moneyball came out.) But I decided to read it now, for whatever reason.
The shocking thing I'm finding in reading Moneyball, though, is how bad it is, how incomplete it is, how wrong it is about how baseball statistics works.
Let us list its flaws
1)Skims over the math that matters, makes too much of the math that doesn't.
One of the early fundamental discoveries of sabermetrics is that baseball games can be modelled as a state machine with a usefully small number of states, since the only parameters these states need to have for the simplest model is number of outs and number of runners on base. This model can be used for a number of things, but perhaps the most basic and important is modelling the expected run value of each state and using it to calculate the relative value of various situations. It allows good, straightforward computational models to analyze all sorts of tactical situations. I learned about this state model when I was a teenager, shortly after I learned what a state machine was, in a brilliant Pete Palmer essay. Lewis spends about a paragraph mangling the explanation of this simple model, dashed off as an aside to half-assed explanations of other unrelated mathematical concepts. As he continues in his discussion, he makes myriad assertions about the relative value of tactics, most of which are derived from this state machine model or variants thereof, but he never mentions state machines again. Instead, he makes appeals to authority, i.e. "According to Billy Beane, stolen bases are only valuable if they're successful 70% of the time." This particular result was baby's first introduction to mathematical modelling, but Lewis has no apparent interest in where it comes from, once he gets it from the genius mouth of Billy Beane, who of course was not the one actually responsible for the result.
Perhaps more glaringly, Lewis explains that there was a debate between various sabermetricians about the weight various mathematical models suggested should be used to analyze the relative value of on-base average and slugging average. People like Palmer and Bill James thought OBP should be weighted 1.5 times SLG, 'stupid baseball insiders' thought they were approximately equally weighted, and Paul DePodesta and Billy Beane thought OBP should be weighted 3 times SLG. This is a pretty substantial disagreement that Lewis cites because according to him, DePodesta's model gave him a huge advantage in pursuing undervalued players... if DePodesta's model is right. Lewis, though, isn't interested in explaining the disagreement between Bill James and DePodesta, because to him, mathematically oriented people are wizards doing magic by plugging numbers into equations. So after reading the section, I honestly couldn't tell you whether DePodesta or James is right, because Lewis glosses over what actually matters about the math. There's this baffling bit where Lewis conflates denominators and coefficients. And if you ask me, a book about the practical use of advanced statistical tools in baseball ought to treat debates like this as questions to try to resolve, not as plot points.
2)Ignores the class and racial implications of Beane's drafting strategies.
Beane puts an emphasis on drafting college players not because college players have better overall outcomes, but because their outcomes are more predictable- they are older and thus more developed, there is more data available about them, the data about them is more reliable, the data about them is based on tougher competition. Needless to say, this has racial and class implications. College baseball players are more likely to be white and more likely to be economically better off than players who need to be drafted out of high school to take a paycheck as soon as they can. This is a pretty significant part of the story, it seems to me, but it's not something Lewis has any interest in talking about. Is Beane's approach likely to extend the forces pushing African-American players away from the game? I don't know, but I think it's important as well as interesting to ask.
Further, Lewis fetishizes the Ivy league pedigrees of Beane's assistant GM Paul DePodesta, a Harvard economics graduate, and Beane's baseball mentor Sandy Alderson, whose Dartmouth and Harvard degrees he mentions multiple times. When DePodesta hires another Harvard grad to work with him, Lewis chortles gleefully about how the Harvard Old Boys club has finally reached baseball. He also fetishizes the contributions of Wall Streeters who launch analytics companies using the same techniques they used to design complicated derivatives. These techniques purportedly bring 'intelligence' to a field Lewis repeatedly claims was being run by absolute morons, and that elitist tone is not something I'm exaggerating. Lewis, recall, is a Princeton man and a former Wall Streeter himself, and he clearly sees such people as superior.
3)NEVER MENTIONS STEROIDS AT ALL.
This is so absurd it defies belief. Billy Beane was a teammate of Jose Canseco and Mark McGwire when they were the 'roid-enhanced Bash Brothers in Oakland in the late '80s. So when he talks about the way that skinny draftees who hit well for average 'develop power' in the minor leagues, it's impossible to think that he's talking about a natural process whereby players 'fill out' as they reach their twenties, as Lewis suggests. Beane is suggesting that somehow, wink wink, when his players are in the minors they start taking steroids and gain power... But there are no steroids that can teach you how to hit a baseball for high average, how to have plate discipline to draw walks, how to play baseball smartly, so those are the things Beane believes in drafting for... steroids can handle the rest. There's a further oblique comment from Beane later on about how Barry Bonds has gone beyond where talent takes him that only makes sense if Beane knows that Bonds is juicing.
Somehow Moneyball is a book about trends in baseball at the height of the Steroid Era that never mentions steroids. Either Michael Lewis is a fool or a liar. As I progress through the book, I alternate between favoring either of these hypotheses.
4a)Valorizes Billy Beane's mathematical errors.
Beane and DePodesta's mathematical model suggests that some players who other teams will not want until the tenth or fifteenth round of the draft are actually as valuable as first round players. Any rational game theorist would say that this means that you should aim to draft them in the fifth round, or the ninth round if you think you can get away with it, because why waste a chance to get a good player who won't be there later taking a good player who will be there later? Beane drafts them in the first round and congratulates himself on how clever he was. When at best you can say he 'reached' for these prospects, and at worst you can say he signaled to the other teams that he was prioritizing different things than them in his scouting, too.
4b)Valorizes Billy Beane for cheating.
The alternative explanation for this is that Beane drafts these players in the first round because he has made rule-breaking handshake deals with these players to take tenth round money despite being drafted in the first round. Lewis defends this by suggesting 'everyone does this', but it's still cheating, and in an echo of my second point about the classism, the people who lose out because of this cheating are the people most vulnerable, the young draftees who lack sound financial advising and get manipulated by Billy Beane as a result. Lewis reports it as an unambiguous triumph for Beane, laughing along with the scouts at the way the draftees are being manipulated. Along with covering up the steroid use, this is pretty typical of how Lewis defends Beane's win-at-all-costs approach to the game.
5)Pretends Billy Beane was the only one doing this stuff.
At the same time that Oakland was using its 'moneyball' tactics to elevate it in spite of a small payroll, the Yankees were consistently beating the As, using a combination of 1)High payroll giving them the ability to sign and keep star players and 2)a style of play emphasizing home runs and walks and on-base percentage, the same style of play Beane emphasized. The Yankees may not have had a Harvard educated economist crunching the numbers (or they may have), but they clearly recognized the value of those baseball assets anyway. In the early 2000s, the Yankees were famous for, under managers Torre and then Girardi, being a patient, disciplined, skilled team that drew more pitches than any other team.
This is not surprising. Billy Beane may not have encountered Bill James until the mid 90s, but James had been gaining followers since the late '70s. It stands to reason that in that time period, some of his ideas would infiltrate baseball's smarter management people. Of course, if you have a mathematical advantage, you don't give it away... this is one of the key themes of the book, the ways Beane tries to disguise his tactics, yet somehow Lewis tries to sell the explanation that nobody in baseball but Beane was smart enough to use these tactics as more plausible than "Everyone else who's using these tactics isn't talking to reporters." Especially if, like the Yankees, they were able to combine sabermetric tactics with bigger budgets to amplify their advantage even over sabermetric teams like the As, it would not be in their interest to publicly talk about what they were doing.
It seems clear that the reason Beane is in fact talking about it is because he wants to raise a conversation about the effect the salary disparity is having on the game, that he's not interested in staying a clever underdog but wishes to be able to compete on salary with the Yankees. But Lewis doesn't do a great job of pointing this out when he trumpets Beane as a loner genius.
Maybe I'm a giant Yankee homer. Okay, no maybe about it. I'm a giant Yankee homer. But it's frustrating when we see teams with giant payrolls fail catastrophically all the time, while the Yankees have consistently built championship teams using their giant payrolls, by way of smart drafting, smart risk-taking, smart baseball tactics, and effective use of their economic advantages. The Cashman Yankees are at least as compelling a business story as the Beane As.
6)Acts like Beaneball is the only right way to run a team
From the 2002 draft that Lewis highlights, a couple As draftees became good MLB players, though none were ever superstars. Most didn't pan out. On the other hand, several of the high school pitchers shunned by the As, notably Zach Greinke, became superstars (while most didn't pan out). Going after high school players in the draft is a high risk/high reward strategy that Beane felt he couldn't afford since he was so dependent on young talent to staff his roster cheaply, but it's nonetheless a valid strategy because when it does pay off, it pays off handsomely. Lewis spends most of the book calling teams pursuing this strategy idiots who listen to their gut instead of the power of math, but several of the teams who pursued it did much better in the 2003 draft than the As did. Possibly in the long run, a strategy like Beane's is safer and more reliable as an engine for bringing in young talent, but when all it takes is one or two transcendent talents to change the fate of your team, it's sometimes worth risking the draft.
There are lots of ways to run a team intelligently. The mathematics of baseball are still not reliable enough to offer any kind of guarantee of success, and there are a number of different, defensible approaches that make reasonable sense. Lewis acts like only Beane in all of baseball had given any thought to draft strategy, since only Beane was using Beane's strategy. But there are lots of other viable strategies that intelligent people could use in the draft.
7)Takes a story about stats and tries to impose a human narrative on it.
Midway through the book, Lewis describes sitting in the A's video room with several front office people during a game. Lewis is watching the commercial TV broadcast of the game and periodically gasping or cheering. The front office people are watching a camera feed from center field that gives them the best view of the strike zone and carefully watching, never reacting emotionally. Lewis realizes, he says, that they are watching a different game than him. He is reacting to and creating narratives. They are slicing and dicing the game and analyzing its fine details.
And yet, for several chapters, Lewis reaches into the Halberstam bag of tricks to tell a digressive story about a couple of particular games and the emotional, sportswritery journeys that brought particular players to those games. Lewis is not as good as Halberstam, but he's not bad at this. But it's awfully wrongfooted. According to everything the theory he's espousing says, these games don't matter. These players don't particularly matter to the system: their weirdness, their narrative, their psychology, none of these explain why they're effective moneyball players. The only thing that matters is how often they create runs.
Lewis is fascinated by the process by which players who were not considered valuable by other teams become valuable to the As, because this process seems like a sort of redemption story to him. His repeated refrain as he talks about players with the As front office is "What's wrong with him?" He delights when he learns that players were considered too fat, or too weird, or too old, or too bad at fielding, and they are saved from the dustbin by Paul DePodesta's computer. He chortles snidely about Brewers draftee Prince Fielder that "He is too fat even for the A's." Lewis particularly enjoys that most of the weird players he writes about have not been directly told by the A's why they have been saved. It's a dramatic irony he enjoys over and over again: he has been let in on a secret about these players that even they do not know about themselves. He, therefore, is vicariously smarter than the players. [All is right in the world. Lewis is a Princeton man.]
So um... yeah, that was Michael Lewis's Moneyball. Let me be clear: It seems apparent from all the numbers that Beane is legitimately, consistently able to outperform his salary limitations, and that some of his analytic tools play a role in that. But Moneyball does not offer an effective or convincing explanation for why this is.
The shocking thing I'm finding in reading Moneyball, though, is how bad it is, how incomplete it is, how wrong it is about how baseball statistics works.
Let us list its flaws
1)Skims over the math that matters, makes too much of the math that doesn't.
One of the early fundamental discoveries of sabermetrics is that baseball games can be modelled as a state machine with a usefully small number of states, since the only parameters these states need to have for the simplest model is number of outs and number of runners on base. This model can be used for a number of things, but perhaps the most basic and important is modelling the expected run value of each state and using it to calculate the relative value of various situations. It allows good, straightforward computational models to analyze all sorts of tactical situations. I learned about this state model when I was a teenager, shortly after I learned what a state machine was, in a brilliant Pete Palmer essay. Lewis spends about a paragraph mangling the explanation of this simple model, dashed off as an aside to half-assed explanations of other unrelated mathematical concepts. As he continues in his discussion, he makes myriad assertions about the relative value of tactics, most of which are derived from this state machine model or variants thereof, but he never mentions state machines again. Instead, he makes appeals to authority, i.e. "According to Billy Beane, stolen bases are only valuable if they're successful 70% of the time." This particular result was baby's first introduction to mathematical modelling, but Lewis has no apparent interest in where it comes from, once he gets it from the genius mouth of Billy Beane, who of course was not the one actually responsible for the result.
Perhaps more glaringly, Lewis explains that there was a debate between various sabermetricians about the weight various mathematical models suggested should be used to analyze the relative value of on-base average and slugging average. People like Palmer and Bill James thought OBP should be weighted 1.5 times SLG, 'stupid baseball insiders' thought they were approximately equally weighted, and Paul DePodesta and Billy Beane thought OBP should be weighted 3 times SLG. This is a pretty substantial disagreement that Lewis cites because according to him, DePodesta's model gave him a huge advantage in pursuing undervalued players... if DePodesta's model is right. Lewis, though, isn't interested in explaining the disagreement between Bill James and DePodesta, because to him, mathematically oriented people are wizards doing magic by plugging numbers into equations. So after reading the section, I honestly couldn't tell you whether DePodesta or James is right, because Lewis glosses over what actually matters about the math. There's this baffling bit where Lewis conflates denominators and coefficients. And if you ask me, a book about the practical use of advanced statistical tools in baseball ought to treat debates like this as questions to try to resolve, not as plot points.
2)Ignores the class and racial implications of Beane's drafting strategies.
Beane puts an emphasis on drafting college players not because college players have better overall outcomes, but because their outcomes are more predictable- they are older and thus more developed, there is more data available about them, the data about them is more reliable, the data about them is based on tougher competition. Needless to say, this has racial and class implications. College baseball players are more likely to be white and more likely to be economically better off than players who need to be drafted out of high school to take a paycheck as soon as they can. This is a pretty significant part of the story, it seems to me, but it's not something Lewis has any interest in talking about. Is Beane's approach likely to extend the forces pushing African-American players away from the game? I don't know, but I think it's important as well as interesting to ask.
Further, Lewis fetishizes the Ivy league pedigrees of Beane's assistant GM Paul DePodesta, a Harvard economics graduate, and Beane's baseball mentor Sandy Alderson, whose Dartmouth and Harvard degrees he mentions multiple times. When DePodesta hires another Harvard grad to work with him, Lewis chortles gleefully about how the Harvard Old Boys club has finally reached baseball. He also fetishizes the contributions of Wall Streeters who launch analytics companies using the same techniques they used to design complicated derivatives. These techniques purportedly bring 'intelligence' to a field Lewis repeatedly claims was being run by absolute morons, and that elitist tone is not something I'm exaggerating. Lewis, recall, is a Princeton man and a former Wall Streeter himself, and he clearly sees such people as superior.
3)NEVER MENTIONS STEROIDS AT ALL.
This is so absurd it defies belief. Billy Beane was a teammate of Jose Canseco and Mark McGwire when they were the 'roid-enhanced Bash Brothers in Oakland in the late '80s. So when he talks about the way that skinny draftees who hit well for average 'develop power' in the minor leagues, it's impossible to think that he's talking about a natural process whereby players 'fill out' as they reach their twenties, as Lewis suggests. Beane is suggesting that somehow, wink wink, when his players are in the minors they start taking steroids and gain power... But there are no steroids that can teach you how to hit a baseball for high average, how to have plate discipline to draw walks, how to play baseball smartly, so those are the things Beane believes in drafting for... steroids can handle the rest. There's a further oblique comment from Beane later on about how Barry Bonds has gone beyond where talent takes him that only makes sense if Beane knows that Bonds is juicing.
Somehow Moneyball is a book about trends in baseball at the height of the Steroid Era that never mentions steroids. Either Michael Lewis is a fool or a liar. As I progress through the book, I alternate between favoring either of these hypotheses.
4a)Valorizes Billy Beane's mathematical errors.
Beane and DePodesta's mathematical model suggests that some players who other teams will not want until the tenth or fifteenth round of the draft are actually as valuable as first round players. Any rational game theorist would say that this means that you should aim to draft them in the fifth round, or the ninth round if you think you can get away with it, because why waste a chance to get a good player who won't be there later taking a good player who will be there later? Beane drafts them in the first round and congratulates himself on how clever he was. When at best you can say he 'reached' for these prospects, and at worst you can say he signaled to the other teams that he was prioritizing different things than them in his scouting, too.
4b)Valorizes Billy Beane for cheating.
The alternative explanation for this is that Beane drafts these players in the first round because he has made rule-breaking handshake deals with these players to take tenth round money despite being drafted in the first round. Lewis defends this by suggesting 'everyone does this', but it's still cheating, and in an echo of my second point about the classism, the people who lose out because of this cheating are the people most vulnerable, the young draftees who lack sound financial advising and get manipulated by Billy Beane as a result. Lewis reports it as an unambiguous triumph for Beane, laughing along with the scouts at the way the draftees are being manipulated. Along with covering up the steroid use, this is pretty typical of how Lewis defends Beane's win-at-all-costs approach to the game.
5)Pretends Billy Beane was the only one doing this stuff.
At the same time that Oakland was using its 'moneyball' tactics to elevate it in spite of a small payroll, the Yankees were consistently beating the As, using a combination of 1)High payroll giving them the ability to sign and keep star players and 2)a style of play emphasizing home runs and walks and on-base percentage, the same style of play Beane emphasized. The Yankees may not have had a Harvard educated economist crunching the numbers (or they may have), but they clearly recognized the value of those baseball assets anyway. In the early 2000s, the Yankees were famous for, under managers Torre and then Girardi, being a patient, disciplined, skilled team that drew more pitches than any other team.
This is not surprising. Billy Beane may not have encountered Bill James until the mid 90s, but James had been gaining followers since the late '70s. It stands to reason that in that time period, some of his ideas would infiltrate baseball's smarter management people. Of course, if you have a mathematical advantage, you don't give it away... this is one of the key themes of the book, the ways Beane tries to disguise his tactics, yet somehow Lewis tries to sell the explanation that nobody in baseball but Beane was smart enough to use these tactics as more plausible than "Everyone else who's using these tactics isn't talking to reporters." Especially if, like the Yankees, they were able to combine sabermetric tactics with bigger budgets to amplify their advantage even over sabermetric teams like the As, it would not be in their interest to publicly talk about what they were doing.
It seems clear that the reason Beane is in fact talking about it is because he wants to raise a conversation about the effect the salary disparity is having on the game, that he's not interested in staying a clever underdog but wishes to be able to compete on salary with the Yankees. But Lewis doesn't do a great job of pointing this out when he trumpets Beane as a loner genius.
Maybe I'm a giant Yankee homer. Okay, no maybe about it. I'm a giant Yankee homer. But it's frustrating when we see teams with giant payrolls fail catastrophically all the time, while the Yankees have consistently built championship teams using their giant payrolls, by way of smart drafting, smart risk-taking, smart baseball tactics, and effective use of their economic advantages. The Cashman Yankees are at least as compelling a business story as the Beane As.
6)Acts like Beaneball is the only right way to run a team
From the 2002 draft that Lewis highlights, a couple As draftees became good MLB players, though none were ever superstars. Most didn't pan out. On the other hand, several of the high school pitchers shunned by the As, notably Zach Greinke, became superstars (while most didn't pan out). Going after high school players in the draft is a high risk/high reward strategy that Beane felt he couldn't afford since he was so dependent on young talent to staff his roster cheaply, but it's nonetheless a valid strategy because when it does pay off, it pays off handsomely. Lewis spends most of the book calling teams pursuing this strategy idiots who listen to their gut instead of the power of math, but several of the teams who pursued it did much better in the 2003 draft than the As did. Possibly in the long run, a strategy like Beane's is safer and more reliable as an engine for bringing in young talent, but when all it takes is one or two transcendent talents to change the fate of your team, it's sometimes worth risking the draft.
There are lots of ways to run a team intelligently. The mathematics of baseball are still not reliable enough to offer any kind of guarantee of success, and there are a number of different, defensible approaches that make reasonable sense. Lewis acts like only Beane in all of baseball had given any thought to draft strategy, since only Beane was using Beane's strategy. But there are lots of other viable strategies that intelligent people could use in the draft.
7)Takes a story about stats and tries to impose a human narrative on it.
Midway through the book, Lewis describes sitting in the A's video room with several front office people during a game. Lewis is watching the commercial TV broadcast of the game and periodically gasping or cheering. The front office people are watching a camera feed from center field that gives them the best view of the strike zone and carefully watching, never reacting emotionally. Lewis realizes, he says, that they are watching a different game than him. He is reacting to and creating narratives. They are slicing and dicing the game and analyzing its fine details.
And yet, for several chapters, Lewis reaches into the Halberstam bag of tricks to tell a digressive story about a couple of particular games and the emotional, sportswritery journeys that brought particular players to those games. Lewis is not as good as Halberstam, but he's not bad at this. But it's awfully wrongfooted. According to everything the theory he's espousing says, these games don't matter. These players don't particularly matter to the system: their weirdness, their narrative, their psychology, none of these explain why they're effective moneyball players. The only thing that matters is how often they create runs.
Lewis is fascinated by the process by which players who were not considered valuable by other teams become valuable to the As, because this process seems like a sort of redemption story to him. His repeated refrain as he talks about players with the As front office is "What's wrong with him?" He delights when he learns that players were considered too fat, or too weird, or too old, or too bad at fielding, and they are saved from the dustbin by Paul DePodesta's computer. He chortles snidely about Brewers draftee Prince Fielder that "He is too fat even for the A's." Lewis particularly enjoys that most of the weird players he writes about have not been directly told by the A's why they have been saved. It's a dramatic irony he enjoys over and over again: he has been let in on a secret about these players that even they do not know about themselves. He, therefore, is vicariously smarter than the players. [All is right in the world. Lewis is a Princeton man.]
So um... yeah, that was Michael Lewis's Moneyball. Let me be clear: It seems apparent from all the numbers that Beane is legitimately, consistently able to outperform his salary limitations, and that some of his analytic tools play a role in that. But Moneyball does not offer an effective or convincing explanation for why this is.