I have been wanting to re-do my draft study but the work necessary to gather up that information has been too much for me to handle (I was going to use R to pull down the data last season, but then BB-Ref changed their URL from http to https, and then I was unable to use R anymore to pull the data; I was that close...). Still, it is instructive to see how bad the odds are in the draft so that one can set their expectations for the prospect properly. So I'll looking at how other teams have done in finding good players with the #2 and #45 picks during the history of the draft.
ogc thoughts
Looking into the second pick overall in the draft (data courtesy of the great Baseball-Reference.com website), we get this distribution for the pick (50 drafts, 1965-2014; I had to estimate for players in eight of the more recent years at the end of that time period, as they were still active, and based on production so far, forecast how high they might go, and slot them below):
No MLB: 12.0%
Neg WAR: 14.0%
Marginal: 24.0% (0.1-8.9 WAR)
Useful/Avg: 18.0% (9.0-17.9 WAR)
Good: 22.0% (18-35.9 WAR)
Star: 10.0% (36.0+ WAR)
I have also been playing with the idea of adding another category of Hall of Famer. Given my ranges, the next logical break would be 54.0 WAR, and from my prior research work, that seems to work. For the second pick, that would yield Justin Verlander (likely future HoF), Will Clark (not in, but many would argue for him; I would too), and Reggie Jackson (in like Flynn). Missing that was JD Drew, at 44.9.
Kris Bryant was included in Star, although he hasn't reached 36.0 WAR, but he's been so good so far in his career that he looks likely to reach that level, and heck, probably 54.0 WAR, as he has averaged 6.6 WAR per season in his first three seasons. But he's been having a good season, but not to his standards set so far, and you need longevity at producing at a high rate to reach HoF levels, so we'll see if my projection is accurate. And that shows how high that bar is, even at 6.6 WAR per season, he would need to produce that over at least 8 seasons to reach the HoF level.
Player Value Ranges: Accurate?
I used the categories that I've been using for years now, though I'm wondering if I should modify them. Still, the guiding principle behind what I'm trying to do with the categories is to label players by whether they are good (or better) or not. Average is considered 2.0 WAR, and we want good, so I used 2.5 as the minimum of a good season. 6 years (pre-free agent years) at 2.5 yields 15.0 WAR, plus some production after that.
About five years ago, I painstakingly captured the draft and career production data to re-do the draft study, covering deeper into the draft, but failed to do the analysis before the next season started. Sorting all the data I found that 18.0 WAR was about right at that time for the smell test: I considered the players with more than 18.0 to be good players, below, not as much. Had baseball-reference.com not redo their WAR methodology and formulation since then, I would use that data for more insights on this.
I originally was using Dave Kingman as the guiding light, sort of, as he was around 18.0 (but under) and as much as I was his fan when he was a rookie, I see him as not a good player. But his WAR value dropped significantly after Baseball-Reference.com revised their WAR system, so he isn't a factor, even as a "obviously not in" guiding light anymore, now that he's nowhere near 18.0 WAR.
If you have any thoughts about the categories, I would love to hear them. I don't like the way most studies just select by the usual human tendencies (1-10, 10-20, etc.; while I appreciate tendencies, I think we can do better). I wanted some logic behind the categories. I've been okay with my categories, though part of me wonder if I'm asking too much of a good player to produce 18.0 WAR.
But I know part of my feelings for that is coming from the fact that draft success is now less with WAR than it was when I did my first draft study (success for the second pick was in the 40% range, which is still less than a coin flip, but less than the 32% above). And so I feel that bias might be clouding my judgement.
I could lower the minimum and craft the success rate. But to reach at least 40%, I would need to add four more players.
I could easily add two (up to 36% now) by lowering to 16.5 WAR, but to add another two could take some doing. A third will probably come in at 16.5 WAR, as there is an active player with 12.6 (Moustakas), he looks pretty likely to reach that level. Buxton looked likely to reach, as well, after last season (7.2 WAR career, 5.2 WAR in 2017), but his difficulties this season (as well as missing most of the makes things murkier. And if he don't reach, there is nobody else looking likely to reach that high.
So that leaves players who are retired, and the next guy on the list has 12.4 WAR. That's just too low, so I don't see any way to reach 40% with these 50 players, unless Buxton recovers from his horrible 2018 season, and rebound somewhere closer to his 2017 season.
And that's aside from the fact that forcing the data to fit expectations (or past studies) should not be the way to design the ranges. That's why I didn't start with a number in mind, other than the guiding principal that the player should be good. And in my original study, good meant not only performing, but playing good enough to reach free agency and be a player in demand. Hence why I chose 2.5 WAR (better than average) and 6 years (number of years before free agency). So 15.0 WAR was the floor I started with, and combing through the names, 18.0 WAR looked like a good dividing point.
And really, that's all I'm ultimately trying to do with my study, separate good from not good. I don't really need all the categories above to do that (and frankly, my statistics skills aren't good enough to deal with multi-category analysis, only binary evaluations), but I thought it would be interesting to see how it falls out, obviously a 0.1 WAR player is different from a 17.9 WAR player. And I could probably segment further within each, as 9.0 WAR is also much different from a 17.9 WAR player, as well.
Another draft study I found, by Royal Review, had these categories, which is very similar to mine, but with a lower threshold for Good:
- Very Poor: less than 3.0 WAR
- Below Avg: 3.0-8.9 WAR
- Average: 9.0-14.9 WAR
- Good: 15.0-20.9 WAR
- Very Good: 21.0-26.9 WAR
- Great: greater than 27.0 WAR
So, for now, I think I'm keeping my ranges unless someone can come up with good logical reasons why the ranges don't make sense. At least for now. Should I capture the data again into a database, I could try to learn how to cluster data and get closer in these categories, and not have it set at multiples of 9, for example (half is a common dividing point).
Not That Many Good Players Drafted with #2 Pick Overall; Half Little to No Value!
So, the history of the 2nd pick overall is pretty sparse concerning finding a good player or better. Only 10.0% become Stars, and 22.0% become Good, meaning only a little less than a third (32%) of drafted players so far has been Good or better.
Another 18.0% produced good value but not the good player that most baseball fans would think they should get from the #2 pick in the draft.
What this means is that half (50.0%) of all prospects selected as the #2 pick overall either were marginal, were below replacement level players (i.e. negative WAR), or never even made it to the major leagues. There were actually 12% of the prospects who never made it to the major leagues.
Hasn't Been Better in Recent Drafts
And it don't look like things got better in the more recent history of the draft. Looking at 2005-2014, it got worse. Again, 50% of the prospects were marginal, below replacement level, or never made the major leagues. But only 20% were Good or better (vs. 32% over the whole span), leaving 30% Useful.
So, as much as some get mad at their teams for not finding a good player or others who don't want to pay $7M for such a prospect, the fact is that it is very hard to find a good player in the draft, even with the second pick overall of the draft. And I don't have stats to back it up, but there are not a lot of good catchers available, and they are most likely to be the ones selected high in the draft, not lower. Piazza is the anomaly. So if you want to try to have a great hitting and defensive catcher, picking early is the best bet, just not a likely bet. That's where scouting and coaches come in.
Very Very Very Hard to Find Good Players with the 45th Pick Overall
The stats are even more gruesome for pick #45, where the Giants selected Sean Hjelle. 58% of all prospects selected never even made the majors. 22% were below replacement level (negative WAR). Only 16% were Marginal, with 0% Useful. 4% were Good with nobody becoming a Star. The Good players were: Jed Lowrie (2005) and Trevor Story (2011; he is only at 7.7 WAR today, but he projects to pass 18.0 as long as he can stay healthy and productive, a big if at SS, where the position can wear on a player).
So when teams are selecting with the #45 pick, which is roughly the second round of the draft (as supplemental picks vary from year to year), they are not even rolling the dice, they are trying to pick up one-eyed Jacks out of a deck of cards, roughly. So I would not have high hopes for Hjelle developing, it is just very hard to develop a player even from the 45th pick overall. Just enjoy him and root for him, but don't expect to get a major league player, let alone a staring pitcher, at least from the get go. He would have to do really well to propel him into a significant prospect.
And it just gets worse from there, the deeper you go in the draft, for the most part.
A Look at Suarez
To continue this theme, Suarez was the 61st pick of the draft, when the Giants selected him with their second round pick in 2015, further back in the second round than Hjelle. He has been a negative WAR player so far, but he has actually pitched well for a rookie, I've thought. I think it will get better for him as he learns to pitch in the majors.
Now, success rates do vary, up and down, and it is better here than at #45. Here, 40% never makes the MLB, 20% have negative WAR, 30% were Marginal, 8% were Useful and 2% was Good. The one Good player? Ken Holtzman, the very first #61 pick in 1965. There has been a total of 87.8 WAR generated with the #61 pick of the draft, much better than the 36.6 WAR generated by the #45 pick of the draft, but neither are anything to crow about, only 3% between the two had Good players when all was said and done.
It is just slim pickings, no matter where you pick in the second round, early or back of the round. And it just gets worse from there.
Why the Giants Farm Looks Bad
And, really, why any team who is regularly a contender, tends to have bad farm systems: it is very difficult to find a good player via the draft. But the conundrum there is that the draft is the best place to pick up prospects.
And the look at Hjelle and Suarez's draft positions shows how bad the odds are. Heck, even Bart at #2, the odds are really against a team, with a quarter of the prospects providing no value, and another quarter providing middling value. And only roughly a third is good.
Hjelle and Suarez's draft positions gives a good view into the second round of the draft and the value you get there. In both cases, less than 5% has ever become a good player, and together, works out to 3% chance of finding a good player. Assuming that's the second round success percentage, that means over 33 years of drafting in the second round, a team on average have found 1 good player. And that's with the second round!
Assuming 3% chance over 33 drafts, here are the probabilities of success, assuming randomness, by number of good players found:
- 0: 36.6%
- 1: 37.4%
- 2: 18.5%
- 3+: 7.6%
Bart's #2 pick gives a good view of why tanking teams generally keep the losing going at least 3-6 years. At 32% success rate, you average one success every 3 drafts. Here are the probabilities of success, based on the number of good players found:
- 0: 31.4%
- 1: 44.4%
- 2: 20.9%
- 3: 3.3%
So, 68.6% of the time, that team will find that good player, but nearly a third of the time, the team has to try again, because they struck out on the three. And in the fourth year of losing 68% of the time they strike out on the four, and thus overall around 20% of the original team still have nothing. And at sixth year of losing, 10% of the original teams still have nothing. Drafting is hard, even if you have a very high pick every year for a number of years.
Odds of Giants Selecting Three Good Players Consecutively
The odds with the #2 pick gives a nice view into how good Sabean and the Giants were in picking Lincecum, Bumgarner, Posey, three good players. Had they the second pick three years in a row, they would have gotten 3 stars only 3.3% of the time. However, they had worse picks - 10th, 10th, 5th - and accomplished that still.
Simplifying the stats, 24% of the 10th picks were successful Good players (Giants actually had 4 of these 50 picks, and 2 of the twelve good players: Craig Landis, Mark Grant, Lincecum, Bumgarner; oddly enough, each has improved on the prior, WAR, respectively were No MLB, 2.1 WAR, 19.7 WAR, and 31.6 WAR and counting). The odds were actually worse with the 5th pick (hence the reason why in my original study, I group picks together collectively, as long as they seemed to be similarly likely to find a good player): 18% of the 5th picks were good.
Combining these odds, by pure randomness, a team could have the same 10, 10, 5 picks overall, and the odds of finding 3 good players was 1.0%. So the Giants and Sabean were either very lucky or very good, because if it was random luck, the odds were very low that he would have hit that trifecta of Lincecum, Bumgarner, Posey.
There is a good history of evaluating their prospects under Sabean, and by comparing the prospects they kept vs. the ones that they let go, they have been much much better at keeping the good players and letting go of the non-prospects, with Francisco Liriano, Keith Foulke, and Adam Duvall being the biggest misses. Sabean has not lost much in terms of prospects in the trades made during his reign as GM and VP of Operations. That's a sign that the Giants have been excellent in evaluating their own prospects, once they had them in their hands, and could work with them.
In addition, research has shown that teams, generally, know who they got that is good, and usually keep their control over them. Sabean has been good at that, in the extremes, I would say, compared to, say, Beane, who regularly trades away good players like Ethier, Cargo, Addison Russell, and Josh Donaldson, without getting a good player back, for his troubles.
Another way to look at this is to examine how much of your team is homegrown. Fangraphs did a study last season looking into WAR production by how the players were acquired and the Giants were among the leaders in WAR production by their homegrown players. You can also see it in the lineup, with Posey, Belt, Panik, Crawford, and sometimes Williamson and Sandoval, in there, plus Bumgarner in the rotation.
All in all, the Giants under Sabean has been pretty good at finding good players in the draft, and just as importantly, keeping them when they had them in their hands. Great first round picks in Cain, Lincecum, Bumgarner, Posey, Panik, and other picks later like Crawford and Belt.
I was musing to myself last night. Chris Stratton was what, a # 20 overall pick? I bet that if Chris Stratton has a career-ending injury tomorrow and never pitches again, he would have to be considered and successful pick based on probabilities of success from that draft position.
ReplyDeleteThat is exactly why I did my draft study long ago, in my prior handle, Biased Giants Fanatic. The best study I could find was by Baseball America and their measurement of success was that the player made the majors, which shows a very high success rate. Instead, I made the dividing line whether the pick became a good starter, instead, which was a much higher standard.
DeleteBaseball Prospectus followed in my footsteps but did not give me any reference or credit, which showed either a lack of ethics, academic training, and/or plain laziness on their part, given that I published my results on Yahoo and that article got discussed by various places, including Baseball Think Factory, which tore me a new one for saying that there is a rationale and logic for punting the draft pick. BP chose a very poor way of examining the draft, by just looking at the average WAR (or whatever they called their metric back then) by pick, showing that the WAR was downward sloping (which anyone could have told you), but ignoring the elephant in the room, the fact that the average is useless when the data points are very skewed, the vast majority of prospects fail to have a significant major league career, even top first round picks, even the first picks overall.
I check every once in a while, and there have been a few that used my categorical method of analysis, but most, instead, just redid the BP study and came up with updated, but still useless results.
For example, they found that the younger players drafted in the first round tended to outperform those in the same year but older. Shankbone asked me about that, and it might be nice to look at the draft, after the fact, but it is not usable for a team when picking who to draft, you can't just pick the youngest prospect, the draft picks are biased by the fact that teams think that they are the best player available, for the most part, when selected. Findings that are not actionable, I find to be a waste, an interesting datapoint, but not usable in any way in making a decision.
From my viewpoint, Stratton is barely useful, at 1.6 bWAR so far in his career, to today. He needs another 7.4 bWAR to reach there, and if he continues what he's doing this season, won't reach that threshold in his career. He would need four average (2.0 bWAR) seasons to reach useful status per my methodology.
Thanks for the comment DrB.
I’m unconvinced that Sabean was wrong to trade Duvall, who was worth 2.6fWAR in his best year, 1.7 the next, and 0.4 this season. Similarly Jeremy Accardo had a good year, then less good, and finally more or less petered out, as I recall.
ReplyDeleteOff this topic, ogc, I thought while reading Russell Carleton’s *The Shift: the Next Evolution in Baseball Thinking*, that you would enjoy it a good deal. It goes beyond sabermetrics, which Carleton knows well (he writes for *Baseball Prospectus*), to add in human elements (he has or had a degree and practice in clinical psychology). One chapter closely analyses, at a level that would have Brisbee-followers agape, whether a power hitter such as David Ortiz should bunt against the shift; another, elaborates on the duty of a manager to keep a troop of twenty-five guys working as a team through the exhausting grind of 162 pre-playoff games, despite the adolescence of some, the domestic turbulence in others’ lives, cultural differences, slumps, crises of confidence, etc. Take a look at it.
I was fine with the Duvall trade, I liked Leake and was hoping we would sign him, when the trade was made. I was happy to end up with Samardzija instead, even at the cost of the draft pick, though with his ups and downs, the success of that signing is still in question.
DeleteStill, he was one of the few trades where the Giants gave up someone worth something. Let's take a look. I use BB-Ref for WAR, so he's been worth 6.3 bWAR. As I noted in the above, that's far from a Good player at 18.0 bWAR, and not even at the threshold yet of Useful player, at 9.0 bWAR. So he's been one of the highest WAR traded away, and yet, he's not even considered useful yet. Which illustrates how good Sabean has been at keeping the Good players.
Thanks for the tip!!! Greatly appreciate it!!!
I have actually bought it already, I love his columns (previously PizzaCutter) at BP and prior to that. But I have not made time to read yet. I will, now that you recommend it to me, I appreciate the recommendation, and now prod, to read it.
And Sabermetrics is never off topic, that is the heart of my blog.