[ogc's note: I tried to publish this on Fangraph's Community Blog section, but just realized that nobody has published there for over a year. If it ever makes it through their review process, I will take down this post, and replace it with a link to Fangraph's version. Thank you for understanding.]
This is a study of how well the top ranked NL teams in Runs Scored and Runs Allowed have fared in the playoffs during the Wild Card era.
I've been interested in the best ways to make the playoffs for a long while because the team I'm a fan of never made the playoffs much, let alone won a World Series championship, for many decades, for the entirety of their existence in this city. In the mid-2000's, I ran across two similar but different studies into Billy Beane's statement that his "stuff" didn't work in the playoffs, so I was very interested in if my team was following the findings.
While my study isn't as overarching or as statistically sophisticated as these two, I wanted to first recap their findings, so that one can see the difference between their datasets and methodologies and what I analyzed.
Studies of Success in MLB Playoffs
The findings of both Baseball Prospectus (in its chapter on Billy Beane) and Fangraphs The Hardball Times was both obvious and controversial: both found that it is pitching and fielding excellence that leads to success (winning) in the playoffs, and in going deep into the playoffs. The more controversial finding was that offense wasn't much of a factor in teams being successful in the playoffs. Each used similar but widely different methodologies.
The Fangraphs article compared dozens of offensive and defensive metrics of each playoff series' opponents and gave wins and losses based on whether the one with the better metric won or loss. Then it narrowed this win-loss data further by requiring a significant difference to record the wins and losses, as a way to filter out the borderline cases. 9 of the Top 12 were pitching or fielding metrics. And only one of the three offensive metrics were related to hitting, the other two were related to base stealing.
Baseball Prospectus used a scoring system, Playoff Success Points (PSP) to rate each playoff teams' success in going deep into the playoffs, and used correlation analysis to investigate various metrics vs PSP, a much different and statistically sophisticated methodology, but still came to the same conclusion, that pitching and fielding defense is vitally important in the playoffs, while offense doesn't. It only found one offensive metric of any statistical significance, which was base steal attempts (not stolen bases, nor successfully stolen, simply total steal attempts). Through a variety of more sophisticated statistical analysis, using regression analysis and tying it to PSP, BP came to the startling conclusion that "while preventing runs correlates with post season success, scoring them does not."
They narrowed down a long list of significantly correlated metrics, only one of which was related to offense (which as noted was base stealing attempts, there were none related to hitting), down to their secret sauce: a dominating pitching staff (as measured by K/9), a good closer (as measured by their WRXL metric, which they no longer calculate), and a good fielding defense. They no longer use this secret sauce, but that's another story*.
(* Side Note: Unfortunately, they no longer use this because a few years after publishing the book, they published a few articles reporting on that year's playoffs, and it did not do well in prediction. But they clearly misapplied the results, which resulted in the poor results that caused them to stop using it. Instead of comparing current playoff teams against each other, as they did in these annual analysis, they should have placed each new playoff team into their historical dataset and see where they ranked historically, and see what similar PSP teams did in the playoffs, as that is what they did to create their original Top 10 list, which had 9 of the 10 making the World Series, 8 of the 9 winning the World Series, with the one loser losing to one of the 8).
Methodology
I wanted to try my own study of the playoffs and came up with this idea: comparing the top NL Runs Allowed teams vs. the top NL Runs Scored teams. I wanted to see what type of findings I could come up with using this simpler methodology, as my data analytics skills is still nascent, but I still wanted to try something.
I decided to just use the years since the Wild Card format was implemented, as that's a slightly different set of data than the divisional playoffs history, given that the original studies above were done to 2003 and 2005, respectively, so my study would have around 20 years of new Wild Card playoff data utilized in my study, while overlapping slightly with the two above, which covered the divisional playoffs which started in 1972. Given a much different set of data, though with some overlap, I was testing to see if the new playoff format led to a different conclusion than the two studies above.
So I investigated how the top 5 RA and RS teams did in the playoffs since 1995. I collected the Top 5 teams over the whole period, despite the recent change from 4 to 6 playoff teams, to keep the data consistent, plus the fact that Top 5 already covers a third of the NL teams. Then I split them into three groups: Dual Leaders, where the teams were both RA and RS Leaders, RA Leaders, where the teams were only RA Leaders, and RS Leaders, where the teams were only RS Leaders.
Playoff Results by RA and RS Rank
Overall, it was clearly better to be an RA Leader and best to be a Dual Leader:
- Total teams: 55 Dual; 90 RA only; 90 RS only Leader Teams
- Total Playoff teams: 53 Dual; 47 RA only; 22 RS only
- Total NLCS teams: 24 Dual; 21 RA only; 8 RS only
- Total WS teams: 11 Dual; 11 RA only; 4 RS only
In total 69% of RA Leader teams made the playoffs vs. 52% of the RS Leader teams. But the closeness of that was influenced greatly by the fact that 53 of the 77 RS Leader teams were also RA Leader teams. Separating them out, 96% of the Dual teams made the playoffs (which makes total sense), 52% of the RA only Leader teams made the playoffs, and only 24% of the RS only Leader teams made the playoffs. Pitching and Fielding excellence is much better at making the playoffs, than Hitting excellence, with over double of the RA only Leader teams making the playoffs vs. the RS only Leader teams.
Clearly, if you want to make the playoffs, you need to be a RA Leader team, as those leaders make the playoffs over double that of the RS only Leader teams. In total, 69% of the RA Leader teams made the playoffs, and took 75% of the available playoff spots, meaning that they took 3 out of every 4 playoff spots usually. Meanwhile, if you were only an RS Leader, only 24% of them made the playoffs, and took roughly one out of every four playoff spots. Being an RA Leader means likely being a playoff team, as well as going deep into the playoffs, whereas RS only Leader teams struggle to make the playoffs (about a quarter), and does not go as deep into the playoffs.
Moreover, 22 of the 29 World Series NL participants (NLCS winners) were RA Only Leaders or Dual Leaders, and only 4 of the RS Only Leaders reached (and to put that success rate into perspective, as we'll see later, 3 non-leader teams made the World Series). And 11 RA Only Leaders (23% of RA Only playoff teams; 12% of all RA Only Leaders) vs. 4 RS Only Leaders (18% of RS Only playoff teams; 4% of all RS Only Leaders) were in the World Series, nearly triple in number, even though RA Only Leader teams slightly doubled RS Only teams in playoff participation (47 RA Only vs. 22 RS Only).
Ranking Comparisons
Overall, the Top RA teams got into the playoffs more often than the Top RS teams, 69% vs. 52%. Comparing the rankings means much less datapoint per population, but it was still interesting to see the results.
Each ranked RA team did better than the similar ranked RS team from 1 to 5, by a large margin:
- RA 86% > RS 66%
- RA 76% > RS 55%
- RA 62% > RS 52%
- RA 76% > RS 59%
- RA 45% > RS 28%
Based on this data, being a Top 5 RA team is a lot more effective for getting into the playoffs than being a Top 5 RS team, when comparing overall ranking. In addition, the addition of the second wild card team, starting in 2012, has made it more of an advantage of being an RA Leader team, as 9 of those 12 RA #5 teams made the playoffs, vs. only 3 of the RS #5 teams.
Ranking Comparison using PSP
Using Baseball Prospectus' Playoff Success Point (PSP) scoring (which awards 3 points for making the playoffs, another 3 points for winning the LDS, another 4 points for winning the LCS, and another 4 points for winning the World Series, and then +1 for each postseason win and -1 for each postseason loss; there were no Wild Card games in their data set, so I just counted the +/- 1 for each Wild Card game), I found similar results.
An average total of 17.8 for the Top RA teams was tallied, versus 13.3 for the Top RS teams tallied, or 34% better overall. So, not only were they more successful in getting into the playoffs, this datapoint confirms that the RA teams also did much better overall in going deeper into the playoffs.
Cutting deep into the data, for RA Only and RS Only, across the Top 5, while the data points count ranges from 14 to 24 (less than the 30 data points that text books teaches), I thought it would be interesting to present:
The RA Only teams clearly outperformed the RS Only teams also when comparing by ranking. The majority of the time, RA Only teams will make the playoffs, and when they do, they are much more successful than the RS Only teams, as we can see with the PSP averages. Oddly, the #5 team in both RA and RS has done very well in the playoffs, second only to the #1 rank.
Here is the data for the Dual leaders by their ranks in RA vs RS (total data points ranged from 5 to 15):
Clearly it is best to be a Dual Leader, as noted before and clear to any baseball fan. The Average PSP is much higher for most of the Dual splits than it is for the RA Only and RS Only splits by rank. There is randomness by rank, but compared to their RA Only and RS Only fellow rankers, they are obviously much more successful in the playoffs, which make obvious sense, they are best in both, and not just one. However, being 4th or 5th in RS hasn't been as good a combination for Dual Leaders going deep into the playoffs, even if they are a Top RA Leader.
In any case, the NL playoffs are dominated by RA and RS leading teams. There were a total of 133 playoff spots in the 29 years of playoffs, and these teams took 122 of them, or 92% of them. These Leader Teams also went to the World Series 26 of those 29 seasons, or 90%.
Recent Trend: Dual Leaders Are More Prevalent
Dual Leader teams have become more common in recent years. In the first 17 seasons, there were 27 Dual Leader teams (1.6 per season). In the 12 years since, there has been 28 Dual Leader teams (2.3 per season, 47% higher), and 12 Dual Leader teams in the past four seasons (3.0 per season, nearly double). As shown above, being a Dual Leader has meant making the playoffs almost 100%, with only two teams not making the playoffs.
Once making the playoffs, things are as expected for Dual Leader teams.
- 21% of the Dual Leader playoff teams made the World Series (20% of all Dual Leader teams)
- 23% of the RA Only Playoff teams made the World Series (12% of all RA Only teams), and lastly,
- 18% of the RS Only Playoff teams made the World Series (4% of all RS Only teams).
Altogether, 45% of the Dual Leader playoff teams made the NLCS (44% of all Dual Leader teams), 45% of the RA Only Playoff teams made the NLCS (23% of all RA Only teams), and 36% of the RS Only Playoff teams made the NLCS (9% of all RS Only teams). Again, RS Only Leaders do not perform as well as the RA Leader teams.
Wild Card Effects
The Wild Card has helped a lot of the RA/RS Leader teams make the playoffs. Out of the 40 Wild Card teams (excluding 2020, when all the teams were Wild Card teams), 35 of them were RA Only, RS Only, or Dual Leader teams (85%). And 5 non-leader teams got into the playoffs via the Wild Card.
The oddity here is that 3 of the 5 non-leader teams made the World Series. Stranger still, gaining the Wild Card spots as a Dual team was not that great, only 1 World Series out of 11 teams, 5 NLCS. Meanwhile the RS Only teams made 3 World Series out of 9 teams, 4 NLCS. The RA Only teams ruled for Wild Card teams, with 4 World Series out of 15 teams, and 7 NLCS.
Making the Playoffs: The Non-Leaders
Altogether, from 1995 to 2023, there were 133 playoff teams, with 53 Dual Leaders, 47 RA Only Leaders, and 22 RS Only Leaders, which means that there were 11 Non-Leader teams that made the playoffs, or 8% of the playoff teams were not categorized as a leader in that season. I thought it could be interesting to explore these 11 teams, mainly because I noticed that sometimes the line between a leader and non-leader was very slim.
In total, there were 133 playoff teams from 1995 to 2023, and 122 (92%) of them were a leader in RA or RS or both, and 11 (8%) were not a leader. That's not surprising, obviously, to make the playoffs, you have to be either pretty good at scoring runs, preventing runs, or both. Still, I was kind of shocked to see 8% of playoff teams being non-leaders. And the wild card helped 5 of the 11 non-leaders make the playoffs.
Looking into the 11 teams, I found that many of them were within 6% of the least of the leaders. 8 of the 11 (73%) were within 6% of the qualifying RA, and 5 of the 11 (45%) were within 6% of the qualifying RS. This reinforces what we saw when comparing RA and RS leadership, where being an RA Leader is better than being an RS Leader. And 5 of the 11 were within 6% in both RA and RS.
As one can see, many of these non-leaders were really close to being a leader. Had I extended the number of teams counted as a leader by one, that would have converted 6 of the 11 teams into leaders. Just amplifies the fact that being a RA or RS leader is what is necessary to being a playoff team, which is obvious to most fans.
In addition, the dual near leaders were a very successful grouping. There were two World Series winners (2003 Marlins, 2012 Giants), one World Series loser (2023 D-backs), and one NLCS losers (1996 Cardinals) out of the eleven non-leaders. They were highly successful, which mirrors what we saw with teams who were dual RA/RS leaders.
Defense Rules!
The findings of Baseball Prospectus and Fangraphs/TheHardballTimes were supported by this study of the Wild Card playoffs era in the NL. Teams that were good at Runs Allowed/Prevention (i.e. defense) got into the playoffs (69% of the RA teams; 75% of all playoff teams) many times more often than teams that were only good at Run Scoring (24% of the RS teams; 17% of all playoff teams). Even the teams that were only good at Runs Allowed/Prevention, but not run scoring, made the playoffs over twice as often (52% of all playoff team vs. 24%) as the teams that were only good at Run Scoring.
Furthermore, 11 teams made the playoffs not as a leader in either category, versus the 22 teams that made the playoffs as only a Runs Scored leader. 8 of the 11 teams were within 6% of being a Runs Allowed leader and 5 of 11 teams were within 6% of being a Runs Scored leader, and basically half of the 11 teams were Dual Leader adjacent, showing again the potency of being good at both Runs Allowed and Runs Scored.
All of these findings from this study, that it is defense (RA excellence) that more frequently leads to a playoff spot, and thus towards deep runs into the playoffs, aligns with the findings from the Baseball Prospectus and Fangraphs/The Hardball Times studies. My study also showed that simply making the playoffs is strongly associated with RA excellence, and, as well, deeper runs in the playoffs.
I actually started this study actually looking into something else and ended up capturing the data to do this analysis. I plan on studying about these effects for AL playoff teams as a follow-up, as well as shorter articles looking into some other playoff data points collected.
No comments:
Post a Comment