ogc thoughts
The old methodology did this, as per the data provided regarding the difference between old and new shows:
PQS: DOM starts had 2.32 ERA (from 2012-2014), DEC had 4.54 ERA, DIS had 11.20 ERA
newPQS: DOM starts had 1.56 ERA, DEC had 3.70 ERA, and DIS had 7.92 ERA.
These make sense, but there are problems when you look deeper.
newPQS Doesn't Make Sense
So DEC looks decent and that's good but when you look at the underlying stats, newPQS does not make sense:
0 newPQS: 10.01 ERA; 2.23 WHIP; 4.8 K/9; 1.0 K/BB
1 newPQS: 6.75 ERA; 1.86 WHIP; 5,8 K/9; 1.4 K/BB
2 newPQS: 4.56 ERA; 1.45 WHIP; 7.3 K/9; 2.4 K/BB
3 newPQS: 2.89 ERA; 1.16 WHIP; 7.7 K/9; 2.9 K/BB
4 newPQS: 2.00 ERA; 0.91 WHIP; 8.5 K/9; 4.6 K/BB
5 newPQS: 0.95 ERA; 0.71 WHIP; 8.9 K/9; 7.5 K/BB
NL 2012-14: 4.03 ERA; 1.30 WHIP; 7.2 K/9; 2.6 K/BB
How is 2.89 ERA and 1.16 WHIP just DECENT? The K/9 rate there is not great, just barely above average, but a 2.89 ERA is very good and 2.9 K/BB is good, and yet it is not categorized as good start.
newPQS Doesn't Work Well
Looking over the majors pitching stats for 2012-2014, the average ERA was roughly 4.00, WHIP was roughly 1.30, strikeout rate was roughy 7.2, K/BB was roughly 2.6, which makes 3 newPQS at 2.89 ERA, 1.16 WHIP, 5.3 K/9 and 2.9 K/BB a pretty good pitcher. The whole point, I thought was to find the dominant pitchers, and 2.89 ERA is pretty dominant, so is 1.16 WHIP.
On top of that, from 2012-2014, there are only 5 starting pitchers out of 131 qualifiers that had an ERA better than 2.89. And for the individual seasons: for 2012, only 9 of 88, for 2013, only 10/80, and for 2014, only 19/88 had an ERA of 2.89 or better. 2.89 ERA is pretty elite, and these pitchers did it over a whole season, not just in one 3 newPQS start.
newPQS works for what it is intended, which is to find dominant pitchers who would be good for fantasy league teams: high strikeout and high K/BB ratio (which is linked to low ERA). For example, I took pitchers in 2017 with at least 9 starts (unfortunately, was unable to separate out starts from relief, so there is a mix) and split them up into various K/BB buckets (that are roughly about the same number in each):
GT 3.5: 3.44 ERA; 1.131 WHIP; 9.62 K/9; 4.42 K/BB; 8.00 H/9
3.0-3.49: 3.70 ERA; 1.231 WHIP; 9.32 K/9; 3.12 K/BB; 8.09 H/9
AVG: 4.34 ERA; 1.352 WHIP; 8.4 K/9; 2.49 K/BB; 9.33 H/9
2.5-2.99: 4.41 ERA; 1.358 WHIP; 7.90 K/9; 2.74 K/BB; 9.24 H/9
2.0-2.49: 4.69 ERA; 1.396 WHIP; 7.33 K/9; 2.21 K/BB; 9.24 H/9
LT 2.0: 5.04 ERA; 1.507 WHIP; 6.88 K/9; 1.68 K/BB; 9.46 H/9
As you can see from the above, if we had a pitcher who pitched 3 newPQS in each and every game in the season, and averaged 2.89 ERA, 1.16 WHIP, and 2.9 K/BB, he would rate among the best pitchers in the league, but according to newPQS methodology, he would only be considered decent.
So I'm at a crossroads. I'm probably not going to use newPQS during the season, as my goal is to illuminate which pitchers have been dominant.
Below are the oldPQS stats:
0 PQS: 12.08 ERA; 2.59 WHIP; 7.4 K/9; 1.3 K/BB
1 PQS: 7.21 ERA; 1.95 WHIP; 3.8 K/9; 0.9 K/BB
2 PQS: 5.84 ERA; 1.72 WHIP; 5.4 K/9; 1.5 K/BB
3 PQS: 4.07 ERA; 1.41 WHIP; 6.0 K/9; 2.0 K/BB
4 PQS: 2.84 ERA; 1.10 WHIP; 7.4 K/9; 3.3 K/BB
5 PQS: 1.74 ERA; 0.86 WHIP; 9.5 K/9; 5.1 K/BB
NL12-14: 4.03 ERA; 1.30 WHIP; 7.2 K/9; 2.6 K/BB (starting pitcher stats)
 |
| oldPQS vs. newPQS Distribution 2012-2014 source: 2017 Baseball Forecaster |
The main impetus, it seems, from the study explaining newPQS, is that the author of the study, who is not the same person as the one who created PQS in the first place, wanted the distribution of pitchers to be a normal curve from 0 to 5 PQS, and that is a good way to separate out the good pitchers from the bad, per fantasy purposes.
But that is not what I've been wanting for my studies, I wanted to be able to identify good starts, and then through a compilation of them, be able to see whether the starting pitcher was good or not, in order to better understand the Giants starting rotation, and whether it is competitively good or not. I don't care that the distribution is now closer to a normal curve, nor do I care that DOM starts means the pitcher has struck out a lot, all I care about is identifying a quality start from a non-quality start, as many as there happen to be, damn the distribution. And he has classified a bunch of good pitchers in the 3-PQS category.
That newPQS is a more normal curve is nice, but again, the purpose there is to find the best fantasy pitchers out there, whereas I want a more nuanced result, which is to find out which pitchers are good and which are bad, at delivering good results, that is, good ERA. newPQS separates out pitchers who are good but not dominant, and I want to see all the good as well as the dominant starters. That's what 5 PQS is for, anyway.
I think I have no choice but to branch off now and do my own thing. Unfortunately, I don't have a huge database of starting pitching game stats to perform analysis on and craft a perfect distribution, so I'll have to work from what was provided in the analysis leading up to newPQS. Which is okay, I was plenty happy with what PQS had previously provided me, so these tweaks should make me happier. I will go over each point category here.
Here is the data chart (if anyone knows how to copy a portion of the screen, and not the whole screen, please share):
 |
| Source: 2017 Baseball Forecaster |
However, I don't like the change to pitching into the 7th. 62% of all pitchers pitch at least 6 innings, but only 40% make it into the 7th and successfully get an out. Even when pitchers were completing more starts than they were taken out, 6 innings was considered a quality start. And in today's world of not allowing pitchers to face the lineup a third time (as exemplified by the Dodger's dogmatic usage of Rich Hill, especially in the playoffs; and to be fair, newPQS was designed before teams started this practice), it will be even harder for good pitchers to finish 6 innings. I'm going to keep the 6 innings minimum.
That was another thing that bothered me about newPQS, as it appeared that the changes were not being made based on what constituted a good start, it was based on making the distribution more normal. The new author took the methodology off it's original goal, in my opinion, of using sabermetric discoveries to inform the Quality Start metric. And that was the original genesis of the PQS methodology, using sabermetric discoveries to identify when a pitcher was pitching well, and not just when a pitcher was having a good start.
I am okay with changing the hits logic to being less than the number of innings pitched. I don't know how related either is to being a good pitcher, other than, obviously, less hits means a better pitched game. I liked this rule because it helped to recognize pitchers who do happen to have a lower BABIP than the general mean, like Cain, but I don't know if there is any way to say what exactly is a good number of hits to give up in a start, as BABIP gods can rear their ugly heads
Looking at the starting pitcher data I collected in the table above categorized by K/BB ratio, there is a clear demarcation, though, at 3.0 K/BB: below that, H/9 is over 9.0, whereas above that, roughly 8.0 H/9. So clearly giving up less hits is a clear sign of a superior pitcher, so this change is a really in line with what the data says about separation between pitchers, of identifying who is good and who is not. And thus I'm satisfied that this change will yield a better definition of a good DOM start.
I especially like the change to a minimum of 5 strikeouts. This makes it much easier to score a start's PQS. Of course, the motivation of the author was to get the percentage down to roughly 50% (he was very fixated on getting each individual metric to be 50%, which I understand from the viewpoint that in fantasy baseball, you want to distinguish who is the better pitcher) and not on what is considered good. I didn't like that a pitcher could get a point for 3 strikeouts in 5 IP, so I'm okay with this.
And as nice as 4 strikeouts in 6 innings are, in today's baseball, that's only a 6.7 K/9, and the average K/9 in the NL for starting pitchers in 2017 was 8.0! Even 5 in 6 innings only get you to 7.5 K/9, but at least that is closer to the reality in the MLB that strikeouts is going way up. And obviously 7 strikeouts in a complete 9 inning start would only be 7.0 K/9. I would have been okay with mentally calculating the K/9 by each start (as I'll explore next, not that many situations to remember), but this does simplify the metric. I think a better rule would be this: minimum of 5 strikeouts, but need IP-2 to earn a point if IP is greater than 6 IP, because now there are situations of 5 strikeouts in 8 or 9 IP, which is a very low K/9. But that's complex to remember, and I like the simplicity of 5 strikeouts, so I'm going with that.
I get why he chose to push K/BB to to 3+, because that's easier to calculate in your head when scanning the boxscore. But the average is still around 2.5 K/BB (2.54 K/BB for NL starters in 2017, in fact), and I've never had a problem calculating fractions in my head, I've been doing it since I was very young, as I loved math as a kid. And as I noted, the author was fixated on getting to 50%, and using 3 (or if BB=0, K greater than or equal to 3), got him to roughly 50%. So I'm going with 2.5, which he even mentioned in his study, but dismissed because of ease of calculation, plus the caveat for zero walks (that'll be harder to remember, though).
Plus, after a short while, you learn what is and isn't greater than 2.5. If there's 1 walk, you need 3 strikeouts anyway, but 2 walks get you to 5 strikeouts (instead of 6), 3 walks get you to 8 strikeouts (not the 9 the new rule would require), and 4 walks gets you to 10 strikeouts (not 12). Any more walks, forget about it, probably not getting that point, even getting the point with 4 walks happens pretty rarely, I would bet. Do it for a while and you quickly remember the above (hence my point above about strikeouts).
So I'm going to use 2.5, as half the time (assuming walks are evenly distributed between even and odd), the ratio is higher than 2.5, and so it will even out over time to be higher than average. And that's the beauty of this methodology that I like, that things do tend to even out for borderline type of stats. And for 0 walks, I won't require a minimum of 3 K's, I'm giving it to the pitcher as long as he has at least one walk. As I noted, don't know how these changes will work out for the overall PQS stats, but this makes sense to me, as a sign of a good starting pitcher.
Lastly, recognizing only no homeruns as a PQS score makes that really easy to calculate, it is an either or situation. I'm kind of shocked that 52% of starts have zero homers and that 34% have only one homer, given the home run surge of the past few seasons. And I would hate having to give a zero to a starter who pitched 7-8-9 innings and only gave up one homer. Again, here the math would be easy to recognize, only give a point to those giving up one homer in 8-9 innings (or simply, a complete game). But it's easier just to go with no homers at all and that covers over half the starts, so I'm good with that.
Since I don't have the started games data, I have no idea what all my changes does to PQS distributions or what a good DOM PQS percentage would be with the ogcPQS. Since these are tweaks to the old formula, and as shown in the graphic above, some reductions, it will definitely be harder to get a DOM going forward, but much harder to get a DIS, given the dropping of the auto-zero rule. I'm okay with my changes being conducive to identifying a well-pitched game, it is better than going with newPQS and my making a 3-PQS start a DOM start, which is another way I could have played it.
I will reference this post in my PQS analysis going forward, as well as the originator's, to give them credit. I will let this sit a few more days, before I go with my new methodology, in case I have any last second thoughts (but this has been worked on for a few weeks already, so I think this is it).
No comments:
Post a Comment