NextGen Scouting Evaluation in Baseball

Being an obsessive, I follow a lot of baseball news to catch news about the Giants, and thus am interested in all parts of a baseball operations.  I've been participating in, basically, a crowdsourcing initiative, and their method of evaluating predictions I think is something that could be used in evaluating scouting, so I thought I would discuss my thoughts.

Baseball Prospectus Secret Sauce: Their Rationale for Ending It Was Incorrect

[NOTE: wrote most of this up before, just getting around to posting now]

Baseball Prospectus did a study for their great book, "Baseball Between the Numbers", of how teams historically, during the divisional baseball era, got deep into the playoffs, answering the question Billy Beane had, of why his sh!t didn't work in the playoffs.

They found, on an overall basis, that offensive stats was random and all over the place, meaning that it is defensively how teams got deep into the playoffs, that is, pitching and fielding.  They narrowed down the variables to three key metrics:  K/9 for the pitching staff, WRXL for their closer, and their Fielding Defense metric, FRAA.  The only offensive metric that seemed tied to going deep was stolen base attempts, which they attributed to the link of team speed to stolen base attempts.

For a few years, BP published analysis based on their "secret sauce".  But after a number of failures, they decided to end the analysis, making the assumption that the playoffs had changed in some way to make it fail as a strategy.