top of page
file.jpg
  • Writer's picturePaul Semendinger

The Yankees Made a Classic Data Mistake

By E.J. Fagan

August 22, 2023

***

NOTE: The following comes from EJ Fagan's substack page and is shared with permission.

Please check out EJ's substack page for more great articles.

***

Recently, former Yankee minor league farm hand wrote about the deficiencies of the Yankee farm system in the comments on Foul Territory. Erik Kratz pretty much confirmed everything he said:

Ruta describes a lot of weird drills that Dillon Lawson and a lot his proteges had hitters going through, like measuring the velocity that a player can throw a medicine ball from his hitting stance. But the important theme to his and Kratz’s comments is that the Yankees were obsessed with maximizing exit velocity. Players would get points, presumably leading to promotions or demotions, based on how often they hit the ball hard.


I’ve been writing about how hitting the ball hard isn’t always a good thing. But let’s ignore that for now. Hitting the ball hard is one way to succeed in baseball. I want to ignore it because I think the Yankees are running into one of the classic mistakes in the use of metrics in an organization: Goodhart’s Law.


Metrics are Dangerous

Goodhart’s Law, which we usually apply to government or private organizations that use metrics to make decisions is that, “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” It means that if you rely on a narrow set of metrics to make important decisions, your employees will adjust their behavior to increase the metric at the cost of other things that might be desirable but don’t increase the metric.


The classic example of Goodhart’s Law is standardized testing in schools. It’s hard to tell if a teacher or school is doing a good job, but we want our schools to be better. In the 2000s, largely due to George W. Bush signing the No Child Left Behind Act into law, we started to use standardized testing to determine if students were being taught successfully. If too many students failed the standardized tests, we punished the school. In some states, individual teachers were rewarded if their students did well on the standardized test.


What happened was entirely predictable: teachers started to teach to the test. They spent more and more of their time making sure that their students learned exactly what they needed to do well on the standardized test. Things that weren’t on the test that teachers used to teach because they thought it was valuable were cut. Students suffered because passing a standardized test is not a very useful skill. The federal standardized testing regime quietly collapsed when both parties decided it wasn’t working in 2015.


When you tell employee that the most important thing is that they maximize a metric, they will maximize the metric. In the case of exit velocity, they will swing harder. All the time.


Regardless of the situation. Swinging hard all the time may be an okay solution in the low minors, but I don’t think it’s a coincidence that so many top Yankee hitting prospects have stalled out at Triple-A or the majors, where batters have to deal with way more good breaking stuff. When you tell prospects that only exit velocity matters, they will sell out for exit velocity.


The Better Way to Make Decisions with Data


Okay, Dr. Fagan, so the Yankees shouldn’t use statistics to evaluate prospects? No! Of course not. Goodhart’s Law doesn’t mean stop using data. It means that you need to do two things: take a more holistic view of data, and use metrics that are better aligned with the incentives that you want to create.


A more holistic view of data just means not to obsess over one metric. Exit velocity can be really good! But you might also care about strikeout rates, situational hitting rates, on base percentage, barrel rate, etc. You want to take a step back and get a complete picture of a player’s performance. You might also use non-metrics, old school evaluative methods to supplement a statistical analysis.


Using metrics that better align with incentives means focusing on outcomes not outputs. An outcome is the end result of something that occurs on a baseball field, such as an out, hit, home run, walk or strikeout. An output is something that is supposed to translate into that outcome, such as velocity, swing speed, spin rate or launch angle.


All outcomes are the product of outputs, but they can come in different ways. A 110 mph ground ball can produce a single, but so can a 70 mph bloop fly ball. Both are worth the same. In the long run*, it doesn’t matter how a batter produced their singles. They all count the same on the scoreboard.


In practice, focusing on outcomes means using real basic statistics like wOBA or slugging percentage to make decisions. Some prospects might increase their wOBA by hitting a bunch of bloop singles while others hit the ball really hard to produce more home runs and hard ground balls, or whatever combination of skills you can imagine.


Of course, the holistic rule still applies. A hitter might adopt a strategy that works in the low minors but won’t in the majors, like taking a ton of walks. For instance, Anthony Siegler hit .236/.405/.369 in 2022 between A-ball levels with more walks than strikeouts. It would be naïve to observe those data and predict that Siegler was about to breakout as a super high-OBP catcher. Higher level pitchers can throw strikes when they think a batter is just sitting there not swinging. And of course, they did. Siegler hit .171/.335/.253 at Double-A this year.


The Yankees need to get smarter about how they make decisions with data. There was a line in Oppenheimer that was something along the lines of, “I see the music in the math.” A good data-literature analyst sees the music, not the notes. They can synthesize a lot of information together and decide if a player is successful. A bad data analyst has a checklist that says, “this thing good, this thing bad.” When those analysts are given power, everyone will conform to their process.


It’s Not Working


Outcomes, not outputs. What is the ultimately outcome for a farm system? Developing major league players. The Yankees have shown success in developing major league pitchers, but utter failure in developing hitters. The Yankees first brought in Dillon Lawson in 2018. The farm system has graduated zero average or better major league hitters since then. Maybe Volpe and Peraza will correct that trend soon, but it’s hard to see their struggles as major league hitters and not attribute some of it to “max exit velocity all the time.”


Ultimately, Brian Cashman is the person responsible for the Yankees playing poorly. I think the Yankees would be better off with someone else, but at the very least he should consider a complete revamp of their minor league hitting coaching system. I wonder if we’re going to hear a lot more stories like Ben Ruta’s soon.

11 comments

11 Comments


etbkarate
Aug 22, 2023

Just like almost everything else today, people want short cuts and convenience. Unfortunately in life, there are no short cuts to success. Baseball is no different. You can use all the stats, data, whatever. At the end of the day you have to train harder, focus and execute. All those analytics do is overload a ball players mind. Its really very simple, see the ball, hit the ball. This isn't brain surgery. Its a game of will and determination. No one has bothered to ask all these analytics people how it was humanly possible for a Tony Gwynn, Pete Rose, or Wade Boggs, etc to hit so well without reams of info. Its a joke, and guys like Cashman…

Like

Frank Graziadei
Frank Graziadei
Aug 22, 2023

Cashman does not change. He is not going to change. He has been in total control since 2010 when he rid himself of the Tampa baseball advisors that he answered to before 2009, In his words, he had "a seat at the table". When he was in control he fired the other basebll voices at the table-all former major league veterans and surrounded himself with analysts and statistics to confirm his actions. Anyone who has ever studied statistics can tell you that they are great for advancing almost any argument. I do not believe that Cashman ever understood how to evaluate baseball players. Their emotions, their desire to win or what it takes to produce a winning team The Yanke…

Like
etbkarate
Aug 22, 2023
Replying to

You nailed it, Frank.

Like

cpogo0502
Aug 22, 2023

Achieving "max exit velocity all the time" results in over-swinging and a loss of natural mechanics, resulting in reduced contact and increased strikeouts. THERE ENDETH THE LESSON.

Like

Robert Malchman
Robert Malchman
Aug 22, 2023

"it doesn’t matter how a batter produced their singles"


Right, because we now know that singles are irrelevant. All that matters are walks, homers and strikeouts. So what if you strike out 18 times a game, if you hit a few homers with the walkers on base already? That's the Yankees way to play baseball. Trust the Process, even when the Process is leading to your first losing season in 30 years. /s


To paraphrase George Orwell, "But it was all right, everything was all right, the struggle was finished. He had won the victory over himself (if not over other MLB teams). He loved the Process."

Like
etbkarate
Aug 22, 2023
Replying to

As Gord Downey said, "now, the struggle has a name".

Like

Mark Reyer
Mark Reyer
Aug 22, 2023

I've read players reinforce Goodhart's Law by stating that exit velocity, home runs, etc increases their value. It's natural, if I can make more money with higher velo regardless of BA and strike out rate, then I'm doing what makes the most money.

Like
etbkarate
Aug 22, 2023
Replying to

That was long before analytics infested the game.

Like
dr sem.png

Start Spreading the News is the place for some of the very best analysis and insight focusing primarily on the New York Yankees.

(Please note that we are not affiliated with the Yankees and that the news, perspectives, and ideas are entirely our own.)

blog+image+2.jpeg

Have a question for the Weekly Mailbag?

Click below or e-mail:

SSTNReaderMail@gmail.com

SSTN is proudly affiliated with Wilson Sporting Goods! Check out our press release here, and support us by using the affiliate links below:

587611.jpg
583250.jpg
Scattering the Ashes.jpeg

"Scattering The Ashes has all the feels. Paul Russell Semendinger's debut novel taps into every emotion. You'll laugh. You'll cry. You'll reexamine those relationships that give your life meaning." — Don Burke, writer at The New York Post

The Least Among Them.png

"This charming and meticulously researched book will remind you of baseball’s power to change and enrich lives far beyond the diamond."

—Jonathan Eig, New York Times best-selling author of Luckiest Man, Opening Day, and Ali: A Life

From Compton to the Bronx.jpg

"A young man from Compton rises to the highest levels of baseball greatness.

Considered one of the classiest baseball players ever, this is Roy White's story, but it's also the story of a unique period in baseball history when the Yankees fell from grace and regained glory and the country dealt with societal changes in many ways."

foco-yankees.png

We are excited to announce our new sponsorship with FOCO for all officially licensed goods!

FOCO Featured:
carlos rodon bobblehead foco.jpg
bottom of page