Do-It-Yourself: When Is a Prospect A Prospect?

There are a number of things, in addition to the statistics, that go into the making of a Prospect

Last week we published our Division I Performance Evaluations. At the time, we told you that, while they had a high degree of correlation to future success, they were not Prospect Evaluations. Today I’d like to touch on that topic a little further.

At Diamond Futures, our approach to Prospect Evaluation is a three step approach. Step 1 is our current state assessment, or what we call Performance Evaluations. What we do at this step is we take every player’s data for a particular period of time (usually a full-season), normalize the data for the park, and then calculate a score based on the five variables that we have found to be most significant (see last week’s Do-It-Yourself for more in-depth information): Age, Power, Average, Strike Zone Judgment and Speed for Hitters and Age, Dominance, Control, Stamina and HR rate for Pitchers. Once we plug that into our formulas we come up with a Performance Rating that is then normalized for Level of Play. It is a very straightforward process, completely systemized and all science (statistics actually). We know that from a full-season’s worth of data, that our formulas will yield a correlation coefficient in the high .40’s with future Major league success. We can tell you from that one number, and on just one season’s worth of data, the probability of that player ever achieving Major League success. At Diamond Futures we perform most of this analysis from the previous season in September and October and publish League by League ratings similar to our NCAA Division I ratings.

Step 2 is our comparative data analysis. Here we use the same statistical variables, but we combine the one year’s worth of normalized data with the data from every year of a player’s professional or collegiate career. We then add to that, things like birth month, height and weight data and their progression, handedness, defensive fielding information, country of origin, age at signing, signing bonus/draft position, etc. We take that information and compare it to our 12,000 player historical database and find the most similar players. From this we are able to not only forecast the player’s probability of success, but we can apply finer gradations to the actual level of future success. Again, it is a relatively straight forward process, completely systemized and all science.

Then we get to Step 3. While the first two steps are pure science, Step 3 is where the ‘art’ enters in. With the first two steps we can tell you what the probabilities are that a certain player will achieve success or how many players from a similar group will succeed, it is Step 3 that allows us to hone in on the other factors that we can’t pick up from our statistical analysis. Occasionally, Step 3 allows us to rank someone slightly higher than their numbers would otherwise suggest, but more often than not, we use Step 3 to drop players. Think of Step 3 as the pieces of information that would otherwise have been gleaned from traditional scouting methods. At Diamond Futures, we focus on essentially six areas. These are the areas that, once the ‘numbers’ highlight a player, we feel are the differentiators in “making a prospect a prospect.”

The first area is Age. While we use Age extensively in Steps 1 and 2, we can’t overstate the significance of age vs. the level of competition. 25yo Robinson Chirnos, may be posting a 1.116 OPS in the Florida State League, where the average age is approximately 23.5, but that doesn’t make him a prospect. While I like James Darnell, and do think he is a legitimate prospect, a 22yo with a .985 OPS in the Midwest League, where the average age is a tad over 22, doesn’t get me excited. By the same token, 17yo Jefry Marte, sitting with a .574 OPS in the South Atlantic League, isn’t really a case for concern. We have done some research and have found that over the last fifty years, approximately 70% of players that play in the Major Leagues, make their debut before their 25th birthday. Furthermore, those that debut before 25 have about a 38% chance of putting together 3 seasons or more where their performance is at or above Major League average. For the post-25’s, the odds drop to 8%. When you look at players that amass 5 seasons or more, at or above league average, the contrast is even sharper—29% to 4%. In other words, a player that debuts prior to turning 25 has a 7-8 times greater chance of having Major League success. If you figure that a standard Minor League progression is one level per year, and the goal is to reach the Majors before he turns 25, then we want prospects that are 24 or younger at AAA, 23 or younger at AA, 22 or younger at Hi-A, and 21 or younger in Lo-A. If you are older than those targets you have some ‘make-up’ work to do, and if you are more than a year older than those targets, it doesn’t bode well for your future.

The second area we look at to test a ‘prospect’ is Projectability. In 2001, with Kane County being the closest Minor League affiliate to where I live, I got to see 18yo, Miguel Cabrera play more than a dozen times. At 6’2, 185lbs it was difficult to envision the 6’4, 240lb, behemoth that he was later to become. In batting practice, you could watch him put on a tremendous display of both power and quickness with the bat, but that translated into a .268/.328/.382 line for the year…barely above a .700 OPS. But what you could see, was that he was going to get both bigger and stronger as he filled out. By the same token, I have seen Jaff Decker play a couple of times this year. At 19yo, he is 5’10, 190 lbs. While he is a hitting machine, and I believe he certainly has the potential to succeed at the Major League level, he will have to prove himself at every level, because no one believes that he will gain much if any size/strength as he matures. The same thing goes for pitchers. We are far more tolerant of bad numbers from an 18yo, 6’4, 180lb high school senior than we are of a 22yo 6’2, 210lb college junior. They both may throw 92MPH fastballs, but only one of theirs is likely to get any faster. We try to account for this, to some degree, with our height, weight and body mass distribution inputs in Step 2, but this is something that requires a certain amount of ‘artistry’.

The third area is Pitch Repertoire. Young pitchers, especially at the lower levels of the minors and in college and high school, tend to get by on one or two pitches. Rarely do they have to go to a Change-up. They may have more than that that they can throw, but they tend not to need them to get out less experienced hitters. The old baseball adage definitely holds true—a pitcher needs three quality pitches to make it as a big league starter. In this area we are looking for depth of a pitching repertoire, or else we are likely to evaluate the player as a RP only—and we will write more in depth at a later date about the success rate of minor league RPs. The other area that we look for here is velocity. The term ‘crafty lefty’ wasn’t invented out of thin air. There are a number of left-handed pitchers pitching in the Majors today that can spot four pitches, but lack any true ‘stuff’. You’ll notice they don’t say ‘crafty righty’. While right-handed pitchers can succeed at the lower levels of the Minor Leagues without ‘stuff’…there are very, very, few Major League right-handers with a mid 80’s fastball.

The fourth area is Pitch Movement. A major league hitter will have little difficulty in hitting a 100mph fastball if it comes in straight. When we are looking for pitchers that will be successful, we want pitchers that show late ‘life’ or late breaking action on their pitchers. This is one of the more difficult areas to evaluate, because it requires some sort of first hand observation. At Diamond Futures we use our own observation, industry contacts, and research. Despite all of these, we still likely have a good feel on about 10% of the pitchers that we would like.

The fifth area that we look at is Defensive Position. The overwhelming plurality of Major League position players came out of high school as shortstops. High school coaches play their top players at that position. As a player progresses, his position is determined by a heightened elimination process, i.e. positions get eliminated by their defensive capabilities, or lack thereof. Only SS and a Catcher can make the major leagues nearly exclusively on their defensive ability. Therefore, as they move outward, from the middle of the diamond, their offensive requirements become greater. A player that might have had enough of an offensive game to play SS, may lack the requirements to play 2B, 3B or CF. A player that might have had enough offensive game to play 2B, may lack the required power to succeed as a corner OF or a first basemen. So when we try to determine if a ‘prospect is a prospect’ we look at what position they are likely to play at the big league level, and then if their bat will meet the requirements of a typical Major League player at that position.

The final area that we look at, is also the most difficult to gauge—Work Ethic. Remember that Michael Jordan was cut from his high school team as a Sophomore because he wasn’t talented enough. At the highest levels of professional sports, talent isn’t as big a differentiator as is work ethic. While I am not writing off Delmon Young just yet, unless his work ethic changes his, almost limitless, talent will be wasted. When we are assessing this area, we are predominantly looking at how ‘coachable’ a player is, how a player responds when things aren’t going well, and how much of a student of the game the player is. We acquire this information through research, observation and first hand reports.

While Step 3 is more ‘art’ than ‘science’, keep in mind that we use this predominantly to weed players out. So while this isn’t really quantifiable or objective, given the methods available to collect this information, we are only using it to eliminate or downgrade players that the statistical work has already identified. So we aren’t dramatically skewing the statistical analysis with this step—only improving on it.