2008 Prospect Rankings

[BigRapidsJackass]

Aspire, we've seen -- at least in the case of the Rockies -- that what I labeled the HF ProspectBot does a remarkably good job of generating prospect ratings that are nearly identical to what the so-called experts come up with.

So there's really only two rational responses to this phenomenon:

1. All prospect ratings are pretty much crapshoots. They may be fun to look at, but ultimately they do not provide terribly useful information.

2. A totally stats-based system can, in fact, be just as useful as a system based on some unspecified mix of stats and scouting (refuting KG's statement); after all, the ProspectBot can pretty much mirror the findings of the other "systems."

I happen to think that response (2) appears to be pretty obviously correct, and that there's some truth to response (1) as well.

As for the Weathers thing: it tells us two things. First, that the ProspectBot can't account for college careers, at least not yet. So the other ratings (BA/Sickels/KG) are probably based on simply finding a reasonable slot in which to add a very high draft pick. Second -- as HF says -- it tells us that the standard prospect rankings really don't look toward any kind of projection of cumulative MLB WARP or any such thing. HF's point isn't that Weathers is a poorer "closer" prospect than Fowler is a "CF" prospect; rather, it's that a guy who can only be expected to throw 70 innings or so per year is almost never as valuable as a guy who gets 600+ PAs and 150+ games in the field. Goldstein must have a reason for ranking Weathers higher, and I suspect it is his overall "gut" feeling that a good closer is really important and that Weathers has a better chance of being a stud closer than Fowler has of being a stud CFer. Maybe that's true, but he should at least recognize the relative value of a FT position player vis-a-vis a short relief man. And he doesn't.

[hiaspire]

Originally Posted by BigRapidsJackass

Aspire, we've seen -- at least in the case of the Rockies -- that what I labeled the HF ProspectBot does a remarkably good job of generating prospect ratings that are nearly identical to what the so-called experts come up with.

So there's really only two rational responses to this phenomenon:

1. All prospect ratings are pretty much crapshoots. They may be fun to look at, but ultimately they do not provide terribly useful information.

Why do you think they are so close with many players? Do you think scouts or Baseball America ignores performance?? Of course not. The scouts are going to like the guys who put up the big numbers in the game too, right?

However, numbers often tell lies or very incomplete pictures. We can't pretend that all numbers mean the same things or that there are no associated factors that go along with the numbers. When you ignore that bit of common sense and just look at numbers exclusively that's where you can make massive errors with very incomplete information.

The best look at both. Scouts don't ignore a player's statistics, or in-game performance beyond the box score. They get the story behind the numbers, though, too.

I disagree that it's all a crap shoot or you don't get useful information. Sure, there are many players who don't live up to their hype, but it would be a fool who doesn't find such information about a player's talent or technique useful. You want to know what kind of bat speed a player has. What kind of pitches a young arm throws. There is a ton of useful information, but it doesn't always mean people with the most talent always succeed. Not in baseball, and not in life either in many respects. It's still much better to make educated opinions on players, though, with as much comprehensive information as possible and even (gasp!) beyond the numbers.

The truth I think is that players are grouped much more closely together than rankings allow. You'll have some that barely belong and are no-hoper's, as well as some players with elite talent that are beyond most everybody on the field. Most of the rest, though, deserve to be there and have as good a shot as anybody else in that group. Especially when most of the ones who "make it" become useful role players who might not have superstar ability but provide a needed service on a big league roster. For all the majority of players with enough talent to make it, it is the little things that determine who does and that involves constantly changing variables and opportunities that may or may not present themselves at the right time.

If you are ranked high enough, though, those opportunities present themselves to you much more frequently. Teams will go out of their way to give those highly rated guys a chance where they won't for others not measured as strongly. So rankings do matter (the team's own rankings more so internally than some outside service to them), and there is useful information in them about the player's abilities and how it might effect the opportunities given if more people think highly of them.

[Heltonfan]

First, that the ProspectBot can't account for college careers, at least not yet.

It can if I want it to. I've chosen not to use college stats, because they're so unreliable (at least they appear to be), but from the research I've done, college ball appears to be roughly equivalent to Rookie league (Pioneer League, Appalachian League) for pitchers and the complex leagues (Arizona League, Gulf Coast League) for hitters (it makes sense that there would be a difference there, because hitters have to adjust to wood bats).

If I put Weathers' college career into the mix, he ends up at #31 on the Rockies' list. His college numbers are excellent, but they're not Wagner/Cordero/Street good (which in itself says something, since none of those guys were picked as high as Weathers was).

One other point here, which I thought was blatantly obvious but seems not to be: if the ProspectBot is just as good as the BA/Goldstein/Sickels lists, that does not mean that scouting is worthless. It simply means that the ProspectBot does a better job of reading the numbers than the other guys do, thereby making up for its lack of scouting input. The claim has never been that the ProspectBot is perfect, just that it's as good as anything else.

[hiaspire]

More complicated statistical evaluations might find more useful details in the numbers alone. I've maintained that you need that side of things too to make a more comprehensive conclusion. Only looking at one side of things is never good, though, unless you purposely want to get wrong answers in cases that fly right into the obvious holes in each perspective (and they all have their advantages and disadvantages). I don't know why anyone would want that.

If you ask a scout to name the top players in baseball overall, you'll probably get a very similar (if not IDENTICAL) list to what a spreadsheet would spit out. Performance is valued by all. There are probably very few differences in terms of the top performers by either list, so it isn't all that unusual that such lists at the top would be similar for anything. When a prospect puts up great numbers, he rises in everybody's rankings. Someone putting up huge stats in the minors is either ranked highly, or has some reason for not being more recognized. That reason would be ignored by the numbers but not by the humans putting those numbers into context. Sometimes that reason is very important to consider.

The real difference between the two is that with the numbers list you'll probably have some names valued much more by numbers that don't seem to fit otherwise (Ross Gload, Luke Scott, or whoever else discussed over the years as such). Sometimes those become found gems where all the numeric digging provided a different perspective that shouts out more attention toward a player, and other times they are just bit players who do great in a limited role by the numbers but get exposed later when those limited numbers really didn't apply when multiplied into more chances as math had expected.

[BigRapidsJackass]

Congratulations, HiAspire!

Ross Gload didn't appear in your comments on this topic until your 8th -- count it, 8th! -- post.

HF, the point isn't that scouting is irrelevant in prospect rankings. I think the more interesting point is this: the so-called experts are really doing exactly the same kind of stats analysis as your ProspectBot is doing. These are guys who stare at minor league stat lines all day every day, and even if they haven't created a formula such as the ProspectBot, that's exactly what they're doing in their minds.

There's an old cartoon that goes something like this:

A kingfisher is perched on a branch overlooking a lake. He sees a tasty fish swim by a few inches under the surface of the water. He is ready to pounce. The thought bubble coming from the bird's head shows a diagram and an equation accounting for the refraction. (O.K., the visual is better.)

Sickels, I imagine, has developed an equation much like the ProspectBot, and he fiddles with his results a little bit before producing his list (e.g., knocking down a Greg Reynolds a few notches based on arm trouble). The others, I think, are like the kingfisher -- doing the complex math in their heads without really being aware of the mental gymnastics they are performing.

[wolf213]

This brings about a very good question... just in general, who's list to you rely upon the most? Sickels? BP? BA? All are similar, I have a BA subscription, which I think has top notch scouting reports, especially when the draft is around, but I have found Sickels to be the best for me...

[Heltonfan]

The kingfisher analogy is perfect. Well, almost perfect; give these guys another few million years of evolution, and their internal statistical analysis might reach the same level as the ProspectBot's, but they're not there yet. If they were, their lists would be clearly superior.

Wolf, I personally prefer Goldstein's lists. Sickels relies too much on stats (by which I mean that 90% of what he does is stuff that I can do better) and intuition (which might have meaning if he's seen the player in action, but when he says he has a good intuitive feeling about some random high school draftee, I don't see any reason to care). And the BA lists go too far in the direction of hyping 17-year-olds. Goldstein's lists, for the most part, are a happy medium.

Of course, there's another interesting little project here: figure out whose lists correlate the best with the ProspectBot's (I assume it will be Sickels, but you never know) and by how much.

[hiaspire]

We all bow to your superiority, including those in the game. If only they had half the brain as some here they might be able to evaluate talent for a living someday.

The lists that include at least some subjective common sense associated with situations beyond pure numbers alone are already clearly superior as they allow for real world injuries and other critical considerations. It would be pretty unreasonable to ignore some of those things entirely.

It's great that there are so many different perspectives out there, though, and many opportunities to see a different side of a player and his possible future. Most of them have at least some balance between pure performance and other developmental issues.

Not all teams are the same either and certain players will fit into their needs and strategies more than others. Many teams lean in one direction more than another for either stats-based analysis or scouted talent, but in the end most of them also try to find some mix in their pipeline and have selected both types of players from the different perspectives. In the end, a team is probably better off not mirroring one list but collecting some of the strongest recommendations off various lists that see things a little differently for different types of players.

[RMF]

I fail to see what most of that has to do with the Prospect bot. The simplicity of it works because of all the variables you so eloquently described. This is the model that doesn't overthink it.

I'm not sure who or what you are debating with. It seems like you are always trying to define the unknown. It seems like you credit favorable and newsworthy outcomes to some higher human intelligence or power.

In the end it was just randomness. A large amount of success in business is total luck. I believe the Harvard Business school proved that. A heck of a lot of life is just pure luck. If you want to call it destiny or god's will or whatever so be it.

But untill you can prove those other elements exist and you can account for them and define them. Those opinions should probably stay out of a rational argument. I think you know all of this and more.

So I'm not really sure why you post these things. We all know none of these lists will predict the future. However, they are excellent tools. And if every so often one of your special guys makes it... Congratulations, the rest of us have those guys too.

Nothing predicts outcomes that will meet your standards.

I would take a system over my own opinions any day.

[hiaspire]

You say "you" a lot there as if talking to someone directly. Are you addressing me?

Stats are often worshiped here as if there is nothing else that matters nearly so much and no other perspectives really welcomed. Many arrogantly call others in the game morons if they don't follow that same reverence for the awe-inspiring power of the numeric conclusions and their undeniable answers. That contempt/disrespect does get under my skin sometimes (as I'm sure I do to others), and I'll certainly argue the other side of issues where I think the statistical evidence is entirely overblown in comparison to ignored obvious risk factors that point the another way that also need to be considered for a reasonable conclusion IMO. Different perspectives often lead to different conclusions argued back and fourth on message boards. That's a good thing I think, although I'm probably in the minority here on that amongst the craving for a spreadsheet full of all the answers we'd ever need.

I like your stated approach to it better there, though, where you also seem to cast some doubt on the importance of such conclusions yourself by calling everything "random". I asked if you were talking about me in confusion because in some ways you are sounding a similar message as myself in between the opening and concluding disagreements otherwise.

Now I won't go nearly as far as what you state in labeling everything as such random chance, but I would think if someone really believes that luck plays that important of a role in everything then logically you would think they also wouldn't put as much value into meaning of calculated predictions otherwise if pure luck is so significant. What you refer to as unpredictable luck there I'd probably point to other real human elements that might have contributed at least partially to those statistically unpredicted outcomes in some cases, which to me provide better justifications than just writing it off as just dumb luck with no further examination. Certainly luck exists, but so do other things we maybe can't add on a calculator but can consider logically anyway.

You mention some things should stay out of a rational arguement. Let's take the most important thing I've mentioned in that regard here. Should INJURIES?? Would it be rational to ignore those in only looking at numbers exclusively? I would call that irrational. Most of what I am saying, in response to a question about numbers alone predicting the value for prospects, is that if you don't at least consider a player's health during the time those numbers were put up then a statistical conclusion is likely to be very flawed if missing such a critical consideration.

Do you disagree? If you have a stud prospect who has a not-so-serious but nagging injury he plays through, should that be entirely ignored in valuing him as a prospect by the lesser numbers alone and putting him equally alongside lesser talents who had healthy seasons of similar stats? If another player is just gettting back from surgery and slowly working his way back into the grove, should those numbers be treated with the same meaning with no regard to the situation for how that impacts his calculated overall value?

Now you certainly can't always put a number on such human considerations, but to make the most logical conclusions overall I would think that shouldn't be utterly ignored either. The best lists, as mentioned by others, probably start with the numberic analysis and then have some adjustments in there for those human situations and common sense. To take that entirely out only creates huge mistakes in those specific cases by design IMO if you are going to extremely under-value a very valuable prospect otherwise for a momentary situation.

If the questions is can you rank prospects with numbers alone and do just as good as anybody else, then the answer IMO is a clear no unless you ALSO consider the context surrounding those numbers in such cases. But the other lists being compared to make those kinds of human adjustments for very logical reasons while also paying attention to performance.

And I certainly don't disagree that there are excellent mathematical tools if that is the charge. I have always said that the best conclusions need to have BOTH perspectives included to see the complete picture of a player. I stress inclusion, not isolation to only one method providing everything needed as pondered here.

[Heltonfan]

More prospect lists. Here's Goldstein's top 100. And here's Keith Law's.

Law and Goldstein's lists have 74 players in common. I made a hybrid of their lists, and that hybrid top 100 has 59 players in common with the ProspectBot. Many of the guys about whom the Law/Goldstein list and the ProspectBot disagree, of course, are small-sample/no-sample cases. I'll have to put together a list of the players for whom the ProspectBot has meaningful data to work with who are causing the most disagreement.

[John Cocktoston]

Originally Posted by Heltonfan

Law and Goldstein's lists have 74 players in common. I made a hybrid of their lists, and that hybrid top 100 has 59 players in common with the ProspectBot. Many of the guys about whom the Law/Goldstein list and the ProspectBot disagree, of course, are small-sample/no-sample cases.

Out of curiosity, where do the players on whom they disagree tend to fall? I'm assuming that they would be towards the bottom half of each list, but maybe that's not a safe assumption. Who is the most highly regarded player on one system who doesn't make the cut on the other?

I tend to look at prospect evaluation and amateur scouting as a "best practices" exercise in that it is an example of a process which you can never fully control, but you can and should manage it as best you can.

I think I've used this analogy before, but it's a little like sitting down at a blackjack table. If you know the book inside and out, you will maximize your chances of success by (1) not doing stupid things like splitting 10's and (2) taking calculated risks, such as doubling down on a 10 when the dealer is showing a 5.

If you don't know the book, you're just playing out the hand and hoping to get lucky. Sometimes you will, but it doesn't make you smart...it makes you lucky.

Conversely, you could sit at a table for two hours and not make a single strategic mistake and still lose your ass...

Now, somewhere along the way, you could probably have hit on an 18 and picked up a 3 to get you to 21. But no system worth a damn would have ever told you to do that.

Performance analysis is a tool. Scouting is a tool. So is minor league instruction, strength and conditioning and psychological/performance coaching. At the end of the day, a system is only as good as its results. And if your system doesn't know (or for that matter care) that Jeff Allison has an effortless 96 mph fastball with arm-side tail, clean mechanics and some serious family/character issues then your system ends up falling in love with a kid who graduated from oxycontin to cocaine to heroin as a high school senior...

[Heltonfan]

I'm assuming that they would be towards the bottom half of each list.

Correct. But there are still quite a few outliers...

Who is the most highly regarded player on one system who doesn't make the cut on the other?

None other than Kevin Mulvey, #14 on the ProspectBot list (I said 15 in the other thread, but I discovered this morning that Justin Upton isn't prospect-eligible anymore, so that bumps him up a spot). Jair Jurrjens is the runner-up here, at #15 on the ProspectBot list. Going in the other direction, Angel Villalona is #17 on the Goldstein/Law list and absent from the ProspectBot list.

Actually, there's a real pattern here. The four most extreme ProspectBot favorites are Mulvey, Jurrjens, Aaron Laffey, and Will Inman, all of whom are pitchers with mediocre stuff and excellent MLEs at very young ages.

The four most extreme Goldstein/Law favorites (small-sample guys excluded) are Villalona, Manny Parra, Eric Hurley, and Carlos Carrasco. Villalona is an easy one to explain, a really young kid with great tools who is so far away that the ProspectBot just refuses to get very excited about him. Parra has everything going for him except for age; if he were two years younger, he'd be in the PB top 15. Hurley and Carrasco are typical good-stuff, poor-results pitching prospects.

[tdubwpb23]

why are you guys hating on my dog Mosebach so much? Just because you dont know about him doesnt mean he isnt a good prospect.

Mark my words, I guarantee you will here this chant in a big league stadium one day:

"MOWS-EM-DOWWN, MOWS-EM-DOWWN, MOWS-EM-DOWWN"

He's an inning eating ground ball pitcher that will give you a chance to win everytime he goes out to the bump. He's only 23 and throws 95+. Do your homework, or you could just read someone else's opinion and regurgitate it.

[indianadrew]

Originally Posted by tdubwpb23

why are you guys hating on my dog Mosebach so much? Just because you dont know about him doesnt mean he isnt a good prospect.

Mark my words, I guarantee you will here this chant in a big league stadium one day:

"MOWS-EM-DOWWN, MOWS-EM-DOWWN, MOW.S-EM-DOWWN"

He's an inning eating ground ball pitcher that will give you a chance to win everytime he goes out to the bump. He's only 23 and throws 95+. Do your homework, or you could just read someone else's opinion and regurgitate it.

.
BaseballAmerica.com: 2007 Player Statistics: Robert Mosebach

ya think so there tdub i did not see anybody "hating" on your dog. who are you anyway, AA entourage? BTW dude theres no reason to be so snarky.