Comments on Basement Dwellers: Player Value, Part 5b: Leverage and Relievers

Miles, I don't have a formula, but here's a few li...

2008-07-31T18:44:00.000-04:00

Miles, I don't have a formula, but here's a few links that may be helpful regarding win probability:

Dave Studeman's article

An empirically-derived Win Probability tool

Also, if I remember correctly, there are complete win probability tables in Tango, Litchman, and Dolphin's The Book (which is also highly recommended).
-j

I'm in over my head here, but can anyone point me ...

2008-07-31T17:19:00.000-04:00

I'm in over my head here, but can anyone point me to a mathematical model that predicts win/lose outcomes based on the score differential in each inning?

To what extent does 1/0 in the first predict the outcome? 4/2 in the eighth? etc. I'm interested in this without regard to team or any other factors... just based on the score.

Someone must have discussed it somewhere! thanks

Perhaps a better way would be:

LI=(tmERA/ER...

2007-12-07T14:19:00.000-05:00

Perhaps a better way would be:

LI=(tmERA/ERA)^x

where tmERA is the ERA of the team's relievers, and x is some number like 1.4 or 1.5.

Play around with it, and you'll probably get something decent.

Hi Tango, thanks for that. Neat set of ideas, and...

2007-12-07T12:54:00.000-05:00

Hi Tango, thanks for that. Neat set of ideas, and it does make sense. I may give that a go at some point here, though Sky, you might get to it before me (which is fine!).
-j

To help in your regression, you can try to convert...

2007-12-07T10:10:00.000-05:00

To help in your regression, you can try to convert LI into a rate.

rateLI = LI/(LI+1)

An average LI will have a rateLI of .500. League leaders have an LI of around 2, so they get a rateLI of .667. League trailers have an LI of around 0.5, so they get a rateLI of .333.

If the team ERA for relievers is 4.50, with a range from 3.0 to 6.0, you can probably due:
rateLI = (tmERA - ERA)/10 + .5

So, an ERA of 3.00 ends up with an expected rateLI of .65. And rateLI of .65 implies .65/.35=LI=1.86.

Seems like a reasonable approach...

Thanks for your comments folks. I'm confident tha...

2007-12-07T01:37:00.000-05:00

Thanks for your comments folks. I'm confident that this is a perfectly acceptable way to evaluate historical value. We could probably do the same thing to every player to get their numbers closer to WPA if we wanted, but pLI's are likely to be repeatable only for relievers, so those are the only ones I'm interested in evaluating. The one exception might be an almost exclusive pinch-hit specialist like Lenny Harris back in the day. Would be interesting to look at that some time.

With respect to Burton, he started the 8th inning a lot of times last year, didn't he? Seemed like they prefered to use him as an 8th inning closer rather than someone who came in to shut down opponents. I could be wrong about that, however. Even so, the adjustment really didn't affect his value much at all because his pLI was so close to 1. Given that he was really only a factor in the second half of the season, the above rankings make sense to me with respect to where he fits in vs. other players.

@Chris, if you're interested in seeing Baker's usage of relievers, just pop over to fangraphs and look up the leverage for his Chicago Teams. They've got data going back to '02. FWIW, my initial take on those data is that Baker tends to have one or two other pitchers in his reliever corps that have pLI's well over 1.0. That means to me that he's using lesser relievers in some high leverage spots...though to be fair, sometimes the difference between his setup guy and his closer was not terribly big (e.g. Dempster vs. Howry).

@Sky, don't forget about the nature of the variable we're trying to predict. I really do think that the fact that pLI is a ratio makes it really difficult to use in a simple linear or even a polynomial regression. It might be worth trying to calculate a separate regression above and below 1.0! Or, perhaps we should change the way we define leverage...more trouble than I want to deal with for now! :)
-j

Chris, can we call that ppLI?

Sure, that's ...

2007-12-06T16:53:00.000-05:00

Chris, can we call that ppLI?

Sure, that's one use of a predictive LI. Given a team and a manager, how will each pitcher be used, and therefore how valuable will he be.

My interest takes another step from "value" towards "talent". Given a 3.50 ERA pitcher, how valuable is that? For starters, you simply do (repERA - ERA) * IP/9. But the bullpen is one place where runs are definitely not created equal. I'm interested in knowing how valuable the typical 3.50 ERA reliever is. Given all the other relievers out there, how will he typically be used? Or, how should he be used?

It might be worth it to sign one 2.50 ERA reliever to a $10 million contract, because his LI can be over 2.0. But you wouldn't sign a second at that price because you won't have 150 innings at a 2.0 LI to split between two guys. The next guy might have a 1.35 LI. Third third guy in the bullpen might be at 1.0, then .9, .75, and .6.

Therefore, 2.50 ERA guys deserve 2.0 LIs. But a 3.50 ERA guy won't get paid to close, making him less valuable beyond his ERA disadvantage. The value equation becomes (repERA - ERA) * IP/9 * ppLI. ppLI is 1 for starters and is a function of ERA for relievers. Except it's difficult to find a function that maps ERA onto leverage thanks to crazy managerial usage patterns, injuries, flukey ERAs, changing skills, etc.

I'm in way over my head, but isn't what you...

2007-12-06T15:35:00.000-05:00

I'm in way over my head, but isn't what you're really looking at in "predictive pLI" the manager's usage pattern? Or the manager's usage pattern compared to his own pattern the prior year? (Or more accurately, whoever managed that reliever the prior year?)

Again, I may be embarassing myself here, but what I'm most interested in is seeing how Dusty Baker has historically used his bullpens -- best pitcher in highest-leverage situations, or designated closer in the 9th, actual leverage be damned.

Sky, I realize that, and I didn't mean to imply th...

2007-12-06T13:06:00.000-05:00

Sky, I realize that, and I didn't mean to imply that all of those situations were created equal. It's just that it seemed like Burton pitched in a lot of tight situations last year, so it surprised me that he didn't have a higher pLI. I'm not saying the system is wrong, just noting how it didn't align with my perception. I'm just not used to my perception being wrong. :)

Joel, you'd be surprised how many situations after...

2007-12-06T11:18:00.000-05:00

Joel, you'd be surprised how many situations after the 7th inning and within three runs actually aren't that high leverage. I'm going off the top of my head here, but a ninth-inning save with a three run lead isn't much more important than an average situation.

In talking with Justin, I agree that you definitely want to use actual LI to judge a player's value in a prior season. But going forward or to judge a reliever's value in a context-neutral review, I think you want to come up with a model that assigns LIs to pitchers based on their talent. For example, take a reliever who has a 3.50 ERA talent. His manager might use him as a closer (2.0 LI) or as a mop-up guy (.5 LI). His talent deserves, however, to be used as a set-up guy, with maybe a 1.25 or a 1.50 LI. So you should value him with THAT LI.

Because not all situations are created equal, not all runs are worth the same. And when you have relievers with varying talents, they should be used in different ways and have their skills rewarded in different ways. There's a reason relievers are important even though they only throw 80 innings. That needs to be taken into account.

Interesting. I was surprised to see that leverage...

2007-12-06T09:20:00.000-05:00

Interesting. I was surprised to see that leverage took Burton's numbers down. After the all star break, 24 of his 36 appearances were in the 7th inning or later in games that were within 3 runs. Granted, he didn't pitch in any high leverage situations before the break, but that was only 11 games. His numbers after the break were comparable to David Weathers, who pitched in 22 of 33 games within 3 runs. Of course, Weathers almost always pitched later innings, so I would expect his pLI to be higher, but I didn't expect Burton's to hurt his value.

Overall I think this is an interesting addition to the measurement. I'm not someone who can validate the technique, but conceptually, I think it's the right direction. I like the fact that it pushes Weathers up above Belisle and Lohse, at least on the Base Runs side. That seems more appropriate (though I can see the argument that starters have more inherent value based on the number of innings they pitch).