1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

This is why ESpN’s qbr ‘stat’ is garbage

Discussion in 'Miami Dolphins Forum' started by Pauly, Sep 9, 2018.

  1. Pauly

    Pauly Season Ticket Holder

    3,696
    3,743
    113
    Nov 29, 2007
    If we look at the stat kines of the 2 starting QBs.
    Tannehill
    20/28, 230 yards, 2TDs, 2Ints.
    1 sack
    Traditional efficiency stats.
    NFL passer rating: 89.9
    Y/A: 8.2
    Adjusted Y/A 6.4

    Marcus Mariota
    9/16, 103 yards, 0TDs, 2Ints
    0 sacks
    NFL passer rating: 36.2
    Y/A: 6.4
    Adjusted Y/A: 0.8

    So by traditional methods Tannehill had the more efficient afternoon.

    After watching the game both QBs had some swings and roundabouts, some missed interceptions and some some missed opportunities for more catches. It’s not like the rceivers were bailing Tannehill out by making freakish catches and Mariota’s receivers were bobbling perfectly thrown balls into interceptions. From my seat the non-QB element wasn’t going hugely in the favor of one player or another.
    There were no fumbles by either QB
    Rushing: Tannehill 4 for 4 yards, 0TDs - but 2 of those attempts were kneel down to end each half.
    Mariota: 3 for 15 yards 0TDs
    So nothing that moves the needle too much

    But according to ESPN
    Tannehill: 34.3
    Mariota: 37.4

    I totally get ESPN’s qbr giving Mariota a failing grade, traditional stats do too, but I can’t see how ESPN’s magic number box can give Tannehill a grade worse than Mariota and retain any credibility.
     
  2. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    Tannehill was more efficient and had two TD's. On the negative side, he was sacked once. So I'm not sure how a better completion percentage, two touchdowns and one sack makes him the lesser of the 2 QB's. You must lose like 30 cool points per sack or something.

    That's why I never look at QB stats until after the fact- I just don't see value in them other than year-long comparisons.
     
  3. Fin-O

    Fin-O Initiated Club Member

    11,375
    11,392
    113
    Sep 28, 2015
    Yea that's weird. Do they factor in 3rd down passing??
     
  4. Nappy Roots

    Nappy Roots Well-Known Member

    10,191
    4,187
    113
    Dec 3, 2007
    Bradenton,FL
    ESPN stats weighs heavy on where you turn the ball over and how well you do in the money downs(3rd down) and in the redzone. Tannehill threw a pick in the endzone. We didnt convert 3rd down or redzone too well.

    Not saying I agree with ESPNs stat, but it simply has some parts of it I think are valuable.
     
    KeyFin likes this.
  5. Unlucky 13

    Unlucky 13 Team Raheem Club Member

    51,930
    63,009
    113
    Apr 24, 2012
    Troy, Virginia
    ESPN's stat is in fact hot, smelly, rancid garbage, and always has been.
     
    danmarino and resnor like this.
  6. heylookatme

    heylookatme Well-Known Member

    902
    438
    63
    Sep 12, 2012
    ESPN's QBR is a joke. They refuse to disclose the full formula for the stat and they factor in completely absurd things like how each quarterback's defense or special teams impacted the game.

    Tannehill literally lost QBR after throwing the 75 yard bomb to Stills because the ensuing kickoff was returned for a touchdown.
     
  7. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    Maybe it's worth pointing out what parts of QBR are valuable and what parts are.. as some have already said.. "garbage".

    The part that's valuable is something called Expected Points (EP), which is defined as the average points scored, historically, for any NFL team in that same down, distance, game situation, etc.. Each play results in a change in EP, or EPA = Expected Points Added. EPA is how ESPN's QBR takes game context into account, and it's a totally sound approach statistically speaking.

    The part that's garbage is what they do after calculating EPA, which is using a proprietary algorithm for dividing up EPA (dividing up credit) among the individual players. There's no statistical approach right now that can tell you how much player X contributed to an outcome, so they're incorporating hidden and subjective assumptions into that part. And it's proprietary because they know they could get ripped for specific assumptions they're making.
     
    danmarino, Tin Indian and resnor like this.
  8. Pauly

    Pauly Season Ticket Holder

    3,696
    3,743
    113
    Nov 29, 2007
    Totally agree that Epected Points Added or Win Percentage Added, both of which are calculated from historical data would be a great way to look at establishing a new QB rating.

    You can do the raw numbers, or modify them to an efficiency stat (X added per snap multiplied by a factor to give a score out of 100). Of course if you do that you no longer have the 11 secret herbs and spices to call it a proprietry blend.
     
    danmarino and resnor like this.
  9. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    Well, in no world should a QB with 2 TD's in a win be ranked statistically worse than a QB with 0 TD's in a loss. That tells me that the formula doesn't take the most important factor into consideration- actual winning or scoring TD's. Because even if it determined that RT only had a 10% influence on both those scores, that's still 10% more than Mariota.
     
    danmarino and resnor like this.
  10. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    I guess while we're on the subject it's worth pointing out the main problem with EPA (or WPA). The basic idea is fine, except that the variances differ so much for different game conditions because of different sample size. For example, the variance for 1st and 10 in tiny because that occurs so often at so many different points on the field, but try 3rd and 12 or so and you get a mess with the data, meaning that there are so many different "valid" trend lines (or curves) you could fit that you really can't get a reliable estimate.

    It's a great idea, but it's really hard to build on something where the reliability of the stat changes from game situation to game situation.
     
    danmarino and mbsinmisc like this.
  11. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    EPA (and thus QBR) takes into account whether you scored a TD or not (EPA is far more positive if you score). But what if the formula now says that the consequence of not scoring was a massive change in EPA (possibly over several plays) and that the QB was primarily responsible for that? Well.. now you can get weird results. So which outcomes they associate with a QB and how much influence they assume the QB had on those outcomes make all the difference in the world (all proprietary). But they do take scoring into account.
     
    danmarino and KeyFin like this.
  12. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    That's why I'm saying it's obviously flawed. RT had a higher completion percentage and two TD's. In no world should that be beaten by a lower completion percentage and zero TD's. I can see them hitting RT with a huge penalty for the interception in the end zone since that took 7 points off the board...but that should never weigh more than actually winning the game. That deep bomb to Stills was freaking beautiful.

    If that's what happened, the formula is just broken.

    I'm heading to bed...I'll argue with you more tomorrow. =) Have a great night!
     
    danmarino likes this.
  13. Pauly

    Pauly Season Ticket Holder

    3,696
    3,743
    113
    Nov 29, 2007
    I can imagine some instances where the 2 TD winning qb could be less valuable than a 0 TD losing qb, but they’d have to be dependant on some extreme factors happening at a really unusual levels things like dropped catches, dropped interceptions, fumbles, qb rushing or sack rates. At least 98% of the time the 2TD winning QB (who btw also had better completion %. interception %, yards/attempt) played the better football that day than the 0TD losing qb.

    And if it did happen you should be able to point to exactly what happened to cause the losing QB to be rated higher than the winning QB. For example PFF do this with their game grades, whether you agree with them or not, they can explain why a player’s rating is unusually above or below what teh taditional stat line would have younexpect.

    There’s nothing I could see in the game that led me to think Mariota was having an equal to or better day than Tannehill.
     
    danmarino, KeyFin and resnor like this.
  14. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    Well.. that's the problem isn't it? No one knows precisely what they're doing.

    The way I view QBR is that as a statistic it's trash, precisely because it's not a statistic. Stats are logical inferences from data and adding in all kinds of subjective assumptions means you're not creating a statistic.

    But as a "model" I wouldn't diss QBR too much. It supposedly has a higher correlation to win% than passer rating (I say supposedly because I've never bothered to double check.. I abhor it too much to even put it in my database lol) so if all you're interested in is higher predictive power, regardless of assumptions, then it's fine I guess.

    And from that perspective, let's remember that what matters is the correlation, not individual cases. I mean, you'll find cases where a QB performs great and have a low (traditional) passer rating, or vice versa. So what matters is the correlation.
     
    danmarino and KeyFin like this.
  15. Unlucky 13

    Unlucky 13 Team Raheem Club Member

    51,930
    63,009
    113
    Apr 24, 2012
    Troy, Virginia
    Most importantly, no stat should ever, ever take into account wins and losses. If it does, or tries to predict that, then it's more than garbage, its offensive.

    In the past when they've shown some aspects of the formula, ESPN showed that RT took huge hits to his QBr because his ol was awful and he was sacked a lot. useless.
     
  16. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    I'd say precisely the opposite. If you can show near zero correlation between any stat and win percentage, then there's no reason whatsoever to care about that stat, because it doesn't affect wins and losses.

    So it's imperative for every football stat used to either directly correlate well with win percentage, or to correlate better with win percentage than a competing stat (if you have to choose between two).
     
  17. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    That's what I meant as well. It should never give "bonus points" for winning, but if the top 5 quarterbacks are all on losing teams then the stats aren't identifying the right things to be used as predictors. After all, the whole point of QBR is to identify the most successful quarterbacks.
     
    danmarino, mbsinmisc and cbrad like this.
  18. Unlucky 13

    Unlucky 13 Team Raheem Club Member

    51,930
    63,009
    113
    Apr 24, 2012
    Troy, Virginia
    Well, I will disagree with every fiber of my being until the day I die, lol. I strongly, strongly, believe that in a team sport, stats should show what the individual accomplished, and allow him to be easily compared to others at his position, regardless of how his team performed.

    Traditional qb rating is obviously flawed in some ways, and is often now seen as a team stat. But all attempts to replace it seem worse to me.

    In a perfect world, a stat would be able to show how a player did regardless of, or in spite of, his teammates. But that's easier said than done. And at the end of the day, most importantly, I never connect players and victories.
     
    danmarino, resnor and KeyFin like this.
  19. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    I think you have to step back a bit and ask yourself why you want to look at stats in the first place. Let's say you're talking to a an NFL coach and one person says to the coach that stat X suggests he should pick QB A instead of QB B in the draft, while another person says stat Y suggests he should pick QB B instead of QB A.

    How do you decide which stat is more relevant? Well..suppose stat X is completely uncorrelated with winning while stat Y is highly correlated with winning. Obviously the coach should use stat Y as the basis of his decision, because the goal is to win!

    This is exactly why moneyball had some success in baseball: some of their stats actually improved win probability. Point is.. if you're going to use stats to suggest how the team (or individual) make relevant improvements, you want to make sure the stat correlates well with win%.

    I can see your side of the argument ONLY if you don't want to use the stat to suggest ways to improve the team's chances of winning.
     
  20. Tin Indian

    Tin Indian Rockin' The Bottom End Club Member

    7,929
    4,404
    113
    Feb 10, 2010
    Palm Bay Florida
    This

    But according to ESPN
    Tannehill: 34.3
    Mariota: 37.4

    Is all you need to know that the stat is not logical and not weighted properly.
     
    danmarino and Unlucky 13 like this.
  21. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    Why? Just because a team loses doesn't mean the QB is playing poorly. Actually, I think that's exactly why ESPN came up with this stat. They want to give a rating of the play independent of the win and loss. At least that's how's it seems to me. And it makes sense, in that respect, for instance, looking at the discrepancy between Mariotta and Tannehill yesterday.
     
    danmarino and KeyFin like this.
  22. Unlucky 13

    Unlucky 13 Team Raheem Club Member

    51,930
    63,009
    113
    Apr 24, 2012
    Troy, Virginia
    I am simply a fan. Never a coach, a scout or an executive. I just want to see which players did well on a given day, or year, or career. And it interests me in the sense of history, it even novelty, when something is rare or special. And unlike some, I do still like counting stats, because they show that a player continued to be effective over a length of time. I like stats for fun.
     
    danmarino, mbsinmisc and cbrad like this.
  23. Fame

    Fame Well-Known Member

    1,043
    1,581
    113
    Mar 20, 2012
    Vero Beach
    News flash: ESPN is the TV version of clickbait. Their business model relies on creating controversy to drive viewership. As such, their content is pure garbage.
     
    Tin Indian, danmarino and mbsinmisc like this.
  24. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    For one game, I agree with you. But these are annual statistics that we add to each week....that helps minimize the terrible/awesome games to give a true average of what a QB's made of.

    For example, Rodgers QB rating last night was a 130.7, which in my opinion is WAAAY too low for the performance he put on while injured. In anyone else's book, he was better than the "perfect" 158.3. And GB almost lost that game- it wouldn't have changed his stat line one single bit. So it is important to have more than "just a number" and I appreciate ESPN trying to do that; I just don't see how RT with 2 TD's (one a 75 yard bomb) doesn't rank higher than Mariota who had two picks and no scores in 13 throws. There's something in that formula that's not correlating with what we saw on the field.

    Like I said-

    RT > Mariota in accuracy %
    RT > Mariota in TD passes (2 to 0)
    RT > Mariota in total completed passes
    RT > Mariota in pass/int ratio

    Mariota > RT in 3rd down conversions
    Mariota > RT in sacks (0 to 1)

    Unless I'm missing something critical here, t's very hard to make the case that Mariota outplayed Tannehill. Yet that's what ESPN is showing. I'm guessing a big part is ball placement and that should be a factor, but I have a feeling it's way too big of a factor given the outcome of the game.
     
    Pauly, danmarino and resnor like this.

Share This Page