1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Thoughts on the Dolphins/Cowboys Game

Discussion in 'Miami Dolphins Forum' started by KeyFin, Sep 22, 2019.

  1. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    I added my response to your edit.

    Regardless, your response here shows you don't understand that correlating to win% is necessary to determine how accurate the model you choose is. Whatever that model says is the proper way to measure QB's is what you choose. There's literally no better way to approach this type of problem. It's what sabermetrics, expected points, etc.. all are based on.

    The key here is accuracy of the model.
     
    Irishman likes this.
  2. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    As I said, if it's questionable why the receiver dropped the ball, then ignore that play for the advanced metrics. We're only looking for clear mistakes (like the bomb Grant dropped two weeks ago). But I'd also want to use that same premise against the QB as well...if he throws a pick that's dropped, then count it as a pick in the final calculations (again, unless it wasn't an easy catch...we only want easy to identify player errors to factor in).
     
  3. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    I understand your point and I almost said the same thing as Resor- I'm not looking for win percentage since I only want to accurately rate a QB. And that's where we have problems with QB rating and QBR...they don't even consider what I'd like to consider for a fair evaluation. Because think about it, we'd want to use this on college QB's as well to see if Tua really is that good or if it's a lie...this type of formula would push us in that direction anyway.

    PS- I forgot pressures...and that opens a whole new category. If the QB is under heavy pressure and throws a pick/TD or throws it away, we'd want to do something different there as well. One's a clear blunder, one is going above and beyond while the 3rd should not even count as a pass attempt.

    Anyway, I'm not sure if we're basing this one a win% model or not. My original thought was just to modify stats and then use standard QB rating with the adjustment....and that's already based on win%. So you're probably right, but we'd have to talk it out.

    By the way- we can end this conversation at anytime but there's basically nothing else to talk about anyway. So I thought it would be fun to argue over and if we all agree on a path to actually bring these stats to life, I could grab a solid coder in India to make us an app and see how far we can get. We're talking $2 an hour on average so it wouldn't be an expensive project; but we'd have to figure out where to get our base data from (stripping it from video footage, paying someone to sift thru all 16 games each week, or whatever). For that kind of data entry I could get us a virtual secretary for about $1 an hour and they'd be happy for the work.
     
    resnor and cbrad like this.
  4. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    Think about how science tests theories. Basically it comes down to predictive power. If you can't directly test the accuracy of the assumptions in a model, what you do is test how accurately the model predicts an outcome (or set of outcomes). The most obvious "outcome" in sports is win/loss. So the most obvious way to determine how accurate a model is is to see how accurately it predicts win%.

    In other words it's not about win% per se. That's where the confusion is arising. It's about determining which model is most accurate.

    Of course you could test the model by predicting something else, like run/pass distribution across different drives. That's fine too, but you'll run into the obvious criticism that you're ignoring other aspects of the game. So if you want a model that is accurate given all possible influences in the game, then you test it against win%.

    Up to you what you want to do and how far you want to take it, but if it were up to me I'd do several things:
    1) Come up with stats that are NOT out there yet but arguably important to consider (this differentiates you from others)
    2) Create a public database with those stats on a website so that others start using them
    3) Develop models using such stats with transparent and solid statistical analysis (i.e., NOT that crap like DVOA or ESPN's QBR)
    4) Slowly influence the type of statistical modeling used in the NFL as a result

    That's taking this thing entirely too seriously, but it would actually be important longer term instead of just being some pet project. It's also something that should be built up over a longer period of time instead of all at once. But again.. that's taking things seriously lol.
     
    Irishman, The Guy and KeyFin like this.
  5. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    Yeah, I completely agree...this would be a project to do in small steps instead of leaps- we'd want V 1.00 to gather info to tell us something we don't already know or have easy access to using traditional stats.

    Maybe it is manually getting closer to a true QB rating evaluation that tells us something we don't already know- would it make sense to start with the "true accuracy" stat and build out from there? That's not even strictly necessary for the larger overall formula but it's one stat we'd always like to see...how many caught passes were actually well placed? Or how many drops should have been caught?

    I could build a site with a web app (from India) where fans can plug in numbers to start...and I think it would be fun anyway just to get something like that off the ground. Basically, we wouldn't count any play until at least x number of people independently came to the same conclusion, a lot like the Galaxy Zoo site I mentioned earlier. Then we could post on some message boards around the US and get our hobbists to help us out with their team and who they played that week.

    I'd need everyone's help with telling the coder exactly what we want in the web app, but I could probably have it live with 10 hours of my time and maybe a few hundred dollars. If folks are interested, I'd gladly get the ball rolling...but I'd want this to be OUR project (all our core members still lingering here), not just mine. I just think it would be fun if we can figure out what we want to solve and how to do it efficiently with crowd sourcing.

    I'll be out of town the next 2 days for a family thing- think about it, argue with Resnor some more and I'll hop back in this conversation late Thursday. I'm all for it though if others want it to play with.
     
  6. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    Sounds good. I think the most important thing is to do this slowly and step by step.

    First step is to decide on what stat you'd like to have that we don't currently have. I'd forget about requiring that it HAS to have something to do with the QB. Let the modeling take care of how different stats are related to each other afterwards.

    The most important thing is that you can operationally define the stat. That is, you can explain to multiple people how they should grade a clip and it's VERY likely they'll all agree (just like refs in principle have to). I'll stay out of this first part because I'm neutral on it, but any final decision should be preceded by a test to see if WE can grade test clips the same way given some operational definition lol.

    Good idea though Key. We'll see how much staying power this has (often things fizzle out but there's no harm in trying!).
     
    Irishman, Pauly and KeyFin like this.
  7. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    Well, I don't agree that that seeing how it correlates to win% is necessary. By correlating to win you're already penalizing the losing QB, as usually they have a lower rating, as we know that higher rating correlates to more wins. However, that doesn't actually tell us which QB played better, just tells us who had better results. We're arguing that there has to be a better way of evaluating what the QB did, not determine if what he did was right purely by the result of the play.
     
  8. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    No, it's telling us which QB was more successful, not necessarily who played better. The current model only rewards good play when the end result is prove l positive. If the QB makes the right read, and makes a good throw, but the receiver drops it, it's a negative on the QB...but it shouldn't be.
     
    freeperjim likes this.
  9. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    But career passer ratings correlate strongly with consensus perceptions of quarterbacks’ individual ability, so presumably that measurement error becomes equivalent enough across quarterbacks over the long haul.

    The smaller the sample size (like the single game Rosen played the other day), the more one has to consider that measurement error, but over the long haul it’s relatively meaningless.
     
  10. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    Also, it’s probably going to be difficult to get people to agree sufficiently about quarterbacks’ performances in small samples of play. Right now in the Club section, Travis and CK for example are disagreeing pretty strongly about the particulars of Rosen’s performance the other day.

    From our vantage point, we may be relegated to the position of awaiting a sufficient sample size of play to make valid conclusions about quarterbacks’ overall abilities. With Tannehill for example this took five to six years.
     
  11. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    So imagine you keep improving "passer rating" to the point where it more accurately apportions credit among players (precisely what DVOA and ESPN's QBR try to do but can only do with subjective assumptions). That's all that would occur here, except that it wouldn't be just a QB rating system but a model of the entire game with an accurate representation of how much credit the QB should get on any play.

    And if that apportioning of credit is accurate, then all of your concerns are nullified. You'd never be saying "well that shouldn't be a negative on the QB" unless of course YOU have the wrong model in your head.

    And yes, correlating to win% is necessary if you want to determine the accuracy of the model and not be subject to the critique that you're only looking at a part of the game. That's how you test accuracy of models in science: see how well they predict observable outcomes.

    btw.. the difference between "success" and "played better" only goes so far because the outcome of the play is positively correlated with the QB doing what he's supposed to.
     
    The Guy likes this.
  12. The_Dark_Knight

    The_Dark_Knight Defender of the Truth

    11,817
    10,321
    113
    Nov 24, 2007
    Rockledge, FL
    No one ever said nor hinted that Rosen would play like Brady behind a better line. All anyone like me has said is without a decent offensive line, you can’t accurately gauge whether or not Rosen can play at the level we need him to play.
     
    resnor and The Guy like this.
  13. KeyFin

    KeyFin Well-Known Member

    10,488
    12,821
    113
    Nov 1, 2009
    I have to get some work finished before I head for the airport in a few hours, but I thought this through last night and I think it makes sense to start with pass accuracy. I won't get into the whole formula now, but basically we'd ask users to identify if a pass was perfectly on target, generally on target, off-target or a throwaway. Then a follow-up would ask if the pass was safe, contested or something else (missed route, bad overthrow, etc. can all be lumped together in this category). Finally, we'd ask if the pass caught, tipped, dropped or intercepted. That gives us a start on QB's, CB's, receivers, LB's and safeties. Of course, we'd verify all the players involved as well and there would be a tutorial on what each term meant so we're crystal clear.

    This should be an easy one to start with and like I said earlier, we wouldn't count the data as "verified" until several users reach the same conclusion on each of the three questions.

    I definitely want to hear other suggestions though- that's just the best I could come up with overnight. Tell me what I overlooked or what should be added..or if this idea just sucks! Remember that we want to keep this short and simple with just a few quick, easy questions and I REALLY WANT everyone's feedback.
     
    Last edited: Sep 27, 2019
    rackhound and cbrad like this.
  14. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    That's a good start. Allows you to calculate the probability of an outcome (caught, tipped, dropped or INT) given how on-target the pass was and how contested it was. Those probabilities can form the basis for evaluating how much above average or below average any QB or WR or pass defense is in different situations. In other words, you'd have z-scores for each for JUST the act of throwing/catching a ball.

    Only thing I'll mention is that I think next-gen stats already does this. But that data isn't publicly available.
     
    KeyFin likes this.
  15. djphinfan

    djphinfan Season Ticket Holder Club Member

    111,652
    67,546
    113
    Dec 20, 2007
    He dropped a ball that was on target
     
  16. djphinfan

    djphinfan Season Ticket Holder Club Member

    111,652
    67,546
    113
    Dec 20, 2007
    Tannehill was below average by every measure, we can and should of done better, many times
     
  17. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    Yes, but that's still an indication of talent around. Rosen, for instance, could have done everything right the past two games, but the results don't reflect that. Why should the lack of results reflect poorly on the QB, unless he's the one making bad decisions and throwing bad passes?
     
  18. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    Yes and no. All too often the Tannehill debates turned into looking at results...win%, overall record, no playoffs, etc. Where the conversation should have been was on whether or not Tannehill was making good decisions, and throwing catchable balls. If a QB is making good decisions, and throwing catchable balls, then he shouldn't be penalized, regardless of the outcome.
     
    KeyFin likes this.
  19. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    What’s difficult to determine from our vantage point is when a quarterback makes a bad decision and it involves a play that doesn’t happen.

    For example, a quarterback could throw a perfect five-yard out to a receiver, while missing the opportunity on the same play to throw a 50-yard touchdown pass to a wide open other receiver.

    When such great plays happen infrequently, you have to wonder how many of those the quarterback is missing in comparison to the better quarterbacks in the league, who make those plays more routinely.

    And again what’s difficult is that we have little way of knowing that, because it’s hard to know much about something that doesn’t happen.
     
    KeyFin and resnor like this.
  20. Two Tacos

    Two Tacos Season Ticket Holder Club Member

    11,121
    5,828
    113
    Nov 24, 2007
    He cannot now. Will he be able to in the future is the question. He throws a great ball, demonstrated good feel for the pocket last week, etc... he needs to make faster better decisions/reads. The O line quality might speed that up for him by necessity. If he could be the answer it'd be best to know that before the draft, or they're just going to be trading him for picks without great leverage. Imagine what trading the Tua pick could turn into. Teams fall in love with potential. Dolphins will too. Winning with that line might be the only way Rosens in Miami long term. Tua gets drafted with the number one next year. If it's Miami picking him, that's it, Tua's the future.
     
  21. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    Of course we can see that. We watch a replay and we see a QB dump it off while not seeing the big play available. So I was actually thinking about this and didn't put it in to my things to judge a QB on...but sort of did with the "did the QB make a good decision". I would assume in every offense in the league, they want the QB hitting the big play if it's there, not the dump off. So the QB should absolutely be penalized if he misses a better option.
     
    KeyFin likes this.
  22. Finatik

    Finatik Season Ticket Holder Staff Member Club Member

    4,323
    4,012
    113
    May 2, 2014
    SO Cal
    I saw him pointing out the MIK and calling out defenses. SO it seems he's starting to pick it up.
     
    KeyFin likes this.
  23. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    Just to explain how one would solve that problem mathematically, suppose you have richer stats (like what KeyFin was proposing or like what next-gen stats provide) that tell you the league-wide probability of a successful completion in different conditions – this could include things like the distance the QB needs to throw the ball, where on the field the ball needs to be thrown, and separation of WR from DB, etc...

    Now.. for simplicity assume that the goal is just to gain as many yards as possible (ignoring for the moment the requirement that you have at most 4 downs to gain 10 yards). Then for each possible throw you can calculate an "expected yards per play" (EYP) by multiplying the probability of completion in that condition by expected yards gained (which could theoretically include expected YAC if you wanted). EYP is greater the higher the completion probability and also higher the more expected yards gained.

    It's now easy to quantify how good or bad the QB's decision was because the larger the EYP the better the choice.

    Of course that's just a single play and ignores the rules of football like having to make 1st downs to keep drives alive. To add in that complexity what you do is simulate different sequences of decisions the QB could have made and see how good total EYP is with restrictions like: if after 3 downs total EYP is less than 10, then you don't continue anymore unless you are in a situation where stats tell you teams are more likely to go for it on 4th down, etc...

    And one can look at more than EYP and add in expected points if you want. Just saying this isn't difficult to model if you have the right stats.
     
    KeyFin, resnor and The Guy like this.
  24. jdallen1222

    jdallen1222 Well-Known Member

    2,752
    1,373
    113
    May 31, 2013
    Plantation, Fl

    They're all dropping the ball but he's clearly better then Ballage(who I was comparing him to).
     
    djphinfan likes this.
  25. Vertical Limit

    Vertical Limit Senior Member

    12,162
    5,057
    113
    Nov 25, 2007
    Btw remember that extension we gave Grant.. boy has that paid off well, hes been dropping evrrything thrown his way..
     
    KeyFin likes this.
  26. AGuyNamedAlex

    AGuyNamedAlex Well-Known Member

    3,582
    2,579
    113
    Sep 12, 2015
    QB rating matches consensus perception not because its are a product of the QB but because 99.9% of people will not remember the context of the situation you were in. They only have the stats to look at to begin with.

    I agree that no supporting cast makes Ryan Leaf into Peyton Manning. However, as I've been maintaining, QB's who put up similar type stats at the very least need to be visually evaluated.

    Also I strongly disagree that sample size matters in the NFL. Games arent a coin flip where given enough time things in theory even out. It's possible to be stuck with a crap team 90% of your career and not have the numbers you could have in another situation, even if you still succeed.
     
    resnor likes this.
  27. The_Dark_Knight

    The_Dark_Knight Defender of the Truth

    11,817
    10,321
    113
    Nov 24, 2007
    Rockledge, FL
    I take all of this QBR with a grain of salt. It doesn’t tell the full story of a team’s performance. Take for instance QB (a)...
    19/22, 227 yards, 2 TD, 0 int

    And QB (b)
    27/45, 357 yards, 3 TD, 2 int

    Now let’s go further, before you guys break out the QBR calculators, let’s say the QB (a) loses and QB (b) wins. By the QB rating system, who’s the better QB?

    Well A has a higher QBR with 140 while B has a much lower rating of 95. If the QBR is so relevant, then why did A lose? Why did B win?

    Let’s flip it...A wins and B loses. B put up a higher offense output than A bit lost. Again, why?

    This is why I love football above all other sports. It is the only true team sport and it doesn’t matter what the QB does, if the rest of the team isn’t doing their job, you’re going to lose
     
    KeyFin likes this.
  28. Sceeto

    Sceeto Well-Known Member

    13,501
    6,246
    113
    Oct 13, 2008
    New York
    Ahhhhhhhhhh!!!!!
     
  29. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    Since 2004 there have been 29 regular season games in the league in which one team's QB had a passer rating between 135 and 145 (inclusive), and the opposing team's QB had a passer rating between 90 and 100 (inclusive).

    The teams with the passer rating between 135 and 145 are 28-1 in those games, and the sole defeat was by three points, 37-34.

    So, please explain how passer rating isn't relevant under the circumstances you outlined.
     
    Pauly and cbrad like this.
  30. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    It’s possible, but not probable, to be stuck on a crap team for 90% of one’s career, because the league is geared toward parity among teams. This is likely why situational factors become equivalent across quarterbacks over the long haul.
     
  31. AGuyNamedAlex

    AGuyNamedAlex Well-Known Member

    3,582
    2,579
    113
    Sep 12, 2015
    Is there really parity among teams though?

    I'm not saying that there is gigantic lack of it in the middle but there are definitely perennial winners and losers, and they generally last quite a while.

    Also I agree with you that ideally you want a QB that succeeds to a certain degree despite circumstance, but i feel like the reality is those guys arent just available to grab every year.

    I guess my real question to you is this.

    If there were two QB available with similar statistics but one came from what is considered a poor supporting cast while the other had what was regarded as a great cast would you choose solely on the numbers?
     
  32. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    There are precisely three teams in the league that have an average record of 10-6 or better in the 15 seasons since 2004. The other 29 teams have a worse average record.
     
  33. The_Dark_Knight

    The_Dark_Knight Defender of the Truth

    11,817
    10,321
    113
    Nov 24, 2007
    Rockledge, FL
    I already did
     
  34. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    No, you said why did team A lose, and why did team B win. But that didn’t happen in reality. That happened only in your mind.

    In reality, team A won, and it was because of passer rating, which completely nullifies your point.
     
  35. AGuyNamedAlex

    AGuyNamedAlex Well-Known Member

    3,582
    2,579
    113
    Sep 12, 2015
    Okay, now look at division winners in the AFC the last ten years.

    Every division has 1 to 2 teams that make up large portion of their playoff appearances, and often when they swapped one was the wildcard generally speaking.


    Actually looking at how much success teams have had, using the playoffs as a barometer, is a better way to see which teams are the perennial winners I was speaking of.
     
  36. cbrad

    cbrad .

    10,659
    12,657
    113
    Dec 21, 2014
    You only need one source of random variation for there to be random variation. So ANY stat you look at should exhibit sample size effects.

    For example: standard deviation of Brady's adjusted passer rating after X games across the 15 years where he started all 16 games.
    [​IMG]

    As you can see sample size matters.

    Also, "coin flip" doesn't necessarily mean "equal probability of heads or tails". You can assign any probability to "heads" which allows you to model cases where the change across time isn't that great. Regardless, the empirical evidence shows that "averaging out" occurs, and I can show that for any football stat.


    It's a question of degree. Clearly, team strength is not random from year to year, but it's also not without an appreciable level of random variation. First, note that IF win% for a given team was totally random, then the standard deviation in win probability is 0.29.

    Compare that 0.29 to the actual average standard deviation in win probability over a 5 year period (can't go too long or it gets closer to 0.29) for NFL teams since 1970 where the beginning of the 5-year period is marked by the x-axis.
    [​IMG]

    Note that the average standard deviation is around 0.15, which is midway between no random variation and pure random variation. So I think it's hard to say there's "little parity" based on that. It's somewhere in the middle.

    In any case, that 0.15 means that 95% of the time an 8-8 team will have 5-11 expected wins across the entire 5 year period, with only 5% of teams going beyond that. It's also interesting to see the effect of the salary cap from the mid-90's increasing parity by a small amount.
     
    The Guy likes this.
  37. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    No. They did not win because of passer rating.
     
  38. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    What did they win because of?

    Remember, what we’re talking about here, under the circumstances outlined by the fellow above, are games in which there is a passer rating differential of 45 points.

    Good luck finding a stronger explanation for winning and losing under those circumstances.
     
  39. resnor

    resnor Derp Sherpa

    16,329
    9,874
    113
    Nov 25, 2007
    New Hampshire
    They won because they were successful on their plays that they ran. The passer rating is an effect, not a cause. IOW, successful passing plays lead to higher ratings, not vice versa.
     
  40. The Guy

    The Guy Well-Known Member

    6,598
    3,323
    113
    Oct 1, 2018
    What you’re saying is the equivalent of, “my house isn’t hot because of its temperature; it’s hot because of the heat inside of it.”

    In other words, we’re talking about a distinction without a difference.
     

Share This Page