Warning: Longish post.
There have been a lot of posts recently which have involved a lot of nerd fights about statistical analysis of the NFL. I fully endorse the use of stats and really appreciate how much more statistical analysis is available now compared to the past. But the NFL has major limitations on how useful stats can be, especially compared with sports like MLB.
Limitation 1. In Football time and field position are assets. So a 1 yard rush on 3rd and 5 is considered a failure in advanced stats, yet for a team that really wants to burn another 40 seconds off the clock, their primary aim on the play might be to complete the play with no fumble and no clock stoppage so the coach would call it a successful play. If the play is in field goal range another goal of the play might be to end the play in midfield to allow an easier FG attempt. Since stats measure yards and downs, not field position or time, what can be successful in-game might be measured as a failure by statisticians.
Limitation 2. Almost nothing in football is what a statistician would call an independent variable. An independent variable is if I change X how does Y change. A simple example in baseball is batting percentage against right handed and left handed pitchers. The pitcher’s handedness is the independent variable and you can measure the difference. Since everything is dependent on other factors then you end up with chicken and egg arguments that are unprovable. For example people try to prove a WR’s performance was helped/hindered by the QB. The QB’s stats are affected by the WR’s abilities, the WRs stats are affected by the QBs abilities and both of their stats are affected by the O-line, the quality of the opposing Defense, ad nauseum.
Limitation 3. Game situation is king. Teams playing behind are much more likely to try high risk/high reward plays. Teams playing ahead become more risk averse. This means that teams that are behind often end up asking their QB to throw more high risk passes, and ask their RBs to extend plays, increasing the risk of fumbles. So teams that are behind are more likely to commit fumbles than teams ahead on the scoreboard. i.e. being behind means you are more likely to commit a turnover, and teams ahead on the scoreboard are less likely to commit turnovers. Teams that are behind often blitz more to try to force a mistake by the offence, yet if the blitz doesn’t get home fast enough it gives the opposing QB opportunities for a big play.
Yet it’s an ingrained belief that the turnovers cause a team to lose, when it can be that teams commit turnovers because they are losing. People might say 20 plus carries for a RB caused a team to win, but it can be the RB got his 20 plus carries because the team was winning.
Game situation dictates what type of passes are thrown, what type of defenses are used, run/pass ratios. So stats are never equal because teams are never in equal positions.
Limitation 4. Stats are compiled without knowing what play was called or what the player’s actual assignment was. So players might get a neutral grade for doing nothing much yet their coach might give them a critical failure for missing their assignment on the play. Other times a player might have a critical success that allows a play to succeed, for example the selling of a route by a WR that drags a DB out of position, yet it gets recorded as a nothing by the stat keepers.
Limitation 5 Playcalling. For example if we have a relatively slow WR with great hands and good agility. If he is asked to run lots of curls and comebacks he’ll have a crappy YAC, yet if the same WR is asked to run a lot of bubble screens he’ll have a good YAC.
Limitation 6. Advanced stats are based on someone else’s perception of reality. The people who compile them compile them based on what they think is important. The simple fact that sometimes DVOA and PFF and other so called advanced sites will sometimes disagree strongly is proof that they are imperfect.
So while stats are valuable, and they have improved by several orders of magnitude in the last 10 years, it is very dangerous to rely solely on stats to evaluate a team or player. In the NFL everything is dependent on everything else.
-
Guest
-
The only valid arguments are limitations 4 and 5, precisely because we aren't "measuring" play call or play design.
The others are in principle wrong. You can measure time and field position, as well as whether a team is ahead or behind, so in principle you can condition on those, eliminating limitation 1 and limitation 3. Limitation 2 isn't true if you use statistical techniques that look for structure in data, like factor analysis or principal components analysis. The statistical tools won't tell you how to interpret the data, but they can tell you which variables are most important and can help create models that can be tested so you don't have "chicken and egg" issues.
Finally, limitation 6 isn't true if it's just simple descriptive or inferential stats, which are just logical implications of the data. Those don't depend on different people's perceptions of what's important. DVOA is NOT a stat despite what some people here like to claim. It's a model based on stats with subjective weights implicitly assigned to things through that model. Descriptive or inferential stats are logical implications of the data, not something that is based on data but includes assumptions in addition to it. -
The NFL does not inherently have any limitations. Everything can be measured and quantified. While people have yet to measure and quantify many things associated with the game of football, it certainly is feasible. Simply requires enough intelligence and initiative.
-
cbrad likes this.
-
http://espn.go.com/blog/oakland-rai...reading-in-dan-marino-territory?ex_cid=espnfb
I like Carr, he is a very promising young QB. But this is a perfect example of "stats" without facts.
Despite some of our more often wrong contributors, these modern day "milestones" aren't all that spectacular. 4k yards is the new 3k yards, TD passes are as common as rushing TD's.
Times have changed folks, you can't judge a QB from today with a QB from the early 90's without considering WHY.
A good example? John Elway had 1 4k yard passing season in 16 seasons, Ryan Tannehill has 4 in 4 years. So all the talks of records falling is due to rule changes and volume...certainly not talent.Rock Sexton, shamegame13 and dolphin25 like this. -
For example a generic projection of 'home teams wins by 3' over the course of a season beats every sophisticated model I've seen. Another example is that passive Index funds out perform over 90% of actively managed funds. What are probably the most sophisticated and detailed models in Human history the climate models by the IPCC have all performed worse than the null hypothesis
I know that stat heads, and I know I've been one, believe that the answer to the problem is more data.
The most reliable and most accurate models in many of the fields I've looked at are the ones with the fewest moving parts. Finding out what the critical moving parts are is much more important than more detailed data.P h i N s A N i T y, Finster and cbrad like this. -
For example, you can do some simple fitting to show that you get a decent trend-line from around 1980-2005 where there's a "normal" (in the statistical sense) distribution around a line that goes from 3200 passing yards per year in 1980 to 3300 in 2005, giving you an average of 4 yards increase in passing per year over that time. Then from 2006-2015 you go from 3300 to 3900 with a very stable slope of 60 yards passing increase per year.
So you want to compare stats from 2015 to say Marino's time in 1983? Using those crude trends, you are looking at 3900 vs. 3312, which is a 17.75% increase from 1983, or calculated from above, it's a 15.08% decrease. So take Marino's total passing stats and multiply them by 1.1775, or take 2015 stats and multiply them by 0.8492 and you can compare.
Crude yeah, but stats give you the ability to make those types of comparisons while eyeball arguments do not.
Oh, and the stats also show which rule changes actually had an effect while without stats you'd have no way of quantifying the effect of any rule change. Clearly the one in 1978 that allowed the OL to grab DL with extended arms, as well as the 10-yard chuck rule reduced to 5, had a tremendous effect, helping Marino shatter records by far more than he otherwise would. What explains the very steady rise in total passing yards from 2006 or so is harder to pin down because it's so steady year after year. -
It has nothing to do with total number of measurements (or parameters in the model). It has to do with total amount of variance introduced by each extra parameter relative to the increase in predictive power. There are many cases where the extra variance introduced is small enough that you get an overall increase in predictive power with each extra parameter, up to some point after which the opposite starts to occur (what you're talking about).
Where that tipping point occurs is different for each type of model. I've worked on models of how humans process visual input that include the physical optics of the eye, the different spatial resolution of the output neurons of the retina (retinal ganglion cells), parameters that estimate the shape of their receptive field (how each neuron responds to different stimuli at different points in the visual field), etc.. and all that is based on estimates from known human or monkey (macaques) anatomy and physiology as well as psychometric functions that describe human detection performance of certain localized spatial patterns at different points in the visual field. Those models get more and more accurate even up to 15-20 parameters, and most of those parameters have a physical or biological basis. Then, you start adding extra stuff to account for contextual effects due to stuff that happens in the next stage of processing in visual processing areas of the brain (so not the retina) and then you start to slowly see what you're talking about (usually predictive power just doesn't change much.. it levels off instead of getting much worse).
All depends on type of model and how much it's anchored to phenomena you can directly test. Those climate models have too many things the computer has to estimate that can't be tested directly so the variance does at some point increase a lot with each added parameter. -
The passing law year was 2004, that's the year the 5 yd chuck rule was actually starting to be enforced, and there's why numbers are going up since 2004, it's true as you said that the rule was put in in 78, but it was never enforced, so Marino didn't actually benefit from it.
"Illegal contact" penalties were the highest ever in 04, and Peyton broke Dan's TD record that year with 49, and then in 07 Brady broke it again, 50 this time, and now we see the dramatic affects, 15 of the top 20 TD seasons are from 2004 or beyond, 16 of the top 20 in passing yds, and it's only been 16 years. -
Anyway, what makes the 2005-present increase problematic to understand is that it's a very predictable and steady increase per year of 60 yards passing over a season, so it's not a single effect after a single rule change. It's happening each and every year. Not sure I can explain that, even accounting for the arguable increase in talent at the QB position. -
So, I think this is what helped the passing era to start, but I also think it coincided with some QB/WR talent that was coming into the league, like Tarkenton, Staubach, Griese, Bradshaw, and Fouts, but from then until 04 the passing game grew slowly, since 04 it's changed drastically.
I think it takes a couple years for everyone to get adjusted, and also the've been trinkling in different QB/WR protection rules since just prior to 04, and more since, which is why the stats really start going up post 06.
1983= 2 QBs over 4000yds, 10 over 3000, 25 over 2000yds__ 1 QB over 30 TDs, 10 QBs with over 20, 18 with over 15.
2003= 2 QBs over 4000yds, 15 over 3000, 27 over 2000yds__1 QBs over 30 TDs, 11 QB over 20, 18 over 15 TDs.
ΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩΩ
2004= 5 QBs over 4000yds, 16 over 3000, 28 over 2000yds__1 QBs over 40 TDs, 4 over 30, 15 over 20, 23 over 15 TDs.
2015= 12 QBs over 4000yds, 23 QBs over 3000, 30 over 2000yds__ 11 QBs over 30 TDs, 21 over 20, 25 over 15 TDs.cbrad likes this. -
One applicable example - for a number of years, the number of 'corner' 3-pt shots attempted was a very high predictor of a team's performance in the NBA. A model using just that measure was able to beat Vegas closing lines. The challenge was actually obtaining that data - many people were manually charting every game by hand to obtain this data.
The more people measure, the more opportunity people have to identify quality predictive data.Pauly likes this. -
I feel like people are saying the OP is wrong because in a perfect world with unlimited info stats can tell us anything, while the Op is basically saying we don't live in a perfect world and the stats we have available aren't telling us what we need to know.
-
But yes, he's more on target when you only restrict it to stats currently used in the NFL. -
Primarily because the interactions are more interdependent in nature than independent.
If we take a stat like yards/attempt which is very solid indicator we really can't pull it apart into separate components like;
QB ability
receiver ability
play design
play calling
game situation
defensive pressure
defensive cover
defensive play design
defensive play calling
and assign a discrete value to each one. There's always going to be a degree of subjective weighting.
While I believe stats will continue to improve, I don't think the nature of the NFL will allow pure statistical analysis to replace eyeball evaluations. BTW I think anyone who relies one the eyeball test over stats will be wrong more often than not.
The future of NFL analysis is a better mix of eyeball and stats.djphinfan likes this. -
With the 3 point data, that's based on an observable and testable phenomona. Whilst that sort of data wasn't being measured in the NFL 5 or 10 years ago, all the major data points are being measured now. Which has lead to much better statistical analysis, but no silver bullets as far as I know. -
NFL teams prepping for RFID data dump