2019 Cleveland Browns Regular Season

[Jordan]

I know those weren't your analytics models, but my honest takeway from all this is that those models are hot garbage. Perhaps the reason for that is, as you say, that they're not good at accounting for context. But that just explains why they're hot garbage -- it doesn't actually make them better.

I would say they are good at evaluating most teams, but this year’s Browns are actually a really unique outlier. They are the first team in NFL history for a lot of things, including the context around Freddie’s hire, Baker’s performance after Hue got fired, getting a receiver as talented as OBJ and pairing him with a young team, etc. Interestingly, I think last year’s Rams are a similar comparison, and the models also had them at 8-8 for comparable reasons.

The thing is, ELO models generally are great at predicting the NFL season. Broadly speaking, historical teams like this year’s Browns - new coach, second year QB, below average offensive line, lots of roster overhaul, etc. - play below their expected points totals. Again, the problem is the Browns are a historically unique situation.

I was trying to do is explain why analytics models are projecting the Browns to go 8-8, and more importantly, why I think they are wrong. I think the models are projecting the Browns somewhere between 30-60 net points (or one to two wins) too low.

 

[red.]

 

[col63onel]

The thing is, ELO models generally are great at predicting the NFL season. Broadly speaking, historical teams like this year’s Browns - new coach, second year QB, below average offensive line, lots of roster overhaul, etc. - play below their expected points totals. Again, the problem is the Browns are a historically unique situation.

/rant/

Analytic models catch so much crap if they aren't PERFECT. It's like people think either the models are perfect or awful.

If a model predicts games 70% correctly, and a person only predicts games at 60%, people will rail against the 30% the model got wrong, but perfectly accept a larger margin of human error.

It's not enough to be better. They have to be perfect.

/end rant/

 

[Jordan]

/rant/

Analytic models catch so much crap if they aren't PERFECT. It's like people think either the models are perfect or awful.

If a model predicts games 70% correctly, and a person only predicts games at 60%, people will rail against the 30% the model got wrong, but perfectly accept a larger margin of human error.

It's not enough to be better. They have to be perfect.

/end rant/

Agreed! 100%

 

[The Human Q-Tip]

/rant/

Analytic models catch so much crap if they aren't PERFECT. It's like people think either the models are perfect or awful.

If a model predicts games 70% correctly, and a person only predicts games at 60%, people will rail against the 30% the model got wrong, but perfectly accept a larger margin of human error.

It's not enough to be better. They have to be perfect.

/end rant/

No, that isn't the issue at all.

Models don't have to predict results with 100% accuracy to be good models. The model for flipping coins isn't flawed just because you got a result of 47 heads and 53 tails. The problem is when there is a systemic flaw with a model/models that leads you to say "that model is off" even before the games are played.

@jking948 gave reasons why these models are believed to be off even before the season begins. They are bad at context, bad at looking at trends, and bad at factoring in uncertainties. A model that doesn't consider things like a coaching change or change in offensive scheme during a season is simply flawed for that reason.

So to use your example of the models being right 70% of the time, and humans being right over 60% of the time, what do you think are the odds that the models are more likely to be right than the humans in the case of the Browns?

 

[col63onel]

So to use your example of the models being right 70% of the time, and humans being right over 60% of the time, what do you think are the odds that the models are more likely to be right than the humans in the case of the Browns?

I have no idea how accurate they are on the Browns.

To me the question is, if they are right more often than human models, but wrong on the Browns, are they still hot garbage?

 

[The Human Q-Tip]

I have no idea how accurate they are on the Browns.

Do you agree with @jking948 regarding his specific criticisms of those models when applied to teams like the Browns and Rams?

To me the question is, if they are right more often than human models, but wrong on the Browns, are they still hot garbage?

Depends on what they get right, and what they get wrong. If they have a systemic bias that is predictable and exploitable, then I'd say yes, they're worse.

 

[col63onel]

Do you agree with @jking948 regarding his specific criticisms of those models when applied to teams like the Browns and Rams?

Maybe, Not as well versed as he is.

Depends on what they get right, and what they get wrong. If they have a systemic bias that is predictable and exploitable, then I'd say yes, they're worse.

So even if they perform better, you think they are worse?

 

[The Human Q-Tip]

Maybe, Not as well versed as he is.


So even if they perform better, you think they are worse?

They're performing worse in what I'd characterize as the more important predictions. And that's actually assuming you're right about them being better overall

 

[col63onel]

They're performing worse in what I'd characterize as the more important predictions.

What makes what they get wrong more important than what they got right?

And that's actually assuming you're right about them being better overall

Well of course if they are less accurate overall, they are bad models. But if they are more accurate overall, I don't see why they're considered worse.

 

[red.]

Spoiler

FLASHBACK FRIDAY!

 

[The Human Q-Tip]

What makes what they get wrong more important than what they got right?

Because a marginal overall advantage (again, unproven) in making predictions for teams that have no special context, different circumstances, etc., is relatively useless. Those are the easy ones. The overall advantage (if it exists) may even be within the margin of error, and it doesn't even help you in terms of predicting specific teams or games. You may believe (for example) that they're going to get 15 out of 29 games right against the spread consistently (which is "better than average"), but you don't know which games, or which teams, are the good bets/wagers.

But if those models make a predictable, systemic error that is of greater than usual magnitude, that is much more easily exploitable, and a bigger vulnerability. You can actually look at a specific prediction, have a high probability that specific prediction is wrong, and act on it.

A good system shouldn't do that.

 

[col63onel]

A good system shouldn't do that.

Fair points.

I don't see how it makes worse than a human model. Are human models better at any of those weaknesses?

 

[Jordan]

Fair points.

I don't see how it makes worse than a human model. Are human models better at any of those weaknesses?

So just to chime in here, I still don’t think the models are bad.

The fact is, new coaches generally underperform, teams with lots of new players generally underperform, teams with bad offensive lines generally do not do well, teams with second year QBs generally underperform, and a team’s performance the second half of the prior season has little impact on predicting their record the following season.

With the Browns, there is additional context here for most of those things, but not all of them. I can write off the new coach and predicting using last season. There is serious context missing. But, having a lot of new players, a poor offensive line, and a second year QB absolutely are making me more conservative on this team than most posters.

Regarding the Rams, models saw Goff as a mediocre QB going into his third year, a team that overperformed it’s net score differential the year prior, with an aging defense, and inefficient offensive playcalling. What it missed is that the coach that took over the season prior clearly had a positive impact on their QB, their aging defensive players all filled specific roles to make their DT excel, and the playcalling was actually setting up for a hyper efficient season the year prior.

With that said, the models usually miss two or three teams every year. RARELY are they projecting great teams to do poorly. Usually the ELO models are wrong by projecting certain teams to do slightly worse than they play, or injuries throw a wrench in the prediction. Rams were a unique outlier.

All I am saying, and it keeps getting twisted, is that I don’t think the models are a great projection for *this* Browns team. That’s it. I’m not saying the models are bad, that the Browns will won the Super Bowl, that the Browns will go 0-16, etc.

 

[Triplethreat]

Well boys. How about we just say fuck it and bring every outrageous personality here and let the inmates run the damn asylum.