We’ve been doing “statsy previews” here and at Rocky Top Talk back as far as 2011. Early on, it was primarily for the purpose of using a different method of analysis to make the conversation interesting. By the time 2017 rolled around, though, it had proven reliable enough that I thought it worth the time to measure its actual effectiveness against the spread, and on Week 6 of that season, I started keeping track. From that point forward, it finished 283-217 (56.6%) against the spread in 2017. It was 65-33 (66.33%) when the machine’s confidence rose to a certain level that I am protecting like a national secret for now.
I tracked it again last season, and it, of course, didn’t do as well. It went 376-380 (49.74%) overall for the entire 2018 season. I wasn’t surprised because it was the first time I’d tracked early season results that out of necessity relied on somewhat stale data from the prior season. But 49.74% is known as “losing” if you were actually wagering.
On the other hand, at a certain confidence level, it went 169-140 (54.69%). So, I considered it still worth watching. By the way, I’m not using any of this data to actually wager, and neither should you. Trust it for fun and discussion purposes, but don’t give it your wallet.
When analyzing the results from last year, I realized that part of the problem, apart from the early-season data problem, was that it didn’t filter out FCS games. This is a comps system, and games against FCS opponents really shouldn’t be considered comparable. That’s a whole different neighborhood, man. So, this offseason, I tweaked the machine to filter out those non-FBS games.
We’ll see if and how much difference removing FCS comps makes. So far this season, the SPM is 6-7 (46.15%) overall and 5-2 (71.43%) when confidence is sufficiently high.
The early-season problem
Part of the problem with this comps model (and really, any predictive statistical model) is what to do with early-season games before there is any or enough data. When there isn’t enough data from the current season, you have to jump off last year’s cliff and make some adjustments and guesses on the way down.
Generally speaking, the way I’ve done it is to use last year’s data and then assign each team an appropriate adjustment. Each team is assigned a benchmark from the prior season, which is a combination of a weighted AP poll ranking and a weighted S&P+ ranking. Basically, this is a measure of how good were they last year. This is then compared to our current-year power ranking, which is a combination of the prior year’s benchmark and a team’s returning production and recruiting measures. This attempts to get at how much production a team has lost from the prior year and how much it’s likely to matter. From there, we assign each team a number that attempts to accurately measure how much that team is expected to change from last year to this year.
So, teams like Alabama and Clemson — those that did well on the field last year and have enough talent on the roster to offset losses in production — are essentially the same teams as last season. Some teams, though, have changed fairly significantly. Our results suggest that teams like North Carolina, Tennessee, and Florida State have the highest probability of the most positive change, and teams like Appalachian State, Fresno State, and Utah State have the highest probability of the most negative change. This is an experimental new component, so again, we’ll see.
For the first game of the season, we’ll simply use last year’s data and this year’s team adjustments. As we progress into the season, we’ll phase that out and start using more and more of this year’s actual data.
Tennessee vs. Georgia State
So, here’s what the SPM has to say about the Vols’ game against Georgia State tomorrow. The team adjustments: Tennessee is expected to be quite a bit better this fall. Georgia State is expected to be somewhat better as well.
From the perspective of Tennessee
Tennessee scoring offense for the season: 22.8
Georgia State scoring defense for the season: 37.4
The Georgia State scoring defense is most similar to the following two prior Tennessee opponents (FBS only):
- UTEP 32.8
- South Carolina 27.2
Tennessee scored 24 points against both of those teams.
Estimated points for Tennessee against Georgia State (after 2019 team adjustment): 29.3
Tennessee scoring defense for the season: 27.9
Georgia State scoring offense for the season: 23.9
The Georgia State scoring offense is most similar to the following two prior Tennessee opponents:
- Charlotte 21.7
- Kentucky 26.6
Against Kentucky, Tennessee allowed 7 points. Against Charlotte, Tennessee allowed 3 points.
Estimated points for Georgia State against Tennessee (after 2019 team adjustment): 5.6
Estimated score: Tennessee 29.3, Georgia State 5.6
From the perspective of Georgia State
Georgia State scoring offense for the season: 23.9
Tennessee scoring defense for the season: 27.9
The Tennessee scoring defense is most similar to the following two prior Georgia State opponents (FBS only):
- Texas State 27.7
- Arkansas State 25.6
Against Arkansas State, Georgia State scored 35 points. Against Texas State, Georgia State scored 31 points.
Estimated points for Georgia State against Tennessee (after 2019 team adjustment): 37 (yikes!)
Georgia State scoring defense for the season: 37.4
Tennessee scoring offense for the season: 22.8
The Tennessee scoring offense is most similar to the following two prior Georgia State opponents:
- Kennesaw State 20
- Texas State 19.8
Against Texas State, Georgia State allowed 40 points. Against Kennesaw State, Georgia State allowed 20 points.
Estimated points for Tennessee against Georgia State: 36.6
Estimated score: Georgia State 37, Tennessee 36.6 (!)
SPM Final Estimates
SPM Final estimated score: Tennessee 32.9, Georgia State 21.3
SPM Final estimated spread: Tennessee -11.6
SPM Confidence level: 13.9
Eyeball adjustments
No, I’m not believing that estimate from Georgia State’s perspective, either. That’s probably an indication of something else that should be tweaked under the hood, namely a negative adjustment to the comps of a team that mostly plays against a lower-tier slate.
So, I’m putting more weight on the analysis from the Vols’ perspective. There, the predicted score is basically Tennessee 29, Georgia State 6, a spread of 23. That sounds more like it, although I am inclined to give the Georgia State perspective just a bit of weight and to give them a few more points.
So, while the SPM is going with Tennessee 32.9, Georgia State 21.3, I’m going with Tennessee 35, Georgia State 13. I hope it’s better than that. That’s still not covering the spread.
The Vegas spread is between -25.5 (when I locked it in on Monday) and -27 with an over/under of 57.5. That basically means something like Tennessee 42, Georgia State 16.
ESPN’s FPI gives the Vols a 97.1% chance of winning.
We usually put the S&P+ prediction here as well, but I am unable to find it now that Bill Connelly has moved from SB Nation over to ESPN. If y’all can locate it, please let me know in the comments.
So with all of that, I’m thinking the Vols don’t cover this week. I hope I’m wrong.
What are y’all thinking?
