The Arc has been going for around six months now, and I’m still figuring out how to improve it. Part of the challenge is figuring out exactly what people want from an AFL graph blog, so thanks to the 230 of you who’ve helped me by answering a reader survey.
It turns out that the overwhelming majority of people who responded to the survey like The Arc’s content. Good stuff! I really appreciated the positive comments and constructive suggestions that a lot of you left on the survey.
Of course, we can’t infer much from a small, self-selecting survey, but this was encouraging nevertheless.
Most (80.6%) respondents are happy with the mix of longer and shorter posts, with the remainder more or less evenly split between people who’d like to see more long posts and those who prefer short posts. A clear majority (70.4%) are happy with the current level of statistical detail in posts at The Arc, although a substantial minority (27.9%) would like more detail. A few of you suggested accommodating both these wishes by keeping posts fairly light on detail, and including more information in footnotes and ancillary posts. That’s a good suggestion and I’ll do it a bit more often, although not on every post.
I also asked what feature you’d most like to see added to The Arc. A slim majority (52.8%) opted for team rankings and game predictions. I was a little surprised this option won. I’ve stayed away from this stuff, because other AFL sites like Matter of Stats, Figuring Footy, Hurling People Now, and plusSixOneblog have it covered. I don’t have much to add to the ratings and prediction models those sites have built. Still, I’ll take this on board and have a ratings system ready to go before the next season starts.
The second most-requested feature was player ratings and forecasts, with 25.5% of you opting for this. Only 11.3% of you rated in-game win probabilities as your most-wanted feature. I was surprised that figure was so low, as I like consulting the win probabilities at sites like Fangraphs to tell me when it’s safe to turn off a baseball game. I’ll put more time and effort into other things, as that doesn’t seem to be a high priority for readers of The Arc.
Twenty-two survey respondents (10.4% of the total) didn’t opt for any of the three options, instead suggesting alternative features that should be added to the site. The most commonly requested addition was more analysis of tactics and game styles. Here are some of those responses:
- No predictions, but analysis of why teams win or why players are better than others.
- Things that explain how a game is won.
- More analysis on what wins games.
- Analysis to generally better understand the game.
- Analysis that helps understand tactics. macro stats are easy. micro stats are where the value-add is at.
I appreciate where these requests are coming from. Footy media has a lot of emphasis on what happened and what will happen – who won on the weekend and who will win this weekend? We’re relatively underserved on ‘why’ questions – why are some teams good and other teams not? I try to answer ‘why’ questions where I can, and I’ll try to do this a lot more next season. The problem is that this sort of analysis is hard to do, particularly with the meagre data the AFL chooses to release publicly. ‘Micro stats’, as one respondent put it, are definitely more useful for analysing game styles and tactics. But I can’t analyse data I don’t have.
There were a few respondents who pooh-poohed the idea of forecasting and predictions. I disagree; I think there’s a lot of value in forecasting. I wouldn’t really trust any theory about what makes a good team or a good player unless that theory was tested by making forecasts about future performance. Predictions aren’t just done to help with footy tipping or betting; at their best, they’re a way to test hypotheses about how the game works.
Here are my responses to some other comments, with the respondent suggestions in bold followed by my responses:
- All of the above. I’m glad there’s interest in team rankings, player ratings and win probabilities! But I probably can’t do all of these things, which is why I asked for readers’ views about which would be of most interest.
- Random facts and stats and other quirky stats. I’m not sure exactly what these requests are getting at, but I suspect these readers would like stats of the “Adelaide has won 3 of its last 5 games when playing in Sydney on days after there’s been more than 2 mm of rainfall” variety. I’m afraid you won’t find much of that here. I think the AFL has too many “random facts and stats” and I don’t find them that interesting, personally.
- More rugby league. Nah.
- What about 70% of games are won if you win contested possession. But CP is made up of 5 statistical categories. Can you break that down further? What combined stats win closer to 100% of games? Unless and until the AFL chooses to release more data, I can’t disaggregate contested possessions or the other counting stats.
- Tactics and effectiveness against specific opposition. I will try to do more on tactics, as it’s an area that interests me greatly, but again we rapidly run up against data constraints here. There’s only so much you can do with the box score stats. Still, sites like HPN and analysts like Ryan Buckland manage to extract some really useful insights from public data, so I will try to lift my game in this area.
- Stats about national women’s league footy when it starts. I’d love to do this! I’m excited about AFL Women’s. I hope the league releases detailed stats in a useable format. If it does, I’ll take a look and see what I can do.
- These options are all forecasting. Yes! Forecasting is interesting, I think! If you have a theory about what makes a good team or a good player, the best way to test that theory is by making predictions about future events and then testing the accuracy of those predictions.
- Keep questioning “common knowledge” – e.g. does Contested Ball win you the game, or other things that everyone knows and says… but might be wrong. OK!
- Long term trends – scoring, margins, anything! OK!
- Big picture assessments like how many teams have ever one from an x point deficit at half-time. I’ve pretty much done that already!
- A better explanation of your graphs. The problem with this suggestion is I’m not sure if it’s from someone with a statistical background who would like more information about the technical detail, or from someone who finds the existing level of detail overwhelming and would like me to explain things in simpler terms. That’s a balance that is hard to strike and I’m not sure I’ve got it right, so please keep giving me feedback on this.
- NONE of the above – keep it entirely different from what we read in the tabloids. I didn’t really understand this… do the tabloids run models that estimate the teams’ and players’ quality? I’ve been out of Australia for a couple of years, but I’d be surprised if they do…
I also asked for other comments and suggestions. This received 75 responses! The most common was just “Thanks Matt.” Thanks, Reddit. A lot of other people just left compliments and positive comments, which I appreciated.
Some other comments:
- It would be interesting to look into whether younger brothers make better footballers than their older brothers. My impression is that this is the case. However it would be interesting to see if it is significant. Interesting question, but not one about which I have data.
- in game win probabilities don’t gel with me. How come?
- Have you considered adopting a footy plus graphs minus footy approach to maximise the graph density? Just a thought. MY TAKE: :/
- A little WA centric. Focus on all teams? Fair comment! Point taken.
- I’d love to see more info about how you put the graphs together. OK! I might do a post walking through the process at some stage.
- Not sure what the tech term is but I find the scattergun graphs v difficult to interpret. This means I’ve failed! I won’t abandon scatterplots, but I will put more effort into explaining them.
- Stop using your Twitter account as a personal soapbox. Nah, sorry.
- Less snark. Nah, sorry.
- Make a tweet for every graph in each post. Sometimes I miss the first or will only find other sections interesting. Use a scheduler to mix through day. I’ve had a few comments along these lines, mostly from Twitter. I don’t really know what the right approach is. If I don’t include a graph in a tweet promoting a new post, then fewer people will read the post. If I do include a graph, I get people telling me they didn’t click the link to read the post, because they thought the graph in the tweet covered it. I’m not really inclined to tweet every graph from every post – that defeats the purpose of having a blog at all!
- Only thing I haven’t liked is mocking losing teams by tweeting “lol” – very lame. Yeah fair enough. I shouldn’t do this, except for Freo.
- I’d like to know where the statistical data is sourced from and how it could possibly be shared to allow others to perform their own analysis etc. All the data comes from AFLtables.com. You can go to AFL Tables, copy some data, and start making your own graphs, just like I did. If I use other data sources in the future, I’ll note it.
- As you have written previously, would love to see stats on where the disposals were, the effectiveness, where the free kicks were paid. I’d love to do this, but the AFL chooses not to publicly release data that would make this possible.
Thanks again to everyone who responded to the survey. This has really helped me to figure out what’s working, what isn’t, and what you’d like from The Arc in 2017. I’ll be doing a lot of work in the off season to refine and develop some new features. I’ll still be posting, but only occasionally.
If you have any more comments about The Arc, please get in touch via email (from the form on the About page) or Twitter.
