Do Mary Sue Tests Work?
Since I first encountered SyeraMikatee’s Universal Mary Sue Litmus Test, I’ve wondered: does it actually work? This isn’t really a fair question, since Syera xirself no longer recommends it, or the word “Mary Sue”. But I spent three years on this analysis, and I’d like to leave it finished:
Successful writers do occasionally create characters who’d fail Syera’s test. But they might have the talent, or skill, to succeed despite breaking those rules. If you’re a new writer starting out, will using the Test make you a better writer? Measurably better?
Well, being a “better writer” isn’t all that measurable. Popular, though — that’s a different story! Everything on the Web has a view counter nowadays, and if you can’t be a great author, writing bestsellers is pretty nice. Can a low score on this test help?
Sadly, it seems not.
Popular Comics Score High
I found a site (the webcomic list) that conveniently lists comics by popularity (views per month). Unlike the other sites I tried, it also lets you sort by date added, to avoid all the “best” comics floating to the top.
I waited for 50 comics to be added, read them, and scored (almost) every character on Syera’s test. And the data showed that higher-scoring comics were more popular:
Here’s how that graph works:
The higher up a comic is, the more popular it was over the month I tracked it (according to
The maths back this up: for non-fanfic (see below), every point scored on Syera’s test means an about 2.6 more views per month (the slope of the dotted line).
Can I Trust these Statistics?
You might have a few questions at this point. Why are some comics on there twice? How come the points don’t match the blue line?
I’ve put some comics twice because the test can be used in two ways: for fanfic, and for and non-fanfic characters. To avoid any judgement calls, I’ve graphed everything both ways: +’s are for fanfic scores, ×’s for “original character”, and you’ll see both on top of each other (✳) where it doesn’t make a difference.
And overall, it doesn’t. The rate for the fanfic-section scores is 2.3 views/month, not much less than the 2.6 if I assume everything’s original. Either way, the maths say it’s not by accident: I’m 95%+ sure this isn’t luck. (If you know your statistics, that’s p = 0.00316 for original characters, and 0.00272 for fanfic).
But the maths isn’t the whole story. With how that graph looks, I suspect a case of Simpson’s Paradox: Do the sums with all the data, and popular comics score high. Slice it differently, and the top scorers might be the least popular. For instance, imagine if I hadn’t scored the arc from Weird Tales of Elderfield to Wargyrl.
Both of these are sort of “right”, but neither one’s telling the whole story. And that’s not to mention all the other things I might still have wrong.
*Isikyus Can’t Science
A lot of the webcomics I looked at had just started, and I didn’t know the characters well enough to really score them. Maybe the higher-scoring characters just had more time for character development.
Even so, there are a fair few over Syera’s recommended 30-point threshold. I could do another analysis to figure out what’s going on here, but that’s a dangerous path. Now I’ve seen the data, there’s a risk I’ll start reading things into it if I analyze any further.
I might follow it up and score new comics someday; in the meantime, you’re welcome to download my data and have a go.
I started this project to learn about writing. In the end, I’ve learned much more about statistics and scientific research.
Everything was more work than expected. I scored over 200 characters, messed up and re-did my calculations at least twice, and now I’m trying to report it without being unreadably dense or straight-up wrong. I have a lot more respect now for people who do this for a living.
As for me, there’s a few webcomics I’d like to catch up on…
I re-read the “risk I’ll start reading things into it” article just now, and realised I’ve already fallen afoul of this.
In the post I sort of suggest high Mary-Sue scores can make a story popular, but I only have an explanation for the opposite correlation. It’s hard to imagine being a better writer makes you *less* popular.
It’s not hard to suggest an explanation now. Maybe most people don’t appreciate good writing. Maybe Syera was right that you need some level of Sueness for a good character, and just set the maximum too low. But to do properly good science I’d need to have that theory *before* I started doing statistics.
Ignore that rule, and you can “prove” all sorts of ridiculous things if you look hard enough (e.g. https://www.tylervigen.com/spurious-correlations).