Battle of the Spaced Repetition Heavyweights: Anki vs. Supermemo Part II

Introduction

I did an experiment to see whether Supermemo’s claims of increased efficiency in learning over Anki passed muster. One commenter on Reddit wondered how my experiment would differ if I added new cards every day instead of having a set number in the collection, since Supermemo’s main strength is in distributing workload. I thought that was an interesting question, so I started a new experiment.

Methods

I’ve been working with a pre-fab deck for the most frequent 5000 words in Spanish for a little while. I had only gotten through about 500 words before I started the experiment. I split the remaining words into two sets, alternating every other word for balanced frequency (for example, 501 went to Supermemo, 502 to Anki, etc.), and set about learning them in frequency order. I did 10 new cards per weekday and kept track of my stats for the month.

Results

Number of Reviews

In total, I did 1308 reviews in Anki and 587 reviews in Supermemo over the course of the experiment. As you can see from the graph, I had spikes in reviews every Monday after the weekend break, with Anki generating more of a backlog after the break (which I also found in my first experiment). I did 36 daily reviews on average in Anki and 33 daily reviews on average in Supermemo, so overall the numbers were fairly comparable, even with the Anki spikes. One thing to note is that the lion’s share of difference in reviews were in learning the new cards. I didn’t often do final drill in Supermemo (which they claim isn’t necessary, and that one should do new cards instead. When I did do it from time to time, I didn’t log the reviews because it was all jumbled up with my other final drill cards, as I keep everything in one collection and was concurrently running another experiment), but if I had, that may have made the number of reviews comparable.

Review Time

I had similar results for time spent. Spikes after a weekend break, worse in Anki, but mostly due to learning the new cards/no final drill in Supermemo. My total review time in Anki was 46.87 minutes and my total review time in Supermemo was 33.37 minutes. My average daily review time in Anki was 2.6 minutes and my average daily review time in Supermemo was 1.85 minutes.

Accuracy

I only tracked accuracy on review cards (non-new cards) each day. My average accuracy in Anki was 91.8% vs. 65.7% in Supermemo. This is consistent with the poorer general retention I had with the Supermemo algorithm in my first experiment, while Anki reached around 90% as it claims to do.

Discussion & Conclusion

It was not surprising that Anki’s workload was significantly higher than Supermemo’s, but I was surprised that it wasn’t old card review that ate up the most time but new card learning. I was not surprised to see the poorer retention in Supermemo after the first experiment, but no less disappointed. Personally, I don’t think the decrease in workload is worth the hit to retention. I do SRS in the first place to be able to remember things for a long time, and it’s frustrating to have old cards show up and fail them. I have also felt that my retention is poor in Supermemo with my collection as a whole, even when I was doing final drill every day. The intervals are just too long for me and my brain, and haven’t adjusted enough in the time I’ve been using it. Maybe things would have been different if I wasn’t learning new items for the first time within the SRSes (which, yes, I know, is not recommended, but for isolated and pretty simple vocabulary that contains many cognates I think it works well enough).

I feel even more confident in my decision to stick with Anki now. I might use Supermemo for incremental reading (unless I can find an alternative I like as much, stay tuned for an article on that) but do my distilled end-product Q&A cards in Anki.

On a tragically hilarious note: Honestly, I intended to continue this experiment for much longer, but despite my daily Supermemo backups, one day it failed to load my collection, and the backup was also corrupted. The next most recent backup, which should have been the previous day, was somehow over a month earlier. I’m not sure what I did wrong, but it kind of put the kibosh on my entire flirtation with Supermemo, and I took it as a sign that it was time to give up. It’s just too buggy, too frustrating to work with, and not rewarding enough to use. Happy studies!

back to garden