top | item 35511357

Effective Spaced Repetition

561 points| g0xA52A2A | 2 years ago |borretti.me

262 comments

order
[+] somsak2|2 years ago|reply
>A common failure mode (and I did this more than once, before I got the hang of it) is to use Anki for two weeks, then drop it, and pick it back up six months later only to find you have 600 cards due for review. This is not encouraging, and it defeats the point of spaced repetition, which is to review the cards on the intervals the algorithm chooses.

For successful long-term use of a spaced repetition program, I believe that this model of thinking is unsustainable for most. I used to have this relationship but had to grow past it as my life circumstances changed and prevented me from having that consistent amount of time every single day.

Now, I look at Anki as a way to prioritize my learning time. When I get to it, I have it present me the things that are the most-overdue first. This has meant that I've gone from a typical backlog of ~0 reviews at the end of the day to flexing between ~500-2500 backlogged reviews. Just because I'm not reviewing the piece of information at the exact right time doesn't mean that I'm "defeating the point" of the piece of software. Spaced repetition, even if done imperfectly, is still many times more efficient than traditional study methods.

[+] reitanqild|2 years ago|reply
> Just because I'm not reviewing the piece of information at the exact right time doesn't mean that I'm "defeating the point"

Some of us, thanks to personality and previous experience (diagnosis and/or childhood and/or early work experiences etc) needs to hear this again and again it seems.

[+] laurieg|2 years ago|reply
When people start using spaced repetition and flashcard software they often end up throwing a big list of "Most common 1000 French words" into it and then promptly being swamped by reviews and giving up.

This is a bad idea for two reasons:

When it comes to spaced repetition, quality beats quantity. Only put in the perfect nuggets of knowledge. You're walking through an orchard and grabbing a single apple, not a whole trees worth.

Spaced repetition is to stop you forgetting, not for learning brand new things. When you study a topic and have an "aha" moment or you make a new connection, that's the thing you should put in. When you make the flashcard it should feel a little too easy. "I could never forget this, it seems so obvious now!". Doing this means your reviews are almost all easy and remind you of your learning experiences. The few things you forget are easily recalled and strengthened.

Joyful and fun review sessions are the most important thing. Maximising adherence (and therefore not giving up) is far more important than squeezing out a 1% more efficient SRS algorithm.

[+] ranting-moth|2 years ago|reply
> Spaced repetition is, by far, the most effective cognitive hack I’ve used.

I totally second that. Well, sleep and exercise do amazing things for you too, but if you need to memorize things, I don't know of a quicker method than spaced repetition.

Check out Leitner system: https://en.m.wikipedia.org/wiki/Leitner_system

Flashcards Deluxe supports Leitner, I don't know about Anki?

https://play.google.com/store/apps/details?id=com.orangeorap... (I have no relation to that app, I just find it incredibly useful).

[+] fwlr|2 years ago|reply
”Individual cards should be extremely brief, but your deck as a whole can be as repetitive as you want.”

Huh! I struggled with using Anki for pretty much this exact reason, always spent too much mental energy figuring out the “correct” number of cards for a given topic. But the author makes a good point here, if there’s too many cards on a certain topic you’ll just hit “I remember” on the repetitive cards and the algorithm will make them disappear for months - so there’s basically no cost to having “too many” flashcards!

[+] tiagod|2 years ago|reply
>A common failure mode (and I did this more than once, before I got the hang of it) is to use Anki for two weeks, then drop it, and pick it back up six months later only to find you have 600 cards due for review. This is not encouraging, and it defeats the point of spaced repetition, which is to review the cards on the intervals the algorithm chooses. >I don’t have much advice in this area, except that if you have persistent problems with conscientiousness, untreated ADHD etc. you should address that first.

This keeps happening to me, and I have somewhat treated (but severe) ADHD.

Does anyone have recommendations to make this easier? Either Anki settings, or using another app.

[+] ekkeke|2 years ago|reply
Big fan of spaced repitition, especially for language learning. Unfortunately I feel like it fares worse for topics that require more application instead of memorisation, like mathematics or electrical engineering. Would love know if there was some super effective way to learn these similar to spaced repitiion.

So far, the only thing that really works for me is solving lots of problems until I have the technique mastered, but even then after a while I'm prone to forget how to solve them. Perhaps there some way to combine the problem solving with the spaced repition? It seems like it would be far harder to make a deck for this and I don't think most flashcard software handles it very well.

[+] nomadpenguin|2 years ago|reply
I've been experimenting with "spaced free recall". So first, I'll read a section of a textbook. Then, I write down everything I can remember about it in a blank text file, organizing things in a way that makes sense to me. Next, look back at the section and compare to my recalled notes, filling in missing information and committing extra attention to missed spots. Repeat the process with increasing intervals between reviews.

From what I understand of the literature, free recall produces better learning compared to cued recall like flash cards. Part of the reason is that it forces you to organize information and associate it with existing knowledge.

Anecdotally, it's much easier to learn conceptual knowledge, and I don't really feel like my recall of specific facts has suffered compared to traditional SRS.

[+] oregoncurtis|2 years ago|reply
I actually used Anki cards to study LeetCode problems when preparing for interviews and it seemed to help. After doing a problem and solving it I created the card as such:

- Front of card is the entire LC problem statement

- Back is a bulleted list of the steps or key points (ie. first I notice this list is unsorted, so I would sort first, next I would do blah blah..)

- Back also contains the code solution that I might just glance through or look at a particular part of it.

[+] maphew|2 years ago|reply
Maybe try drawing the key points instead of text cards. Idea sparked by the below, which is awesome but requires someone else who already understands to create the learning material first.

"Each 5-minute video, or 'cartoon', is the equivalent of 50 minutes of a university-level computer graphics class. ... there was no statistically significant difference in learning effectiveness between [cartoons & lectures] as measured by exam, homework, and project scores. In other words, the cartoons were just as effective as traditional classrooms for teaching the material."

https://g5m.cs.washington.edu/

https://youtube.com/playlist?list=PLWfDJ5nla8UpwShx-lzLJqcp5...

[+] zetalyrae|2 years ago|reply
In my own experience using spaced repetition for math: math has both semantic and procedural knowledge. The procedural knowledge comes from doing problems and rewriting proofs. But the semantic knowledge is also important, and you can acquire and retain this through spaced repetition.

I was going to write some rules specifically about math but I might write those as a separate post because they got too long. I think I've benefited specially from memorizing the proofs of theorems, though refactoring proofs into multiple lemmas to make each proof small enough to fit in a flashcard is a tedious process.

[+] schneems|2 years ago|reply
> require more application instead of memorisation, like mathematics or electrical engineering

I’ve dreamed of having some app that mixes in bite sized learning lessons with otherwise “fun” internet (social media, news, etc.)

I could imagine it could give you a little tutorial and then ask you a quiz (to force application). If you get it wrong it keeps you at the same concept and explains it a different way next time, maybe asks if you want to revisit prereqs.

Even if you can’t memorize the answers, you can change your understanding and intuition.

[+] sn9|2 years ago|reply
You can use SRS to schedule the review of problems you've understood how to solve.

Front of card: where to find the problem (e.g., book, page number, problem number).

Back of card: where to find a solution (e.g., solution manual, page number, maybe a personal notebook with cleanly written solutions, etc.).

I initially tried writing up the problem and solution in Anki, but that was too much of a hassle and realistically I'm not gonna be reviewing problems without the book in front of me anyway.

[+] _gfwu|2 years ago|reply
General advice for spaced repetition is to make flashcards atomic i.e. as small as possible, as in the OP, but general advice for language learning is to always learn words in context instead of on it's own, for example in example sentences. Have you figured out a solution combining those two goals?
[+] onos|2 years ago|reply
Could you save “representative problems” to your cards? Eg a particular integral that uses a particular method etc.
[+] typon|2 years ago|reply
Is Duolingo basically spaced repetition for language learning?
[+] MisterPea|2 years ago|reply
Having tried spaced repetition methods for studying for swe interviews, I can concur that it is the most effective way for me learn.

It does require an intense amount of discipline though, so wonder how well it will work for me in execution for hobby learning.

[+] koofdoof|2 years ago|reply
Are there any good premade decks you could recommend? Or particular topics you found well suited to spaced repetition?
[+] ngai_aku|2 years ago|reply
What kind of cards did you make for interview prep? Reviewing algorithms?
[+] albert_e|2 years ago|reply
I read so many good things about spaced-repetition but havent had the discipline to stick with it and make it work for me ... I wil give it one more shot with flashcards on a exam prep I am about to embark on.

Is there any gentle kid-friendly introduction to this topic with a fun exercise that I can introduce my K-12 kids to so they might grow up with better tools than me?

[+] submeta|2 years ago|reply
A question to the experienced Anki users:

The recommendation is to learn before you memorize.

Many times I hack the infos (from an article) right into Anki. Now if it‘d want to review the infos in the Anki app, that totally destroys my stats.

What are some of you doing? Extract the info into some other tool, review the infos there and then quiz yourself in Aki?

[+] zetalyrae|2 years ago|reply
I wrote this. Ask me anything!
[+] s-video|2 years ago|reply
1. Have you read Andy Matuschak's guide to prompt writing? Your post reminded me of it. https://andymatuschak.org/prompts/

2. Do you have any sort of guide or principles for note-taking? I'm always debating whether or not it's worth taking notes, and when I do take notes I'm debating what the best way to do it is. (Hierarchal/bulleted information like in your post, or summarizing things in paragraphs, or what) A lot of times it's unclear to me what information is worth writing and it frustrates me.

[+] LVB|2 years ago|reply
Great article? I'm curious what your Anki settings are, if you've changed anything. I'm pretty new to the app, but when I hit things like "you'll need see this card in 5 years" I had to dive into settings and start tweaking stuff. I'm more concerned about definitely remembering it in say 12 months than having too high of a load. But there are many adjustments to achieve this, so any thoughts?
[+] bluechair|2 years ago|reply
Thanks for the write up.

Could you share with us how you’re applying this knowledge in your work?

[+] larsrc|2 years ago|reply
Nice write-up! Haven't tried SRS yes, I definitely see how it can work well for learning facts like you show. For fuzzier subjects like history or psychology, I've had some success with writing questions for myself that require more of an understanding of the subject than mere facts. (Never got to the repetition part, though.) I found that writing non-trivial questions also helped me understand the subject matter better. How would you work with such subjects?
[+] ivvve|2 years ago|reply
How would you reccomend I learn history with spaced repetition? I'm studying a detailed subject independently (I.e. not for an exam with a set curriculum) and I'm finding it hard to atomise the cards down bevond dates and names. I suppose I should start there first and then build more complex cards, but I'm not sure what the best approach for those is. Thanks for the detailed article!
[+] hermanschaaf|2 years ago|reply
Thanks for the great article, you've inspired me to take another shot at making a habit of learning through spaced repetition!

I had a question: how much time do you typically spend on this activity in a day? Do you have tips for how to adjust based on the time you have available?

[+] hyeomans|2 years ago|reply
Thanks for this. What topics have you learned/memorized with this technique?
[+] hendry|2 years ago|reply
I find writing cards / decks the challenge. IIRC the ones you download from Ankiweb are all such low quality!

Could you link to some good quality decks for inspiration please?

[+] untech|2 years ago|reply
Thanks for the article! I noticed a small issue: the “Powers of two” subsection didn’t render properly, probably due to a Markdown syntax error.
[+] cdelsolar|2 years ago|reply
I am a tournament Scrabble player and the state of the art for studying words is spaced repetition. You quiz on "alphagrams", like ABEISTT, and after a few times you just see BATISTE BISTATE. There are at least 80K words between 2 and 8 letters long though, so it does take many hundreds or thousands of hours to learn them all well. I have very poor studying discipline so I have my own methods of studying that don't use spaced repetition, instead I just study all the words periodically and focus on the harder ones. But I don't know them as well, and for the words I did spaced repetition on more than 15 years ago I can still recall those immediately.
[+] LVB|2 years ago|reply
This is interesting to me. Do you make many permutations and use those as cards?
[+] endisneigh|2 years ago|reply
spaced repetition is dying for some amazing UX. it doesn't even really make sense for you to make flash cards. ideally they could be contextually created based on what you're viewing, e.g. you read a wikipedia article, it infers what you're reading from scroll position, takes content, makes cards, presents later, etc.
[+] _dain_|2 years ago|reply
>it doesn't even really make sense for you to make flash cards. ideally they could be contextually created based on what you're viewing

It does matter. You shouldn't train on a flashcard until you have learned the idea. The computer doesn't know if you actually learned what you read. Making the flashcard based on your own understanding is an important part of encoding the memory -- it's an active process, rather than passive.

It's well established in communities that use Anki a lot (like language learners) that someone else's pre-made decks aren't as effective as making your own. The exceptions are either small and simple (e.g. NATO phonetic alphabet), or had a lot of thought put into them with community feedback, like the ones medical students use.

[+] n8henrie|2 years ago|reply
Used SRS via Mnemosyne (and later Repetitions.app) heavily in studying for all of my US medical board exams. The effort to payoff ratio seemed very satisfactory.

For pre-clinical rotations, a few nerdy peers and I collaborated on a shared deck of a couple thousand slides -- many of them pathology images -- synced via Dropbox.

For the Step exams, I mostly used practice test questions. Any question I missed prompted me to read up on the topic to determine what piece of knowledge would have helped me come to the right answer, and then figure out how to make a decent card for that principle. Every morning I would start by reviewing all my SRS cards, then do a few hours of practice tests. It was really nice being able to be able to take a core component m of my study material on the road by just bringing my phone! A few of the practice question apps had protections in place to prevent copying text (copy and paste saved a fair bit of time, even if there was also a lot of rewriting) -- but figuring out workarounds like running in a VM or screenshotting from the iOS app was never too hard, and I would just queue up the screenshots to batch process toward the end of the day.

I used a similar technique to pass the tech, general, and extra exams for amateur radio with near perfect scores (the verbatim questions and answers are freely available -- I did try to learn the concepts behind most questions, though a few were admittedly memorization without understanding). Unfortunately my small town is too rural to have a local club, and I have too many hobbies to shell out $1k for a HF rig, so I have yet to make a single QSO. I'll eventually find time to put together the QRP CW kit I got for my birthday :)

[+] robertbob|2 years ago|reply
Does anyone have any insight into deciding what information should be memorised, and what information it is sufficient to simply store in a searchable digital knowledge base for rapid retrieval when needed?

Takes a lot of effort to commit my notes from a book into my head, but a tiny amount of resources to store them on my computer.

[+] _dain_|2 years ago|reply
Gwern's classic monograph[1] addresses this:

>The most difficult task, beyond that of just persisting until the benefits become clear, is deciding what’s valuable enough to add in. In a 3 year period, one can expect to spend “30–40 seconds” on any given item. The long run theoretical predictions are a little hairier. Given a single item, the formula for daily time spent on it is Time = 1⁄500 × nthYear−1.5 + 1⁄30000. During our 20th year, we would spend t = 1⁄500 × 20−1.5 + 1⁄3000, or 3.557e-4 minutes a day. This is the average daily time, so to recover the annual time spent, we simply multiply by 365. Suppose we were interested in how much time a flashcard would cost us over 20 years. The average daily time changes every year (the graph looks like an exponential decay, remember), so we have to run the formula for each year and sum them all; in Haskell:

    sum $ map (\year -> ((1/500 * year**(-(1.5))) + 1/30000) * 365.25) [1..20]
    # 1.8291
>Which evaluates to 1.8 minutes. (This may seem too small, but one doesn’t spend much time in the first year and the time drops off quickly55.) Anki user muflax’s statistics put his per-card time at 71s, for example. But maybe Piotr Woźniak was being optimistic or we’re bad at writing flashcards, so we’ll double it to 5 minutes. That’s our key rule of thumb that lets us decide what to learn and what to forget: if, over your lifetime, you will spend more than 5 minutes looking something up or will lose more than 5 minutes as a result of not knowing something, then it’s worthwhile to memorize it with spaced repetition. 5 minutes is the line that divides trivia from useful data.56

[1] https://gwern.net/spaced-repetition

[+] outlace|2 years ago|reply
Ask yourself if the cost of having to look it up at a potentially inconvenient time (e.g. in the middle of a busy work day) is greater than the cost of memorizing it during scheduled less busy times (e.g. doing flashcards while eating dinner or during a bus commute).
[+] rjh29|2 years ago|reply
Language is really the ideal use case, you cannot stop and look up words every few seconds when talking to someone. And doing it while reading or watching TV spoils the enjoyment somewhat. You need to frontload a ton of data into your mind and flashcards are the best way to do it.

Another good use case is country flags, because you can't easily look those up (other than pulling up an image of ALL the country flags)

[+] dalmo3|2 years ago|reply
Tried Anki for a few months for learning programming concepts and internalizing ideas from books. I created all flashcards based on my own notes, for stuff I understood, did everything by the book.

After a while I realised I was visually learning the flashcards, i.e. when reading the prompt side I would see a photographic memory of the answer and get it right, but not necessarily think about the meaning of the contents. After a 15-20 min session I would remember very little of what I had just studied. Then I quit.

It reminded me of how trained chess players visualise the game as patterns instead of a collection of pieces.

Anyone with a similar experience?

[+] michaelcampbell|2 years ago|reply
> Rule: Understand First

Good in theory. Sometimes time crunches don't allow that, and to be honest after having used Anki for many years there are quite a lot of times where I rote-memorized a card and as I learned more about the subject the understanding came later.

Usually this is when there are several cards hitting different aspects of a given topic; as I gain understanding in one area, it comes in another.

So this is very much a "nice to have" rule of thumb.

For me, people vary. But I'd rather have the card and come to understanding than not have it.

[+] yanis_t|2 years ago|reply
Spaced repetition looks very promising to me. I've been a long-time Anki [1] user, and it allowed me to learn Czech much faster.

Recently I launched a website [2] where I try to blend a markdown-based knowledge manager with spaced repetition. It's not an easy task and there's a long way to go, but after adding and maintaining > 400 cards, I already

[1] https://apps.ankiweb.net/ [2] https://retaind.io/

[+] throwaway675309|2 years ago|reply
"This is the most important thing. By far the worst failure mode is to put too much in a flashcard."

He's wrong on this one. The problem with making extremely atomic flashcards is that many times it's impossible to prevent slight amounts of overlap between the cards. What this means is that answering card A might contain information that allows you to answer some subsequent card B.

As a result you end up effectively getting a hint for some dependent cards, as opposed to if you had to recall all that information from scratch.