The intended audience is people who are knowledgeable about spaced repetition, and such people typically know what generalization means in the context of learning.
TLDR: I learned from those kinds of resources myself, and while I came a long way, for the amount of effort I put into learning, I could have gone a lot further if my time were used more efficiently. That's the problem that Math Academy solves.
Just a heads up that there's an explanation for this; I responded to the original comment (https://news.ycombinator.com/item?id=41556644) but I'll also paste here in case anyone misses it:
1. New tasks are selected only as you complete existing tasks (so if you come back after 2 weeks, you need to complete some existing tasks from 2 weeks ago to get new tasks selected based on your knowledge profile right now).
2. We are often able to implicitly knock out due reviews with new lessons. We're not just doing plain vanilla spaced repetition. We're doing a highly efficient novel version of it that we call Fractional Implicit Repetition (FIRe). I have a writeup on this that gained some traction on HN recently: https://news.ycombinator.com/item?id=40954571
Regarding spaced repetition, keep in mind the following:
1. New tasks are selected only as you complete existing tasks (so if you come back after 2 weeks, you need to complete some existing tasks from 2 weeks ago to get new tasks selected based on your knowledge profile right now).
2. We are often able to implicitly knock out due reviews with new lessons. We're not just doing plain vanilla spaced repetition. We're doing a highly efficient novel version of it that we call Fractional Implicit Repetition (FIRe). I have a writeup on this that gained some traction on HN recently: https://news.ycombinator.com/item?id=40954571
Yes, engaging in flashcard-based spaced repetition would qualify as valid retrieval practice. But if you're in a skill hierarchy like math, then you would need to make sure you're not only recalling isolated bits of information (facts, formulas, theorems, etc) but actually practicing pulling together this information to solve problems. Like this: https://news.ycombinator.com/item?id=40954571
> For papers with algorithms in them, it helps me to write out my own copy of the algorithm. It's a similar mental process as if I were to code an algorithm from a paper -- that's when I really understand it.
Yeah, that makes sense. It sounds like you're talking about "listening on paper" (I made a distinction between "note-taking" and "listening on paper" at the end of the post).
In mathematics, deliberate practice amounts to working out a high volume of practice problems at the edge of your ability, with proper scaffolding and rapid feedback, and continually pushing forward into more advanced problems.
You don't have to be blazing fast but you do need to have sufficiently tight feedback loops so that you can get through a high volume of action-feedback-adjustment cycles. It's the compounding of a massive number of those cycles that produces a massive gain in performance.
Also, in order to make mathematical discoveries -- or more generally, in order to grapple with advanced content at all -- you need to develop a level of automaticity on your foundational skills.
"Beyond the edge of one's capabilities" means that you're working on things outside of your current repertoire. This could mean any of a number of things, e.g.:
1) Maybe you can do it with scaffolding, but you are unable to do it without scaffolding.
For instance, a musician might not be capable of playing a difficult section of a musical piece at full speed. So they might practice it while playing slowly (a type of scaffolding), and then gradually ramp up the speed while maintaining accuracy.
2) Maybe you can do it sometimes, but not consistently/accurately.
For instance, a gymnast might not be capable of landing a particular flip consistently with proper form. But maybe they can land it 50% of the time with shaky form. So they might practice improving their consistency and form on this skill.
---
Working on things outside of one's repertoire, is a core aspect of deliberate practice. Non-experts are often misled to practice within their level of comfort. This tends to be more effortful and less enjoyable, which can mislead non-experts to practice within their level of comfort.
For instance, Coughlan et al. (2014) observed this as a factor differentiating intermediate and expert Gaeilic football players:
"Expert and intermediate level Gaelic football players executed two types of kicks during an acquisition phase and pre-, post-, and retention tests. During acquisition, participants self-selected how they practiced and rated the characteristics of deliberate practice for effort and enjoyment.
The expert group predominantly practiced the skill they were weaker at and improved its performance across pre-, post- and retention tests. Participants in the expert group also rated their practice as more effortful and less enjoyable compared to those in the intermediate group.
In contrast, participants in the intermediate group predominantly practiced the skill they were stronger at and improved their performance from pretest to posttest but not on the retention test."
---
The idea of practicing outside of one's repertoire can be generalized to the idea of engaging in a cycle of strain and adaptation. This is done in, e.g., Ericsson (2006). Here's a snippet:
"When the human body is put under exceptional strain, a range of dormant genes in the DNA are expressed and extraordinary physiological processes are activated. Over time the cells of the body, including the brain (see Hill & Schneider, Chapter 37) will reorganize in response to the induced metabolic demands of the activity by, for example, increases in the number of capillaries supplying blood to muscles and changes in metabolism of the muscle fibers themselves.
These adaptations will eventually allow the individual to execute the given level of activity without greatly straining the physiological systems. To gain further beneficial increases in adaptation, the athletes need to increase or change their weekly training activities to induce new and perhaps different types of strain on the key physiological systems."
---
In general, in the phrase "deliberate practice," the word "deliberate" is not just an adjective. "Deliberate practice" has a very specific meaning in the research literature.
The way you describe it -- "Practice is certainly still deliberate. It's planned. Every workout has a purpose." -- is not as strict as the meaning in the research literature.
For something closer to a proper definition, I'll quote (Ericsson, 2006):
"The core assumption of deliberate practice (Ericsson, 1996, 2002, 2004; Ericsson et al., 1993) is that expert performance is acquired gradually and that effective improvement of performance requires the opportunity to find suitable training tasks that the performer can master sequentially -- typically the design of training tasks and monitoring of the attained performance is done by a teacher or a coach.
Deliberate practice presents performers with tasks that are initially outside their current realm of reliable performance, yet can be mastered within hours of practice by concentrating on critical aspects and by gradually refining performance through repetitions after feedback.
Hence, the requirement for concentration sets deliberate practice apart from both mindless, routine performance and playful engagement, as the latter two types of activities would, if anything, merely strengthen the current mediating cognitive mechanisms rather than modify them to allow increases in the level of performance."
---
I don't know much about serious running, but based on how things work in other domains: if "performance-improving adjustments on every single repetition" is incomprehensible at the level you're looking, then it's an indication you need to zoom out a bit.
The same confusion can happen in, e.g., deliberate practice in math, if you zoom in too much. When a student solves a math problem, do we really expect every single pen stroke to involve feedback and improvement? No. You have to zoom out to the level of the problem.
Correct me if I'm wrong, but I would expect that in running, the appropriate level to view these deliberate practice cycles is not the level of a single step, but rather, a cohesive group of a taxing "deliberate practice" runs and easier "recovery" runs. At this level, it looks more like that cycle of strain/adaptation that is characteristic of deliberate practice.
(And that level seems to align with what's discussed in the literature -- for instance, I was just skimming Casado et al, 2020, Deliberate Practice in Training Differentiates the Best Kenyan and Spanish Long-Distance Runners, which mentioned that "systematic training ... included high-intensity training sessions considered deliberate practice (DP) and easy runs.")
---
References
Casado, A., Hanley, B., and Ruiz-Pérez, L.M. (2020). Deliberate Practice in Training Differentiates the Best Kenyan and Spanish Long-Distance Runners. European Journal of Sport Science, 20 (7). pp. 887-895.
Coughlan, E. K., Williams, A. M., McRobert, A. P., & Ford, P. R. (2014). How experts practice: A novel test of deliberate practice theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 40(2), 449.
Ericsson, K. A. (2006). The influence of experience and deliberate practice on the development of superior expert performance. The Cambridge handbook of expertise and expert performance, 38(685-705), 2-2.
As detailed in the article, my conclusion of there being a limit does not rest on the assumption that higher math is intrinsically incomprehensible to a subset of people (though, unrelatedly, I would expect that to be true in some cases).
In the article, the key underlying assumption is that the further you go in math, the more energy it requires to learn the next level up -- and everyone's "energy vs level of abstraction" curve is shifted based on their cognitive ability and degree of motivation/interest.
Here is a quote from the article that gets at the main argument:
"As Hofstadter describes, the abstraction ceiling is not a “hard” threshold, a level at which one is suddenly incapable of learning math, but rather a “soft” threshold, a level at which the amount of time and effort required to learn math begins to skyrocket until learning more advanced math is effectively no longer a productive use of one’s time. That level is different for everyone. For Hofstadter, it was graduate-level math; for another person, it might be earlier or later (but almost certainly earlier)."
Totally. It's a vicious cycle. Once you get knocked off course, you fall into this current that's pulling you further off course. And the further off course you go, the stronger that current is.
Thanks! Yeah, I guess I should probably link to some of those later posts about active learning and deliberate practice in the article. If you want to read more about that part you liked, here's the main one I'd follow up with: https://www.justinmath.com/deliberate-practice-the-most-effe...
Agree, it depends highly on goals. Using off-the-shelf ML/AI models (to make great software) requires far less background knowledge than implementing new models being introduced in papers, which in turn requires far less background knowledge than producing new models that improve upon the state-of-the-art.
Thanks for the kind words about Math Academy! It's true that we focus on students who are trying to acquire math skills to the highest degree possible -- we teach math as if we were training a professional athlete or musician. We maximize learning efficiency in the sense that we minimize the amount of work required to learn math to the fullest extent.
I realize that there are many learners who only want to devote an hour or two per month, but, at least right now, such learners would be better served elsewhere. It's a totally different optimization problem -- maximize surface-level coverage subject to some fixed, miniscule amount of work -- and as a result it would require different different curriculum and possibly different training techniques (or at least, differently calibrated techniques).
But it's definitely an idea to think about in the future. :)