Is "backward learning" well-documented?

Question

I was initially in the field of pure math, but I successfully transitioned to being a researcher of machine learning with much less effort than I expected. My strategy was, instead of learning CS and ML from scratch, to read many interesting recent ML papers and, whenever I encounter with unfamiliar concepts or papers that are cited and seem to contain the information necessary for my understanding, I tried to find the relevant documents online to understand it.

While there is a vast amount of knowledge accumulated in ML, this unstructured way of pursuit strategy of knowledge turned out to be sufficient for me to cover the important information to contribute to the current state of research in a way I like, probably much more efficiently than by learning in a more structured way. I call this "backward learning strategy."

I think this may be similar to how researchers learn when they transition from an area to an area after PhD. Of course, feasibility of this learning must be dependent on the field. Even for pure mathematicians, one cannot understand an arithmetic geometry paper without at least having learned AG in textbook-level. However, it was not difficult for me to understand a paper on a certain kidney disease after reading some relevant papers, without much knowledge in medicine.

Unfortunately, I've not been able to find relevant information about this strategy online. I want some paper references for this learning.

Edit: Asking for anectodes may be not suitable for this site, so I changed it to paper references instead.

Answer 1

Documentation I don't have, but anecdotes, sure.

Actually, I'd be surprised if what you describe as "backward learning" isn't pretty ubiquitous in mathematics for new researchers, say doctoral students. Another name for it might be "just in time" learning, or "just enough".

Suppose a doctoral student is trying to get going on some problem suggested by an advisor. The advisor gives the students a paper or three that might be promising for extension. Those papers might have been authored by the advisor.

But the advisor had a certain history of learning and a certain base of understanding when the papers were written that the student is very unlikely to match. Even though they have taken advanced courses in the general area, this is different and very specific. So, to understand that paper(s), the student looks, also, at the papers referenced there, or even just their abstracts. If that is enough to provide a base, then fine, but one level is not always sufficient. I'd suggest several levels is usually required. My own doctoral research required me to work through a forty year development of the topic, just to be able to state a problem.

Forty years might not be much in a sub-field that isn't very active, of course. But the student needs enough of that backward look until most of the things stated in the papers are actually captured in the typical course. Or, if not captured, derivable from them in a straightforward manner.

However, once you become thoroughly familiar with some highly specialized topic, you have gained a lot of knowledge that isn't in text books and probably isn't worth putting in text books due to its specialized nature (and appeal). So, such a person can continue to move forward without so much backing up as was first required.

And math papers are written for other math experts, not for school kids. The jumps between steps can be quite large. So, a working mathematician will fairly often ask "where did that come from" while reading a paper. They may just assume the jump is valid, but if not, then it requires a deeper look and perhaps going backward to specialized sources (i.e. the referenced papers).

So, pretty common in math, I think. With no basis for saying so, I'd guess it is similar in other technical fields as well. The basics that you find in text books can be pretty far from the research frontier in math and maybe other fields as well.

Answer 2

Yes, it's called "learning." Mathematics is highly atypical because we currently structure mathematical training as a pipeline as though this were the best or only way to go about it. (I would argue that it's not even a good way to go about it, and it certainly isn't true that it's necessary to have plodded through Hartshorne in order to begin reading papers in algebraic geometry.)

The only research on learning in the manner you describe is likely to be some learning-styles research on "holistic" vs. "sequential" learners. There seems like there might be some differences among students' comfort with these two approaches, but like all learning-styles research this needs to be taken with a grain of salt.

On the teaching side, however, it has become very voguish lately to use "backwards design" when creating courses, where an instructor starts out with a set of learning objectives and then, well, works backwards. I don't personally know of any examples of college- or grad-level mathematics courses doing this right. I know I have seen studies done on the efficacy of this approach in other fields, including STEM fields, but I don't have any references to point you to at my fingertips. Probably the best-studied examples are in foreign language instruction, where it has been well-established that an immersive approach is vastly superior to the old-fashioned vocab-and-grammar approach.

Answer 3

To give an analogy from your own current field of ML/DL, this is the essence of transfer learning. You train a model (~learn particular background and acquire skills) for a certain task (~a certain field). You then use this learning to solve a related, but different task (~a related, possibly applied, but not wholly different field) without re-training your model (~ learning the background/required skills from scratch).

It isn't surprising that 'learning' is common to the human and the machine, because ML is essentially mimicking human learning patterns.

If you are interested in a more neuroscience/sociological academic understanding of this, I'd refer you to researchers like VS Ramachandran. He calls it 'imitation learning' and relates it to the structure of neurons in the brain (mirror neurons to be precise) and suggests that the primary way of learning is to see something, imitate it and emulate it. The 'emulate' part is essentially what you are doing by applying skills developed somewhere to a different task/field.