Chris Olah coins the term “research debt” to discuss a bundle of related destructive phenomena in research communities:
- Poor Exposition – Often, there is no good explanation of important ideas and one has to struggle to understand them. This problem is so pervasive that we take it for granted and don’t appreciate how much better things could be.
- Undigested Ideas – Most ideas start off rough and hard to understand. They become radically easier as we polish them, developing the right analogies, language, and ways of thinking.
- Bad abstractions and notation – Abstractions and notation are the user interface of research, shaping how we think and communicate. Unfortunately, we often get stuck with the first formalisms to develop even when they’re bad. For example, an object with extra electrons is negative, and pi is wrong.
- Noise – Being a researcher is like standing in the middle of a construction site. Countless papers scream for your attention and there’s no easy way to filter or summarize them. We think noise is the main way experts experience research debt.
Shout it from the rooftops (my emphasis):
It’s worth being clear that research debt isn’t just about ideas not being explained well. It’s a lack of digesting ideas – or, at least, a lack of the public version of ideas being digested. It’s a communal messiness of thought.
Developing good abstractions, notations, visualizations, and so forth, is improving the user interfaces for ideas. This helps both with understanding ideas for the first time and with thinking clearly about them. Conversely, if we can’t explain an idea well, that’s often a sign that we don’t understand it as well as we could…
Distillation is also hard. It’s tempting to think of explaining an idea as just putting a layer of polish on it, but good explanations often involve transforming the idea. This kind of refinement of an idea can take just as much effort and deep understanding as the initial discovery.
This leaves us with no easy way out. We can’t solve research debt by having one person write a textbook: their energy is spread too thin to polish every idea from scratch. We can’t outsource distillation to less skilled non-experts: refining and explaining ideas requires creativity and deep understanding, just as much as novel research.
Here’s footnote 6 speculating on causes:
There are a lot of perverse incentives that push against explaining things well, sharing data, and so forth. This is especially true when the work you are doing isn’t that interesting or isn’t reproducible and you want to obscure that. Or if you have a lot of competitors and don’t want them to catch up.
However, our experience is that most good researchers don’t seem that motivated by these kind of factors. Instead, the main issue is that it isn’t worthwhile for them to divert energy from pursuing results to distill things. Perhaps things are different in other fields, or I’m not cynical enough.
Unfortunately, but not surprisingly, Olah doesn’t have a great solution. He would like to see more interactive visual explanations. I’d like to see a “serious” wikipedia for physics, somewhat in the vein of the Review of Particle Physics.
Edit: Good HN discussion with the author.
