Is it common for research disciplines to have a large gap between theory and practice?

Question

TL;DR: Is it normal that the theory sub-community and the applied sub-community of a single research field to have different goals and different standards for peer review - to the extent that the research field itself suffers?

I work in a subfield of computer networking. During the last two years I've been in a quite unusual position (for the subfield) as a person whose job was to bring some mostly-theoretical results of big research project to practice. Personally I've greatly enjoyed this role; however, the job also brought a lot of frustrations. It seemed that neither the theory people nor the applied people are particularly interested in a work that bridges these two sub-communities.

In general, there is a very little overlap between what the theory subcommunity cares about and what the applied subcommunity cares about. To give one example, the assumptions made by the people in the theory camp often are wildly wrong. In my opinion, that renders most of their results quite useless; however, they do not spend a lot of effort trying to recognize and correct these invalid assumptions, as it's possible to publish their results anyway, especially in sub-premier conferences. On the other hand, I feel that applied conferences in the field are resistant to novel ideas; a paper that simply demonstrates how to achieve state-of-art results a different approach is not recognized as novel enough. In my opinion, the typical top-level applied paper is therefore quite boring; due to the peer-review standards, they must focus on the engineering work and on applying known ideas to existing problems, rather than on novel ideas and novel problems.

To give some other anecdotal examples:

• after I reported that a specific algorithm (developed by one of my theory-oriented collaborators) does not seem to give good results on a real dataset, I was suggested that "maybe we should evaluate it on synthetically generated data instead";
• in a different conversation, a senior applied researcher commented that "we skip over any mathematical parts when reviewing a paper".

To me it seems very likely that the progress in the subfield was faster if both communities were working closer together. The objective of science, after all, is not to simply produce more papers; it is to discover more knowledge.

Are other disciplines facing a similar problem, and how do they deal with it?

Answer 1

Significant splits between theory and practice, such as you are describing, are not at all unusual. What many people fail to recognize is that research spanning between theory and practice is effectively interdisciplinary research!

Moreover, what you call "theory" others may call "applied" and vice versa. I have noticed that whether I feel like the "theory person" or the "applied person" in the room depends strongly on which community I am participating in at the time. In fact, what you are experiencing is only a small part of the theory/application spectrum. Many funding organizations attempt to quantify the theory/application spectrum with some sort of technology readiness level rubric, and you are almost certainly experiencing only TRL 1-3 communities, as that is where almost all academic research lives.

Why this matters, and why there is a gap in communication between communities, is that the problems encountered and solutions required at different TRLs are often fundamentally different. At one end of the spectrum is: "Might this be plausible?" and at the other end is "How much does it cost to get one shipped to me by Tuesday?" All of the questions on this spectrum can be valuable, even if some seem "farfetched" and others "boring" to you based on where you personally happen to be sitting on the spectrum right now.

Spanning different levels is difficult, and to do so you will need to figure out how to speak the languages of the different communities. If you are able to do so, however, it can be of great value both to your career and to the larger communities in which you live.

Answer 2

This is so common that there are entire subfields dedicated to bringing theory (and even some applied research) out of the lab and into practice, such as "translational medicine."

In my own field (library and information science; I'm a practitioner and educator, not a researcher) there is considerable friction between the research and practice arms, each finding reasons to despise the other. I don't find that particularly healthy, but I've also seen more than one giant unnecessary mess created by researchers sailing in to Fix All The Things without a good enough grounding in the practicalities. (OAI-PMH, enough said.)

If there's a fix for this, I don't know it; most of the attempted fixes I've seen (like translational medicine) are pretty kludgy.

Answer 3

Almost every field is "theory" for some people and "practice" for others. It may be more useful to talk about interdisciplinary vs. intradisciplinary work instead of theory vs. practice.

The "normal" way to do research is intradisciplinary. You work on ideas from your field, produce new results, and publish them to other people in your field. The more established the field is, the easier it generally is to evaluate the quality of your work. This is obviously a good thing when applying for jobs or funding.

In interdisciplinary research, you take ideas from a theoretical field, produce results in an intermediate field, and publish them to people in an applied field. This can be hard and frustrating, because you need to communicate with two incomprehensible alien cultures. The upside is that even minor results can have significant impact, because true interdisciplinary work is too frustrating for most people. On the downside, the significance of your work can be hard to evaluate, because everything is so incomprehensible for outsiders.

I think there may be a natural balance between the two modes. When the intermediate field becomes more established, the incentives start favoring people working within the field. If the field grows too insular, the rewards for interdisciplinary work grow greater, attracting people who can accept the higher risks associated with it.

Answer 4

Practice is engineering and building systems that actually work. Theory is science and about understanding the environment and why systems work.

There are both social and natural reasons for this. Reality is too messy to be directly analyzed mathematically without lots of simplifications. Figuring out what is essential and what is not is a difficult task. There is mismatch between theoretical models and reality. When is a theoretical model applicable in practice? What assumptions can be made about reality to simplify it and make it amenable to mathematical analysis? Even if we get the models right, our understanding often lags behind what people want to do in practice and seem to be able to do even without understanding why it works when it works and when it fails. Do you want a system that seems to work in practice most of the time even if you don't understand why it is working and when it can fail, or do you want a system that you can mathematically understand and prove results about? Sometimes you cannot have both!

There are also social constraints. Theoreticians typically do not have the skills to implement their systems and test them in practice. Practitioners typically do not have the skill to mathematically model their problem and prove statements about them. Both typically lack a knowledge of the language of the other which makes a discussion between them a very frustrating experience.

The issue is not simply about individuals. There is little positive feedback for any effort to improve and cross the gap. The exceptions that succeed succeed magnificently but that seldom happens. Most of the time you spend a lot of time on something which your reviewers/peers/managers (people who evaluate your work) do not care about or value. Systematically people learn to avoid those efforts and we end up with people who have a very narrow expertise and no craving to cross the gap.

What can be done to improve the situation? There can be more systematic efforts from both communities to value and give actual positive feedback to people who try to close the gap. System conferences can require mathematical statement of the problem being solved and mathematical description of the system it is solved in at the least. It is frustrating for theoreticians to try to figure out what is the exact problem being solved by reading engineering documents and articles. Theory conferences can require implementation of any algorithm developed and a comparative benchmark of the algorithm on a standard data set against other algorithms as well as a description of when the assumptions about the model holds with relevant examples from practice. The communities can push for every person to have enough an understanding of the language of the other community and to be able to hold a constructive beneficial discussion.

Answer 5

Sometimes it happens that theory is advanced using methods that are not of common knowledge among empirical researchers. McElreath and Boyd write in their book:

Imagine a field in which nearly all important theory is written in Latin, but most researchers can barely read Latin and certainly cannot speak it. Everyone cites the Latin papers, and a few even struggle through them for implications. However, most rely upon a tiny cabal of fluent Latin-speakers to develop and vet important theory.

Replace Latin by Maths and you will have a fairly good description of some social sciences and evolutionary biology. For example, I have lost the count of how many Sociology empirical studies about Becker's new home economics which try to test hypotheses attributed to him, but who seem to be based on a misunderstanding of the mathematical models in his writings. This divide between empirical and theoretical work is caused by a lack of basic mathematical knowledge by empirical sociologists.

Answer 6

Are other disciplines facing a similar problem, and how do they deal with it?

There is a recognized chasm between research and practice within information systems research [1].

Here is a quick summary of some of the ways that scholars are trying to deal with it (or at least the solutions they are proposing):

1. Joint University-Industry Appointments where universities appoint expert practitioners to paid part-time university posts [2]
2. Focus on using better forms of dissemination than research journals, for instance websites with actionable content [2]
3. More systematic reviews of literature in the same way as the Cochrane Collaboration does for medical research [2]
4. Ensuring that editorial boards and program committees have equal representation from academics and practitioners [2]
5. More action research; research where practitioners and researchers work together to test and refine principles [2]
6. Applicability checks - e.g., academics should assess whether research relevant to practice to determine if they should write it. They do this using focus groups with practitioners [1]
7. IS researchers should develop closer links to business and technology, for instance; (i) conducting sabbaticals in corporations, (ii) having industry-based projects for students (iii) encouraging internships for junior faculty, (iv) doing business consulting, and; (v) building partnerships and alliances with business groups [1]
8. Universities should improve IS faculty levels of practical knowledge by (i) re-evaluating tenure criteria, (ii) realigning faculty reward processes, and (ii) changing standards for evaluation
9. IS researchers should involve practitioners directly in research, (ii) actively seek problems from practitioners [1] (iii) run more surveys about practitioner issues to develop a research agenda that aligns with practice [1]
10. Universities should revise Ph.D. Program Requirements to (i) require at least a minimal level of business experience and managerial involvement as a requirement for admission or as a supplemental part of doctoral programs, (ii) adopt a strategy where interdisciplinary dissertations and studies of actual business practices are viewed positively within the dissertation process (iii) develop business experience of students in doctoral programs [1].
11. Universities should form partnerships with professional and discipline-based organisations: "Individual schools, and the AACSB, should encourage disciplinary-based academic organizations to include practitioners in their annual meetings to help define new issues of which the membership would be aware" [1]
12. Researchers should produce short and concise research reports (ii) use traditional practice reports,management briefs,white papers, and the Internet to disseminate research where possible, (iii) produce special issues on topics of interest to IS managers and hold themed conferences with both academics and practitioners, (iv) publish online to shorten research to publication times and (v) write practice orientated books [1]

References:

[1] Rosemann, M. and I. Vessey (2008). "Toward improving the relevance of information systems research to practice: The role of applicability checks." MIS Quarterly 32(1): 7-22.

[2] Moody, D. and A. Buist (1999). Moody, Daniel L. "Building links between IS research and professional practice: improving the relevance and impact of IS research." Proceedings of the twenty first international conference on Information systems. Association for Information Systems, 2000.