Reevaluating Scientific Methodology: Navigating Continuity in Theoretical Physics
Paragraph 1
One reason for the relative sterility of theoretical physics over the last few decades may well be precisely that the wrong philosophy of science is held dear today by many physicists. Popper and Kuhn, popular among theoretical physicists, have shed light on important aspects of the way good science works, but their picture of science is incomplete, and I suspect that, taken prescriptively and uncritically, their insights have ended up misleading research.
Paragraph 2
[This] has given rise to disastrous methodological confusion: the idea that past knowledge is irrelevant when searching for new theories, that all unproven ideas are equally interesting and all unmeasured effects are equally likely to occur, and that the work of a theoretician consists in pulling arbitrary possibilities out of the blue and developing them, since anything that has not yet been falsified might in fact be right.
Paragraph 3
This is the current “why not?” ideology: any new idea deserves to be studied, just because it has not yet been falsified; any idea is equally probable, because a step further ahead on the knowledge trail there may be a [theory] that was not predictable on the basis of past knowledge; any experiment is equally interesting, provided it tests something as yet untested.
Paragraph 4
I think that this methodological philosophy has given rise to much useless theoretical work in physics and many useless experimental investments. Arbitrary jumps in the unbounded space of possibilities have never been an effective way to do science. The reason is twofold: first, there are too many possibilities, and the probability of stumbling on a good one by pure chance is negligible; more importantly, nature always surprises us and we, limited critters, are far less creative and imaginative than we may think. When we proudly consider ourselves to be “speculating widely,” we are mostly playing out rearrangements of old tunes: true novelty that works is not something we can just find by guesswork.
Paragraph 5
…The radical conceptual shifts and the most unconventional ideas that have actually worked have indeed been always historically motivated, almost forced, either by the overwhelming weight of new data, or by a well-informed analysis of the internal contradictions within existing, successful theories. Science works through continuity, not discontinuity.
Paragraph 6
It is not fishing out un-falsified theories, and testing them, that brings results. Rather, it is a sophisticated use of induction, building upon a vast and ever-growing accumulation of empirical and theoretical knowledge, that provides the hints we need to move ahead. It is by focusing on empirically successful insights that we move ahead. The scientific enterprise is founded on degrees of credibility, which are constantly updated on the basis of new data or new theoretical developments…I think this lesson is missed by much contemporary theoretical physics, where plenty of research directions are too quick to discard what we have already found out about Nature.
