One of the papers I stumbled upon when reading about machine learning in science
last week was
this fascinating piece that looks at the role of
surprise in scientific advances. The authors’ hypothesis is that much scientific progress comes from surprising combinations of ideas - or combining ideas from surprising places. They formalise and test this using a huge corpus of journal articles in medicine and physics, as well as recent patents.
The results are striking. Most “ordinary” scientific work combines ideas that are “nearby” in terms of time and intellectual distance; that is, they are combinations of ideas from within a discipline and in recent years. But this is not true of the most highly cited - and presumably most important - work: these have much more surprising (i.e. statistically unexpected) combinations of both content and contexts.
One way this happens is when authors stray from their immediate field and apply ideas across disciplines. The papers with the most “surprising” authors - i.e. those whose academic backgrounds are furthest from the journal in which they’re publishing - are most likely to be hits. This could be a statistical artefact (to get published in a journal outside your field, your work has to be outstanding, presumably), but nevertheless seems important. How can we encourage scientists to wander from well-trodden paths? And how can we help them find new areas where their expertise might shine a line?