It pays to keep a little craziness

This week's New Scientist has an editorial entitled It pays to keep a little craziness, which makes a case for supporting a small number of maverick scientists. The entire editorial is reproduced here:

Time was when all scientists were outsiders. Self-funded or backed by a rich benefactor, they pursued their often wild ideas in home-built labs with no one to answer to but themselves. From Nicolaus Copernicus to Charles Darwin, they were so successful that it's hard to imagine what modern science would be like without them.

Their isolated, largely unaccountable ways now seem the antithesis of modern science, with consensus and peer review at its very heart. Yet the "outsider" tradition persists. Think of Alfred Wegener, the father of plate tectonics and, more controversially, of Gaia theorist james Lovelock. Both pursued their theories in the face of strong opposition from their peers.

Such mavericks can be crucial to progress, but are they a dying breed? Beyond young disciplines such as neurobiology, where the territory is largely uncharted, or esoteric areas like quantum theory, where it's hard to prove anything, the consensual nature of science can make it hard for lone voices to thrive.

This may be inevitable. Peer review is inherently conservative, and increasingly only proposals that fit the research framework get funding. The sheer number of ideas in circulation means we need tough, sometimes crude ways of sorting geniuses from crackpots.

The principle that new ideas should be verified and reinforced by an intellectual community is one of the pillars of scientific endeavour, but it comes at a cost. We shouldn't allow it to freeze out individuals who are courageous, brilliant or foolhardy enough to go it alone.

Why does this issue grab my attention? It's because it strikes very close to "home" for me, because throughout my research career I have marched to the beat of a differerent drum. Except for very early on (e.g. during my PhD years) I have never been attracted to the idea of doing "fashionable" research, i.e. of a sort that would gain the approval of my peers, lead to lots of publications in high profile journals, and lots of recognition generally. Surely, this list of potential rewards is enough to make the decision about what sort of research to do a no-brainer? For most researchers this is indeed the case, but for a small subset of researchers the mere idea of running with the herd is deeply repulsive.

My rate of publishing in respectable journals has reduced as my research work has advanced, because I have gradually moved away from the accepted mainstream of research in information processing. I have a magnificent collection of rejection letters and referee's reports that I have collected over the years, which clearly demonstrate how conservative and cliquey the peer review process can be. Oh well, I will have to find other publication channels to disseminate my ideas, and with some imagination it isn't too hard to find unconventional ways to publish one's work. Wouldn't it be fun at some point in the future for mainstream researchers to ascend a hitherto unscaled peak, only to find an old flag with the monogram "SPL" already planted there?
Is it just a dream? Who knows? It all depends on whether I am courageous, brilliant or foolhardy (to quote from the above editorial).

Update: In Richard Dawkins book The God Delusion I found this very relevant passage (on page 196 in my 2006 edition):

As with genes in the gene pool, the memes that prevail will be the ones that are good at getting themselves copied. This may be because they have direct appeal ... or it may be because they flourish in the presence of other memes that have already become numerous in the meme pool.

So, to get your paper published it has to either stand all by itself as a self-contained body of work, or it has to dovetail neatly with the papers that have already been published. So what happens to papers that depend on other papers that have not been published? Memetics is a harsh midwife, methinks.