Constructive versus principled theories

In this week's New Scientist there is an article entitled Power of the mind in which the relative merits of two types of scientific theory are discussed: constructive versus principled theories.

The article points out that Einstein stressed the distinction between these two types of theory, where the constructive approach aims to describe phenomena by working backwards from known experimental observations to induce what the underlying models might be, whereas the principled approach aims to do the same by starting from a set of underlying principles and then deducing what the experimental observations should be. Einstein's own approach was typically principled theories.

The article also points out that Martin Rees (President of the Royal Society) has criticised Einstein's favoured approach as being "armchair physics", which should make way for an approach based more in rigorous experimentation. I wonder if he is taking a swipe at string theory!

This all seems to be rather black and white to me. Sure, there are the polar extremes of top-down theories (inducing a model from data, or the so-called constructive approach) and bottom-up theories (deducing data from a model, or the so-called principled approach), but these are definitely not mutually exclusive.

To see what I mean, you need look no further than Bayes theorem, which says that you can break up a joint probability Pr(model, data) in the following two ways:

Pr(model, data) = Pr(model│data) Pr(data)

and

Pr(model, data) = Pr(data│model) Pr(model)

where every term that appears on the the right hand sides can be deduced from Pr(model, data) alone.

This means that Bayes theorem treats the model and the data symmetrically, so it must be wrong to claim that either the bottom-up (principled) or the top-down (constructive) approach is somehow fundamentally superior to the other approach.

Bayes theorem has the following quantitative consequences (expressed very informally, pace Bayesians!):

1. A theory has to be rooted in experimental data for it to be science, which ensures a large value for Pr(data).
2. A theory has to be principled for it to have a large value of Pr(model). The theories that have a simple internal structure (i.e. satisfy Occam's Razor) tend to have a large Pr(model).

The Bayesian approach gives you the means of computing a quantitative measure of how good your theory is irrespective of how you arrived at it, whether through artistic inspiration, or through the sweat and labour of inspecting experimental data, or a combination of both of these approaches. The Bayesian approach is impartial in this respect.

The best sort of theory will thus combine both the bottom-up (principled) and the top-down (constructive) approaches. When I work on a new theory I am aware of being influenced both by Occam's Razor (i.e. a sense of elegance and beauty) and by the experimental data (i.e. hard-nosed pragmatism), which together create a very interesting "tension" that I have to resolve. This part of my scientific work is really satisfying, in the same way that composing music or painting is satisfying.

So, constructive versus principled theories? No! It is not a matter of either/or, it is both please.