Continuing on my series on Bayes’ Theorem, recall that * the* question of any rational investigation is this: what is the probability of the theory of interest T, given everything that I know K? Thanks to Bayes’ theorem, we can take this probability and break it into manageable pieces. In particular, we can divide K into background information B and data D. Remember that this is just convenience, and in particular that B and D are both assumed to be known.

Suppose one calculates for some theory, data and background information. Think of it as a practice problem in a textbook. This calculation, in and of itself, knows nothing of the real world. So what follows? We can think of the probability as a conditional if-then statement:

1. If DB, then the probability of T is .

To draw a conclusion from this, we must add the premise.

2. DB.

Only then can we conclude,

3. The probability of T is .

But wait a minute … the whole point of this exercise was to reason in the face of uncertainty. Where do we get the nerve to simply assert 2, that DB is true? Where is the inevitable uncertainty of measurement? Isn’t treating the data as certain hopelessly idealized? Shouldn’t we take into account how probable DB is? But there are no raw probabilities, so with respect to what should we calculate the probability of DB? We’re headed for an infinite regress if we keep asking for probabilities. How do we get premise 2? Are probabilities all merely hypothetical?

We can put this another way: what counts as data, given that it must be treated as certain? I can know some necessary truths, like mathematical and logical truths, with something like certainty, but what about the real world? It is often said, in a kind of “*of course*” tone of voice, that all observation is theory laden. If that is the case, then there would be nothing to be taken as given, and all probabilities would be merely hypothetical. Science would just be a game of “imagine if …”. To overcome this, there must be some raw data from which we can begin.

Perhaps you think that there are some ideas that are *practically* certain, that we can just assume those and get on with it. But which ones? Remember that these are the statements *from which* we calculate probabilities, so we can’t just say that any statement with a probability greater than 0.999 … is practically certain. In any case, I think we can do better.

Let me head out on a limb (I still haven’t totally convinced myself). I think that part of the answer is the certainty of first-person experience. There seem to be contingent truths of which I can be certain. I can be certain that I am having the experience of seeing a red chair. There may not actually be a red chair – I may be dreaming, the light may be fooling me, I may be colour-blind, and so on. But I know my own thoughts immediately and incorrigibly. I can be certain that I * think* that there is a red chair. I know my own mind

*immediately*, directly, without having to infer it from anything else.

We now at least have a starting point. If I want to know whether there really is a red chair over there, I will need to calculate the probability of there being a red chair over there, given that I think there is a red chair, plus the sum total of my other experiences of the world and any other necessary truths I know. That probability may be not be obvious, or may turn out to be lower than I expect, but at least we are asking more than a hypothetical question.

These rather abstract considerations have a practical application: the data D (and the background B, for that matter) should be as raw as possible. Jaynes gives the example of an experiment in which a psychic predicts a randomly drawn card with a success rate much higher than chance. What is the data? If we take D = the experiment, then we are liable to be too easily convinced of psychic powers. But D is not given. What I know is that that some ESP experimenter *claimed* that the experiment occurred as described. Psychic powers remain a possible explanation, but now several others come into view – hoax, poor procedure, gullibility, selective reporting etc.

In practice, we rarely need to retreat all the way back to subjective experience to draw useful probability conclusions. But they at least provide a bedrock of contingent yet certain beliefs from which we can build. The regress stops here.

on December 5, 2013 at 5:43 am |Brendon J. BrewerNice post.

It is possible to update probabilities without assuming some proposition is certain (your “DB”), but I’ve never seen anyone do it in a real problem. MaxEnt allows you to update based on constraints in the space of possible probability assignments. Your “2. DB” thing could be replaced by, for example, “P(DB) = 0.99”. This is sometimes called Jeffrey conditionalisation (not Jeffreys!).