Abstract
In what follows, I’ll consider Carrier’s claims about the mathematical foundations of probability theory. What Carrier says about probability is at odds with every probability textbook (or lecture notes) I can find. He rejects the foundations of probability laid by frequentists (e.g. Kolmogorov’s axioms) and Bayesians (e.g. Cox’s theorem). He is neither, because we’re all wrong – only Carrier knows how to do probability correctly. That’s why he has consistently refused my repeated requests to provide scholarly references – they do not exist. As such, Carrier cannot borrow the results and standing of modern probability theory. Until he has completed his revolution and published a rigorous mathematical account of Carrierian probability theory, all of his claims about probability are meaningless.
Carrier’s version of Probability Theory
I intend to demonstrate these claims, so we’ll start by quoting Carrier at length. I won’t be relying on previous posts. In TEC, Carrier says:
Bayes’ theorem is an argument in formal logic that derives the probability that a claim is true from certain other probabilities about that theory and the evidence. It’s been formally proven, so no one who accepts its premises can rationally deny its conclusion. It has four premises … [namely P(h|b), P(~h|b), P(e|h.b), P(e|~h.b)]. … Once we have [those], the conclusion necessarily follows according to a fixed formula. That conclusion is then by definition the probability that our claim h is true given all our evidence e and our background knowledge b.
In OBR, he says:
[E]ver since the Principia Mathematica it has been an established fact that nearly all mathematics reduces to formal logic … The relevant probability theory can be deduced from Willard Arithmetic … anyone familiar with both Bayes’ Theorem (hereafter BT) and conditional logic (i.e. syllogisms constructed of if/then propositions) can see from what I show there [in Proving History] that BT indeed is reducible to a syllogism in conditional logic, where the statements of each probability-variable within the formula is a premise in formal logic, and the conclusion of the equation becomes the conclusion of the syllogism. In the simplest terms, “if P(h|b) is w and P(e|h.b) is x and P(e|~h.b) is y, then P(h|e.b) is z,” which is a logically necessary truth, becomes the concluding major premise, and “P(h|b) is w and P(e|h.b) is x and P(e|~h.b) is y” are the minor premises. And one can prove the major premise true by building syllogisms all the way down to the formal proof of BT, again by symbolic logic (which one can again replace with old-fashioned propositional logic if one were so inclined).
More specifically it is a form of argument, that is, a logical formula that describes a particular kind of argument. The form of this argument is logically valid. That is, its conclusion is necessarily true when its premises are true. Which means, if the three variables in BT are true (each representing a proposition about a probability, hence a premise in an argument), the epistemic probability that results is then a logically necessary truth. So, yes, Bayes’ Theorem is an argument.
He links to, and later shows, the following “Proof of Bayes Theorem … by symbolic logic”, saying that “the derivation of the theorem is this.”
For future reference, we’ll call this “The Proof”. Of his mathematical notation, Carrier says:
P(h|b) is symbolic notation for the proposition “the probability that a designated hypothesis is true given all available background knowledge but not the evidence to be examined is x,” where x is an assigned probability in the argument.
Like nothing we’ve ever seen
I have 13 probability textbooks/lecture notes open in front of me: Bain and Engelhardt; Jaynes (PDF); Wall and Jenkins; MacKay (PDF); Grinstead and Snell; Ash; Bertsekas and Tsitsiklis; Rosenthal; Bayer; Dembo; Sokol and Rønn-Nielsen; Venkatesh; Durrett; Tao. I recently stopped by Sydney University’s Library to pick up a book on nuclear reactions, and took the time to open another 15 textbooks. I’ve even checked some of the philosophy of probability literature, such as Antony Eagle’s collection of readings (highly recommended), Arnborg and Sjodin, Caticha, Colyvan, Hajek (who has a number of great papers on probability), and Maudlin.
When presenting the foundations of probability theory, these textbooks and articles roughly divide along Bayesian vs frequentist lines. The purely mathematical approach, typical of frequentist textbooks, begins by thinking about relative frequencies before introducing measure theory, explaining Kolmogorov’s axioms, motivating the definition of conditional probability, and then – in one line of algebra – giving “The Proof” of Bayes theorem. Says Mosteller, Rourke and Thomas: “At the mathematical level, there is hardly any disagreement about the foundations of probability … The foundation in set theory was laid in 1933 by the great Russian probabilitist, A. Kolmogorov.” With this mathematical apparatus in hand, we use it to analyse relative frequencies of data.
Bayesians take a different approach (e.g. Probability Theory by Ed Jaynes). We start by thinking about modelling degrees of plausibility. The frequentist, quite rightly, asks what the foundations of this approach are. In particular, why think that degrees of plausibility should be modelled by probabilities? Why think that “plausibilities” can be mathematised at all, and why use Kolmogorov’s particular mathematical apparatus? Bayesians respond by motivating certain “desiderata of rationality”, and use these to prove via Cox’s theorem (or perhaps via de Finetti’s “Dutch Book” arguments) that degrees of plausibility obey the usual rules of probability. In particular, the product rule is proven, p(A and B | C) = p(A|B and C) p(B|C), from which Bayes theorem follows via “The Proof”.
In precisely none of these textbooks and articles will you find anything like Carrier’s account. When presenting the foundations of probability theory in general and Bayes Theorem in particular, no one presents anything like Carrier’s version of probability theory. Do it yourself, if you have the time and resources. Get a textbook (some of the links above are to online PDFs), find the sections on the foundations of probability and Bayes Theorem, and compare to the quotes from Carrier above. In this company, Carrier’s version of probability theory is a total loner. We’ll see why.
A Pythagorean Parallel
To draw out the various idiosyncrasies of Carrier’s account of Bayes Theorem, consider this parallel discussion of a different mathematical theorem:
Pythagoras Theorem (PT) is an argument in formal logic. It’s been formally proven. It follows from two premises (a and b), from which the conclusion (c) follows according to a fixed formula, where a and b are assigned a value in the argument. PT is reducible to a syllogism in conditional logic as follows:
(1′) If the two shorter sides of a right-angled triangle are a and b, then the hypotenuse is c.
(2′) The shorter sides of a right-angled triangle are a and b
(3′) Therefore, the hypotenuse is c
One can prove (1′) by building syllogisms all the way down to the formal proof of PT. So, Pythagoras theorem is an argument, or a form of argument. Its conclusion (c) is necessarily true when its premises (a and b) are true.
The problems are legion.
- This “argument form” of PT is missing PT itself; we must add “where c2 = a2 + b2” to premise (1′) to give it any meaning.
- It does not show that Pythagoras theorem is an argument or a form of argument. It shows that PT can be used in an argument. But that’s trivial – any statement can be a premise in an argument.
- The “form” of the argument is just modus ponens, “If A then B. A. Therefore B”. There is no particular “form of argument” associated with PT.
- Don’t call a, b and c “premises” or “symbolic notation for a proposition”. You can’t multiply and add premises or propositions, and that’s what happens in PT. They’re numbers. You put them in a formula.
- The discussion above is not a proof of PT because PT is (or should be; see A) included in premise (1′). Nor does it show how PT follows from an axiomatization of mathematics or reduces to symbolic logic. It shows how to argue from PT, not to PT.
- You cannot prove PT from the axioms of arithmetic because those axioms don’t define what a triangle and a right angle are, or what to do with them. You need axioms of plane geometry, such as Euclid’s axioms (or their more modern descendants).
Back to Bayes
We can apply A-F straightforwardly to Carrier’s discussion of BT. For convenience, I’ll number Carrier’s premises:
(1) If P(h|b) is w and P(e|h.b) is x and P(e|~h.b) is y, then P(h|e.b) is z
(2) P(h|b) is w and P(e|h.b) is x and P(e|~h.b) is y
(3) Therefore, P(h|e.b) is z
- This “argument form” of BT is missing BT itself. As it stands, (1) states that P(h|e.b) is equal to some unspecified, arbitrary number. We must add “where z = xw / [xw + y(1 – w)]” to premise (1) to give it any meaning.
- It does not show that Bayes theorem is an argument or a form of argument. It shows that BT can be used in a syllogism … as can any other statement. It is also an unnecessary complication to use BT in this form – what Carrier calls the “reduction to a syllogism in conditional logic” every mathematician would call “putting numbers into a formula”.
- The “form” of the argument is just modus ponens, “If A then B. A. Therefore B”. There is no particular “form of argument” associated with BT.
- Don’t call P(e|h.b) etc premises or “symbolic notation for [a] proposition”, because you can’t multiply and add and divide premises and propositions, and that’s what happens in BT. They’re numbers. You put them in a formula.
- The syllogism (1)-(3) is not a proof of BT, because BT is (or should be; see A) included in premise (1). Nor does it show how BT follows from an axiomatization of mathematics or reduces to symbolic logic. It shows how to argue from BT, not to BT.
- You cannot prove BT from the axioms of arithmetic because they don’t know what a probability is, or what to do with it. Read, for example, the Peano axioms. They define natural numbers, equality, succession – but not probability. You need the axioms of probability theory, such as Kolmogorov’s axioms.
“The Proof”
All of which brings us to “The Proof”, which is nothing of the sort. It is an elementary probability exercise, the kind of thing you’d set a first year student: show that Bayes theorem follows from the product rule (Statement (1) in “The Proof”). Actually, the problem is so easy so most textbooks just do it in one line and move on. Venkatesh (page 56), for example, presents “The Proof” in one line and says that it is “… little more than a restatement of the definition of conditional probability.”
Showing that a statement follows from some other statement does not prove it. You have to show that it follows from the relevant axioms, or a theorem derived from those axioms. Given that the product rule follows from the axioms of probability theory or Cox’s theorem or from the definition of conditional probabilities (which it does), “The Proof” does in fact establish BT. But Carrier is claiming more: “The Proof” is supposedly “the formal proof of BT … by symbolic logic”, showing how this mathematical theorem “reduces to formal logic” in the rigorous tradition of the Principia Mathematica.
This is just wrong. “The Proof” is a derivation of BT from the product rule. We’re a million miles from the axiomatic foundations of mathematics. Worse, Carrier thinks that “The Proof” is “by symbolic logic”, when it is quite plainly an exercise in algebra. The product rule involves the logical operator “and”, but all the manipulations are algebraic (multiplying and dividing), not logical. I’ll repeat that point: this supposed proof “by symbolic logic” uses none of the rules of symbolic logic.
This is not a small technicality. A trivial algebraic exercise, too easy for any competent student, is being presented by Carrier as a rigorous, formal, first-principles proof of Bayes theorem using symbolic logic alone. This is decisive evidence of Carrier’s utter cluelessness when it comes to probability theory. No mathematician, when asked about the foundations of probability theory, will point to “The Proof”.
The Foundations of Probability Theory
Point F is perhaps the most important, so I’ll expand on it. As we saw above, there is a substantial academic literature on the foundations of probability, the status of Bayes Theorem, its relation to the interpretation of probability, and the various ways in which it can be derived. Generally, Bayesians go with Cox’s theorem or Dutch book arguments, while frequentists go with Kolmogorov.
Carrier, all out on his own, needs none of this. He is not using the established results of modern probability theory. He isn’t really a Bayesian or a frequentist or anything that mathematicians have seen before. He has his own, revolutionary approach to probability theory. Carrierian probability does not need Kolmogorov’s axioms, or indeed any probability axioms, or Jaynes’s desiderata, or Cox’s theorem, or Dutch book arguments. Every probability textbook, frequentist and Bayesian alike, is wasting their time. Just give Carrier symbolic logic, modus ponens and the axioms of arithmetic, and stand back.
In fact, in Proving History he makes these claims explicit. In the section “Bayesianism as Epistemic Frequentism” (Chapter 6), he outlines an approach to probabilities, according to which “all Bayesians are in fact frequentists“. When Bayesians claim that probabilities are not frequencies “they are simply wrong … They just sometimes don’t realize what their probabilities are frequencies of. … Always at root you will find some sort of physical frequency that you were measuring or estimating all along”.
What is this frequency that all Bayesians everywhere have been ignoring all this time? According to Carrier, degrees of belief are really the ratio of the number of “beliefs that are true” to the number of “all beliefs backed by a certain comparable quantity and quality of evidence”. For example, “When a Bayesian says that the prior probability that a royal flush is fair is 95% … they are really saying that 95% of all royal flushes drawn (on relevantly similar occasions) are fair. Which is a physical frequency. Thus, epistemic probabilities always derive from physical probabilities.”
Just one thing is needed
It may seem ungrateful to nit-pick such a monumental intellectual achievement, but he’s not quite done. To complete his exposition of Carrierian probability, and show those so-called mathematicians how probability should be done, Carrier has one more task: justify a mathematical method that allows us to compare two sets of “evidence” with regards to their “quality and quantity”, so we can decide when they are “relevantly similar”. That’s all. Just quantify “evidence”.
Heck, I’ll even get him started. Perhaps, to allow for comparison, “quality and quantity” could be represented by a real numbers q. More evidence could mean larger q values, and adding evidence should never decrease q. We should stipulate consistency: equivalent states of evidence receive equal q values. In which case, good news! Just such a method already exists! The principles that govern q are basically the premises of Cox’s theorem, from which we get Bayesian probabilities.
So let’s review Carrier’s method. To calculate probabilities, we need to define the reference classes in which to place our various beliefs. And to define those, we’ll need a measure of the quantity and quality of evidence. This measure, it turns out, is basically Bayesian probabilities.
Carrierian probability, as presented, is thus hopelessly incomplete. And when completed, circular. The circularity is staring us in the face when Carrier says that the relevant reference class, from which we calculate probabilities, is defined as containing those beliefs “backed by the kind of evidence and data that produces those kinds of prior and consequent probabilities.” Only once we have probabilities can we form such reference classes (ignoring the issue of when two “amounts of evidence” are “comparable” – exactly equal? Within 1%?). It follows that these reference classes cannot be used to define the probabilities.
Conclusion
So, it’s time to put up or shut up. When invited to outline the foundations of his approach to probability theory (which I first did 2 years ago – question 5), Carrier snubs modern probability theory. Axioms, Kolmogorov, Cox, de Finetti … who needs them! But a mathematical theory without foundations is just hot air. Until a rigorous basis for Carrierian probability theory is provided, all his probability claims are meaningless.
Carrier must publish a series of papers in mathematical journals that substantiate his extraordinary claims about the foundations of probability theory, proving – in the face of centuries of work by mathematicians – that:
- None of the usual axioms or arguments or theorems are needed.
- Probability reduces to formal/symbolic logic alone.
- Bayesianism is really a kind of frequentism.
- “Quality and quantity of evidence” can be uniquely and precisely quantified.
He made these claims, so this is a task that he has set for himself. Citing his own books is not good enough.
Or else, we’ll know that he is all bluff and bluster. Pressed to present the foundations of probability theory, he has failed utterly. He could have just plagiarised any probability textbook, but instead invented a pile of garbage about conditional logic, building syllogisms, Principia Mathematica, the axioms of arithmetic, and quality of evidence.
Hence, this is my final word. If Carrierian probability is hailed as a revolution by mathematicians, then I will concede Carrier’s probabilistic credentials and be forever silenced. If he continues to talk about probabilities, then – since he doesn’t mean by “probability” what the term means in any rigorous mathematical theory – this must be regarded as literally meaningless. We need only reply: where are the papers?
If, alternatively, he realises that he is completely out of his depth, that he hasn’t got the first clue about the foundations of probability theory, he may (after learning probability theory – for the first time, it seems – from a textbook) try to claim that he has been a follower of Cox/Kolmogorov all along. However, as we have seen, this is complete shift in the foundations of his approach. All of his previous work that relies on Carrierian probability – including its extension to historical investigation in Proving History and On the Historicity of Jesus – must be discarded.
Great post as usual. I might add that elsewhere on RCB Richard has claimed that his approach to probabilities is the only one, i.e. all other methods reduce to his approach (which is odd since strictly speaking Kolmogorov and Cox/Jaynes are not formally equivalent)
I have had quite long discussions with Richard on his interpretation and I believe it might have shifted a bit. As you note, his original interpretation in PH is that to say the probability of X is 1/4 this means that of all things one know on the same strength of evidence as X, 1 in 4 things will be true; asides the problems you mention this would seemingly rule out irrational probabilities. After some discussion if probabilities could be irrational, I think he now believes that if you can imagine a thought-experiment that produces a particular probability, the interpretation of that probability is that it is the result of the thought experiment possibly after taking a limit. So for instance a probability of 1/pi “means” that you imagine a quantum gun (his example!) that shoots against a disc of radius 1 and the probability of 1/pi is then defined as the limit frequency of the shots which hit the disc vs. those who hit the encompassing square. So right now I do not think Richard actually believes “probability” is any single thing and that his position is better characterized as a collection of different attempts at a definition that he changes between depending on the situation.
Btw. I might also suggest Terenin and Draper (2015) as a relevant paper on the foundations of Bayesian probabilities:
Click to access 1507.06597.pdf
I’m not sure I understand. Even if Carrier’s derivation of Bayes’s Theorem is flawed, can’t we just nevertheless take him at his word that he’s a frequentist? One can, I’m guessing, apply a frequentist interpretation of Bayes to say, historical questions, even if one has no idea whence the formula comes. That would still leave Carrier open to charges of incoherence when he switches between frequentist and non-frequentist interpretations, but that would be a separate issue, no?
He’s not a frequentist. Frequentists don’t believe that prior probabilities exist, but Carrier does.
sotlane: Well, obviously you can just use Bayes theorem/the rules of probabilities and say probabilities are whatever you fancy. Indeed a more or less ‘interpretation-free’ view is found in for instance “Bayesian data analysis” by Gelman et.al.
I think the difficulties with Carriers view is that he will sometimes justify things that are nonsensical on traditional interpretations of probability theory (for instance particular uses of reference classes for one-off events) and then justify this based on his particular interpretation of what probabilities are — and which everyone should accept because it is supposedly proven in Proving History. That makes a conversation very difficult because he is so to speak determining the rules of what’s allowed as you go along.
[…] Final Word on Richard Carrier […]
[…] using probability, and Barnes responded with two posts mainly on the fine-tuning and two (here and here) on probability – which is what I’m focusing on […]
I used to be a fan of Richard Carrier. Before Carrier’s book PH was published, I was excited about the idea of BT being used to argue for historical probability. However, after seeing numerous blunders by him on his blog and in his books, I have now come to regard Carrier as a quack historian. I admit: I have a lot of homework to do as far as BT goes. In fact, I am considering taking a university course in statistics and a few courses in calculus when I can afford it. I have a desire to learn more about calculus, so I can better understand the mathematics that J.M. Keynes uses in his books *Treatise on Probability* and *The General Theory*.
What I really role my eyes at is the utter arrogance of Carrier. He writes as though he’s the world’s foremost expert on BT and its application to history. He writes as though he’s the world’s newest expert on the historical Jesus and he’s managed to outshine the vast majority of New Testament scholars with his unique and revolutionary approach. Add to this his very judgmental and abusive reviews of other scholars (Ehrman, Casey, McGrath,..etc) and one sees someone who is not only a quack with a delusional opinion of his own brilliance and revolutionary mind but one also gets a very clear insight into his utterly unprofessional approach to scholarship.
Matthew
Carrier got so beaten by Barnes, I have absolutely no clue how he could ever muster himself to get up after this. I think we should give a name to this smackdown, getting ‘Barneslapped’ (think getting hitchslapped by Hitchens)
Nice post. Carrier’s arrogance in dismissing 100 years of work in the foundations of probability is stunning. Still, his idea that Bayesian probabilities can be reduced to frequentist probabilities is at least interesting. I suspect that it *can* be fleshed out in a way that makes it better than simply circular. It seems related to the notion of calibration of subjective probabilities. Perhaps one could make an argument that subjective probabilities, if they have any meaning at all, have meaning only in so far as they are well-calibrated. Calibration, which is clearly a frequentist notion, could perhaps thus be seen as the foundation of subjective probability and not merely a desired feature of such probabilities. How (and even if) this could be fleshed-out, and whether or not the resulting theory would address the objections that Bayesian’s have raised against the frequentist approach, I don’t know. I am somewhat skeptical. Making it work in some sense might not be too hard. Convincing Bayesians that this is what they *really* meant all along would be much harder.
[…] https://letterstonature.wordpress.com/2016/02/05/final-word-on-richard-carrier/ […]
To paraphrase from Carrier’s own rules about what history to trust (PH, p. 17):
Beware of scholars who make amazing claims about mathematics but who are not experts in the specific field or aren’t even experienced mathematicians themselves.
Hi dr Barnes,
Recently an interesting online exchange between Richard Carrier and philosopher Robert Koons took place, not on the fine tuning argument but on the Argument from Contigency (see: https://www.youtube.com/watch?v=naoYElI-tko&ab_channel=CCChannelArchive ).
Carrier proposes his alternative argument that the ultimate foundation of reality may well be a state of nothingness and tries to show that from this the probability that this universe would arise is virtual equal to 1.
There is a lot of talk about probability theory, Robert Koons presented several objections to Carriers argument, but I’m interested to see what your thoughts were given your expertise in probability theory.