I forgot to add: it is similar, because in the case where the prior is uniform (no previous knowledge of the distribution), minimising the relative entropy corresponds exactly to maximising the entropy. One can also perform iteratively updates, and when done right (one has to always stay on a so called mixed geodesic), the relative entropies obtained by each update add up to give the entropy of the final posterior.

]]>By various reasons that Jaynes was first pointing out, one is interested in maximising the entropy of a distribution, given constraints, to obtain a distribution that fullfills the constraints and having the least additional information. As an example, we use Gaussians to describe distribution where only mean and variance is known.

When going from a prior (original) distribution Q to posterior (updated) distribution P, that is, updating Q to obtain P, one does quite similar.

Entropy, Relative entropy (also called Kullback Leibler divergence) are commonly defined as below.

S(P) = – \sum p_i log p_i

D(P||Q) = \sum p_i log (p_i / q_i)

The relationship between entropy and and the relative entropy is as follows:

D(P || uniform) = log(N) – S(P)

where N is the support of uniform, that is, the number of events considered (e.g. 2^n for repeated coin flips.)

So in order to maximise the entropy, one has to minimises the relative entropy.

In your definition, you added a minus sign in front of the relative entropy. Of course, one then has to maximise instead of minimise. This is however a bit unusual, as one wants the relative entropy to be always positive, as a measure of uncertainty. Without minus sign, it is, and the relative entropy is only zero if P equals Q.

(Aside: Maximising the relative entropy D(P || Q) by varing P is easy. Just find a P with supp(P) > supp(Q). Then, D(P||Q) = inf, because the term P log (P/Q) will be infinite when P>0, Q=0)

You’re quite right in saying that a “reversed relative entropy” is better for approximating a distribution with another one. This however is different scenario then the one described above. Before, you wanted to update a distribution, going from the Q prior to the posterior P. Now you look for a best approximation, that is, looking for the best prior distribution Q (fullfilling a reduced set of constraints), that needs the minimal amount of update to obtain the posterior P. For this case one minimises D(P,Q), where Q is the best approximation, fulfilling some but not all constraints put on P.

To keep these things apart, on has only to remember that the posterior (updated distribution) is in the first slot, and the prior in the second slot of the relative entropy.

D( Posterior || Prior)

I think this article covers most points: Maximum Entropy and Bayesian inference: Where do we stand and where do we go?

http://bayes.wustl.edu/etj/articles/stand.on.entropy.pdf

Regardless of life or not its essentially an impossible structure When you see the trajectories it had to take to stand as a structure. This all has to do with the initial conditions prior to the big bang. To have a stable manifold and then also have stars , galaxies, and planets is enough to say the universe is fine tuned to have planets.

But are planets the ultimate objects of our reality.? If the DNA Molecule were the end product , an outside observer might say its FTuned for this virtually impossible molecule. But its concious embodied Observers that ultimately are the pinnacle of creation-so its most correct to use the Ultimate object in existence– which of course completely contradicts and is the ultimate defeater of the atheist mantra that we are insignificant nobody’s on an ordinary planet etc etc….hence, we put our foot in mouths proclaiming the parameters to a stable UV & life were wide but the data has proved precisely the opposite. Ha, it poetic.

Enter Multiverse. Just embarrassing.

]]>Yes, but this is like saying that straying from a local minimum moves one to higher ground. It does not show that there is not another minimum lower, or even much lower, than the local minimum one is in.

A coarse example: If I take a conventional radio (“wireless”) and slightly change some of the components, it very probably won’t work. But I can listen to internet radio on a computer, which differs in essentially every respect from the radio.

I still think that the multiverse argument is a plausible explanation of why *we* are here. But I don’t think that a very different universe would necessarily be devoid of life, even if a slightly different one would be.

I wanted to make a comment on your question about “changing forms” where you used the example of a triangle slowly changing into a circle.

The key thing to understand is you brought up an epistemological question rather than a metaphysical question.

Your epistemological question was: At what point does the triangle cease to exhibit the form of triangle and become something else as it moves into becoming a circle.

Epistemologically, it is possible that we could never pinpoint the exact point at which the triangle is objectively no longer an “imperfect triangle” and it becomes an “imperfect circle”. What we can say, metaphysically, is if a triangle very slowly turns into a circle, then objectively there is some point where it ceases to be an imperfect triangle and becomes something else. Again, it doesn’t matter if one could never know exactly where this point happens. One does not need to mathematically or scientifically analyze the specifics or the percentages of how close this imperfect triangle is to a triangle to say that this is the case metaphysically.

——

Now in regards to science and metaphysics–part of the essay above seemed like it wanted to turn metaphysics into physics. This is exactly what you don’t want to do, less you end up very confused and make some very bad philosophical (more specifically metaphysical) statements.

The key is to understand how to do metaphysics properly, because it is an area of study with its own methods and proper way of discovering underlying truths about reality. If you try and do metaphysics using science, it is Iike using a hammer to try and saw a board in half.

So what you are going to have to do is keep studying the method of metaphysics while leaving the scientific method at the door since science is not metaphysics (I know this may be hard for some). The key to remember is that metaphysics is not “unscientific”. It simply is a study that seeks to discover the underlying structure of reality. A rational account of the physical sciences relies on a good underlying metaphysics in the first place. Though many scientists do do science without a clue about their unconscious metaphysical assumptions, and that’s okay. They can still do science, in the proper sense of the word, without this knowledge.

So keep up the the asking of questions and your study of how to do metaphysics. Then once you have established a coherent metaphysics, then you will be well equipped to understand how to analyze the things that the physical sciences are studying (like quantum fields and such). I personally hold that the general Aristotelian-Thomistic metaphysics framework is the most coherent and correct description of reality, and I would have no problem describing any actual truth or entity that science discovers in terms of formal, material, efficient, and final causality.

]]>that others are talking about and you wont go wrong.

In culinary preparation for the past many years, heated stones have been used to cook some of the

best dishes. Using this correction factor, the model estimates that approximately 21,000 total cases will have

occurred in Liberia and Sierra Leone by September 30, 2014.

As people ease their grip on their wallets, the car industry is hoping they spend their money on a new car. ]]>