Eliezer Yadkowsky's page is a nice intro to Bayes's theorem, the understanding of which is critical for understanding why the posterior is proportional to the prior times the sampling distribution.
But Bayesian stats isn't just about applying Bayes's theorem. Its key feature is using probabilities for model parameters and incorporating posterior uncertainty of their values into inference.
Here's what I actually said in response to all this on the original blog, which showed up as a huge spike in our traffic!
There was a sudden spike in traffic, and it turns out it comes from Y Combinator Hacker News, where there's a discussion of this post with seven comments as of today.
The criticisms were sound -- it's too technical (i.e. jargon filled) for someone to understand who doesn't already get it. Ironically, I've been telling Andrew Gelman that about his Bayesian Data Analysis book for years.
Unix man pages are the usual exemplar of doc that only works if you mostly know the answer. They're great once you already understand something, but terrible for learning.
I think Andrew's BDA is that way -- it's clear, concise and it actually does explain everything from first principles. And there are lots of examples. So why is this so hard to understand?
I usually write with my earlier self in mind as an audience. Sorry for not targeting a far-enough back version of myself this time! The jargon should be familiar to anyone who's taken math stats. I don't think it'd have helped if I'd have defined the sum for the prior defined as a marginal.
But Bayesian stats isn't just about applying Bayes's theorem. Its key feature is using probabilities for model parameters and incorporating posterior uncertainty of their values into inference.