Bayesian Analysis

In this post we perform a simple but explicit analysis of a curve fitting using Bayesian techniques.

The Model

Consider the problem of curve fitting:

where $X$ is a random variable representing errors with some probability density function (PDF) $f_X(x)$. Within this model, given $t$ and $a$, $Y$ is a random variable with PDF:

Maximum Likelihood

Give a set of data ParseError: KaTeX parse error: Undefined control sequence: \vect at position 4: D=(\̲v̲e̲c̲t̲{t}, \vect{y}), one can precisely formulate the question: what is the probability (likelihood) $P(D|a)$ that this set of data would be obtained from our model given a parameter $a$:

Maximum likelihood techniques choose the best fit for the parameter $a$ to maximize the likelihood:

Bayes' theorem allows us to compute the a posteriori distribution of the parameter $a$ given the observation of data $D$, updating the prior distribution $P(a)$ normalized by the probability of obtaining the data $D$: