An Intuitive Guide to MCMC (Part I): The Metropolis-Hastings Algorithm

Markov Chain Monte Carlo (MCMC) methods are introduced as a solution for sampling from complex probability distributions where the normalization constant is intractable. This is a common problem in Bayesian statistics and high-end quantitative finance. The core concepts of Markov Chains, which are memoryless stochastic processes, and Monte Carlo methods, which use repeated random sampling, are explained. The algorithm's goal is to reach a stationary distribution, an equilibrium state where the sampling process converges to the desired target distribution. The text then begins a detailed derivation of the Metropolis-Hastings algorithm, which achieves this by satisfying the condition of detailed balance.