{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11f07861-960a-4166-8b15-77d298ac66be",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy import stats\n",
    "\n",
    "plt.rcParams.update({\"figure.dpi\": 110, \"axes.grid\": True, \"grid.alpha\": 0.3, \"font.size\": 11})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6300105c-6e03-4bd9-bd31-a27614c56b43",
   "metadata": {},
   "source": [
    "---\n",
    "## A. Maximum likelihood for a mixture\n",
    "\n",
    "Events $x\\in[0,1]$ come from two populations with known shapes,\n",
    "$$\n",
    "P_A(x)=\\frac{2e^{-2x}}{1-e^{-2}},\\qquad P_B(x)=3x^2,\\qquad\n",
    "P(x\\mid f)=f\\,P_A(x)+(1-f)\\,P_B(x),\n",
    "$$\n",
    "and we want the fraction $f$. The true value is $f = 0.3$.\n",
    "\n",
    "**Simulating the data.** Draw each event from A with probability $f$, otherwise from B. Use the inverse-CDF method with $u\\sim U(0,1)$:\n",
    "$$\n",
    "x_A=-\\tfrac12\\ln\\!\\big[1-u\\,(1-e^{-2})\\big],\\qquad x_B=u^{1/3}.\n",
    "$$\n",
    "\n",
    "**TODO A1.** Complete `sample_mixture(N, f, rng)` and check it against $P(x\\mid f)$ with a histogram."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "754111fd-47a0-47d4-b29a-436aa23e9383",
   "metadata": {},
   "outputs": [],
   "source": [
    "def P_A(x):\n",
    "    return 2 * np.exp(-2 * x) / (1 - np.exp(-2))\n",
    "\n",
    "def P_B(x):\n",
    "    return 3 * x**2\n",
    "\n",
    "def P_mix(x, f):\n",
    "    return f * P_A(x) + (1 - f) * P_B(x)\n",
    "\n",
    "def sample_mixture(N, f, rng):\n",
    "    u = rng.random(N)\n",
    "    from_A = rng.random(N) < f                    # which population each event comes from\n",
    "    xA = -0.5 * np.log(1 - u * (1 - np.exp(-2)))\n",
    "    xB = u ** (1 / 3)\n",
    "    return np.where(from_A, xA, xB)\n",
    "\n",
    "rng = np.random.default_rng(42)\n",
    "frac =0.5\n",
    "x_test = sample_mixture(50000, frac, rng)\n",
    "xx = np.linspace(0, 1, 200)\n",
    "plt.hist(x_test, bins=50, density=True, histtype=\"step\", label=\"simulated\")\n",
    "plt.plot(xx, P_mix(xx, frac), label=\"P(x | f = 0.3)\")\n",
    "plt.plot(xx, frac*P_A(xx), '--')\n",
    "plt.plot(xx, (1-frac)*P_B(xx), '--')\n",
    "plt.xlabel(\"x\"); plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "200969c9-bbd2-4e44-a350-58a2b20b08ec",
   "metadata": {},
   "source": [
    "**TODO A2.** Write the negative log-likelihood\n",
    "$$\n",
    "-\\ln L(f)=-\\sum_{i=1}^{N}\\ln P(x_i\\mid f)\n",
    "$$\n",
    "for a **grid** of $f$ values, and a function `ml_interval` that returns $\\hat f$ and the interval where $-\\ln L$ rises by `level` above its minimum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65946faa-7817-4d46-a2a5-3c9b2f75be47",
   "metadata": {},
   "outputs": [],
   "source": [
    "F_GRID = np.linspace(0.0005, 0.9995, 1999)\n",
    "\n",
    "def nll(fgrid, x):\n",
    "    out = np.zeros(len(fgrid))\n",
    "    for k, f in enumerate(fgrid):\n",
    "        out[k] = -np.sum(np.log(P_mix(x, f)))\n",
    "    return out\n",
    "\n",
    "def ml_interval(x, level=0.5, fgrid=F_GRID):\n",
    "    L = nll(fgrid, x)\n",
    "    dL = L - L.min()\n",
    "    inside = fgrid[dL <= level]\n",
    "    # if the interval reaches 0 or 1 it is truncated by the physical boundary\n",
    "    return fgrid[np.argmin(L)], inside.min(), inside.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f158fd36-d21e-4f66-bd54-1e0be0475fe5",
   "metadata": {},
   "source": [
    "**TODO A3.** Simulate one experiment with $N = 20$ (use `np.random.default_rng(30)`; for $N=500$ use seed 43, the answer to everything). Plot $\\Delta(-\\ln L)$ vs $f$, mark $\\Delta\\ln L = 0.5, 2, 4.5$ and quote $\\hat f$ with its 1σ, 2σ, 3σ intervals. Overplot the parabola $\\tfrac12\\big((f-\\hat f)/\\sigma_f\\big)^2$, with $\\sigma_f^{-2}$ from the numerical second derivative of $-\\ln L$ at $\\hat f$. Repeat for $N = 500$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae3153e9-0570-4b68-b949-cd13266c13eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(13, 4.3))\n",
    "for ax, N in zip(axes, [20, 500]):\n",
    "    x = sample_mixture(N, 0.3, np.random.default_rng(11 if N == 20 else 43))\n",
    "    L = nll(F_GRID, x); dL = L - L.min(); fhat = F_GRID[np.argmin(L)]\n",
    "    h = 1e-4\n",
    "    curv = (nll([fhat + h], x)[0] - 2 * nll([fhat], x)[0] + nll([fhat - h], x)[0]) / h**2\n",
    "    sf = 1 / np.sqrt(curv)\n",
    "    ax.plot(F_GRID, dL, lw=2, label=r\"$\\Delta(-\\ln L)$\")\n",
    "    ax.plot(F_GRID, 0.5 * ((F_GRID - fhat) / sf) ** 2, \"--\", label=\"parabola\")\n",
    "    print(f\"N = {N}:  f_hat = {fhat:.3f},  curvature sigma = {sf:.3f}\")\n",
    "    for lev, c, n in [(0.5, \"C1\", 1), (2, \"C2\", 2), (4.5, \"C3\", 3)]:\n",
    "        _, lo, hi = ml_interval(x, lev)\n",
    "        ax.axhline(lev, color=c, ls=\":\")\n",
    "        print(f\"   {n} sigma:  [{lo:.3f}, {hi:.3f}]   ->  +{hi-fhat:.3f} / -{fhat-lo:.3f}\")\n",
    "    ax.axvline(0.3, color=\"gray\"); ax.set_ylim(0, 6); ax.set_xlabel(\"f\"); ax.set_title(f\"N = {N}\"); ax.legend()\n",
    "    if N == 500: ax.set_xlim(0.1, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ca92a7e-b08f-452e-87cc-e4b4f80dcb4a",
   "metadata": {},
   "source": [
    "**TODO A4 — coverage.** Repeat the experiment 500 times for $N = 20$ and for $N = 500$ (true $f = 0.3$). For each:\n",
    "1. histogram the $\\hat f$ values; compute their mean and RMS;\n",
    "2. compute the mean of the lower and upper $\\Delta\\ln L = \\tfrac12$ errors;\n",
    "3. compute the **coverage**: the fraction of experiments whose 1σ interval contains $f = 0.3$.\n",
    "\n",
    "*(N = 500 × 500 repetitions takes some seconds — be patient, or use a coarser `fgrid`.)*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1c74b62c-cd56-4aa4-a0d7-22808f08e1ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 2, figsize=(13, 4))\n",
    "r = np.random.default_rng(43)\n",
    "for ax, N in zip(axes, [20, 500]):\n",
    "    fh, elo, ehi, hit = [], [], [], 0\n",
    "    fgrid = F_GRID if N == 20 else np.linspace(0.0005, 0.9995, 999)\n",
    "    for _ in range(100):\n",
    "        x = sample_mixture(N, 0.3, r)\n",
    "        a, lo, hi = ml_interval(x, 0.5, fgrid)\n",
    "        fh.append(a); elo.append(a - lo); ehi.append(hi - a); hit += lo <= 0.3 <= hi\n",
    "    fh = np.array(fh)\n",
    "    print(f\"N = {N:3d}: mean f_hat = {fh.mean():.3f}, RMS = {fh.std():.3f};  \"\n",
    "          f\"mean errors from likelihood -{np.mean(elo):.3f} / +{np.mean(ehi):.3f};  coverage = {hit/500:.3f}\")\n",
    "    ax.hist(fh, bins=30, histtype=\"step\", lw=1.5); ax.axvline(0.3, color=\"gray\")\n",
    "    ax.set_title(f\"N = {N}\"); ax.set_xlabel(r\"$\\hat f$\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "467f5bc7-cc12-42a7-9072-95fbb3f0673f",
   "metadata": {},
   "source": [
    "**Questions A.**\n",
    "1. Is the likelihood for $N=20$ parabolic? Are the errors symmetric? What about $N=500$?\n",
    "2. Does the RMS of $\\hat f$ over the simulations agree with the error from $\\Delta\\ln L = \\tfrac12$? Is the coverage close to 68.3%?\n",
    "3. Is $\\hat f$ biased for $N = 20$? *(Hint: what happens near the boundaries $f = 0$, $f = 1$?)*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a0a201f-28ee-48a6-8fe0-89c7e706721f",
   "metadata": {},
   "source": [
    "**Answers A **"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d15c16b-5229-4a38-a291-4c64645b57a2",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f5ad87e4-c4db-452b-b389-abac2cd7f431",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "976e378b-4509-4b2f-8261-c9fa587ae1c3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de1b3371-8178-4b9e-b089-706dd96ee3e7",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "b1198a67",
   "metadata": {},
   "source": [
    "# Hands-on — Flat or sloped line?\n",
    "**Observational Astrophysics, NKUA** · frequentist vs Bayesian model comparison\n",
    "\n",
    "The six points of Lecture 4 (slides 21–23). The errors are Gaussian and **known**. Two models:\n",
    "\n",
    "- $M_0$: $y = b$ — a flat line (1 parameter)\n",
    "- $M_1$: $y = m\\,x + b$ — a sloped line (2 parameters)\n",
    "\n",
    "In Lecture 4 we saw that the flat line has an *acceptable* $\\chi^2$, but the slope is $\\approx 3\\sigma$ from zero. Today we compare the two models in every way we know:\n",
    "\n",
    "1. **Frequentist:** $\\chi^2$ goodness of fit, $\\Delta\\chi^2$ (likelihood-ratio) test, AIC, BIC\n",
    "2. **Bayesian:** posterior of $(m, b)$, evidences, Bayes factor and odds ratio — and the role of the **prior**\n",
    "3. **Simulations:** 1000 fake data sets from each model — how often does each criterion pick the right model?\n",
    "4. **Discussion:** connect to Gregory's spectral-line problem (Homework 3).\n",
    "\n",
    "Most of the code is given and documented; the cells marked **TODO** are short. Answer the questions in text cells."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d18af9b0",
   "metadata": {},
   "source": [
    "## 0. Setup and data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59bbce16",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy import stats\n",
    "from scipy.special import logsumexp\n",
    "\n",
    "plt.rcParams.update({\"figure.dpi\": 110, \"axes.grid\": True, \"grid.alpha\": 0.3, \"font.size\": 11})\n",
    "rng = np.random.default_rng(2026)\n",
    "\n",
    "x   = np.arange(1, 7.0)\n",
    "y   = np.array([4.705, 4.861, 5.963, 5.959, 6.794, 7.351])\n",
    "sig = np.array([0.7, 0.8, 0.6, 0.7, 0.9, 0.8])\n",
    "w   = 1 / sig**2\n",
    "N   = len(x)\n",
    "\n",
    "plt.errorbar(x, y, sig, fmt=\"o\", capsize=3)\n",
    "plt.xlabel(\"x\"); plt.ylabel(\"y\"); plt.title(\"the data\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06994138",
   "metadata": {},
   "source": [
    "## 1. Frequentist fits\n",
    "\n",
    "For Gaussian errors, maximum likelihood = least squares. Both models are linear in their parameters, so the best fits are analytic (weighted means and the normal equations of Lecture 4). We write everything with weighted sums $S = \\sum w_i$, $S_x = \\sum w_i x_i$, … so that the same functions can be reused in the simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84090acc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def fit_flat(y):\n",
    "    # M0: y = b.  Returns b_hat, sigma_b, chi2_min\n",
    "    b = np.sum(w * y) / np.sum(w)\n",
    "    return b, 1 / np.sqrt(np.sum(w)), np.sum(w * (y - b)**2)\n",
    "\n",
    "def fit_line(y):\n",
    "    # M1: y = m x + b.  Returns (m, b), covariance matrix, chi2_min\n",
    "    S, Sx, Sxx = np.sum(w), np.sum(w * x), np.sum(w * x * x)\n",
    "    Sy, Sxy = np.sum(w * y), np.sum(w * x * y)\n",
    "    D = S * Sxx - Sx**2\n",
    "    m = (S * Sxy - Sx * Sy) / D\n",
    "    b = (Sxx * Sy - Sx * Sxy) / D\n",
    "    cov = np.array([[S, -Sx], [-Sx, Sxx]]) / D        # [[var m, cov(m,b)], [cov(m,b), var b]]\n",
    "    return np.array([m, b]), cov, np.sum(w * (y - m * x - b)**2)\n",
    "\n",
    "b0, sb0, chi2_0 = fit_flat(y)\n",
    "p1, cov1, chi2_1 = fit_line(y)\n",
    "sm1, sb1 = np.sqrt(np.diag(cov1))\n",
    "print(f\"M0: b = {b0:.3f} +/- {sb0:.3f},                      chi2 = {chi2_0:.2f}\")\n",
    "print(f\"M1: m = {p1[0]:.3f} +/- {sm1:.3f},  b = {p1[1]:.3f} +/- {sb1:.3f},  chi2 = {chi2_1:.2f},  \"\n",
    "      f\"rho(m,b) = {cov1[0,1]/(sm1*sb1):.2f}\")\n",
    "\n",
    "xx = np.linspace(0.5, 6.5, 50)\n",
    "plt.errorbar(x, y, sig, fmt=\"o\", capsize=3, color=\"k\")\n",
    "plt.plot(xx, np.full_like(xx, b0), label=\"M0: flat\")\n",
    "plt.plot(xx, p1[0] * xx + p1[1], label=\"M1: sloped\")\n",
    "plt.legend(); plt.xlabel(\"x\"); plt.ylabel(\"y\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab393793",
   "metadata": {},
   "source": [
    "**TODO 1.** For each model compute the number of degrees of freedom and the goodness-of-fit p-value $P(\\chi^2_{\\nu} > \\chi^2_{\\min})$ (use `stats.chi2.sf`). Is either model *rejected* by its own $\\chi^2$?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6b7d3a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO 1\n",
    "dof_0 = ...\n",
    "dof_1 = ...\n",
    "p_gof_0 = ...\n",
    "p_gof_1 = ...\n",
    "print(f\"M0: chi2 = {chi2_0:.2f} / {dof_0} dof, p = {p_gof_0:.3f}\")\n",
    "print(f\"M1: chi2 = {chi2_1:.2f} / {dof_1} dof, p = {p_gof_1:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5001a62d",
   "metadata": {},
   "source": [
    "## 2. Comparing the models: $\\Delta\\chi^2$, AIC, BIC\n",
    "\n",
    "- $\\Delta\\chi^2 = \\chi^2_0 - \\chi^2_1$ is $-2\\ln$ of the likelihood ratio. If $M_0$ is true it follows $\\chi^2$ with **1** dof (one extra parameter; Wilks).\n",
    "- $\\mathrm{AIC} = -2\\ln L_{\\max} + 2k$, $\\quad \\mathrm{BIC} = -2\\ln L_{\\max} + k\\ln N$, with $-2\\ln L_{\\max} = \\chi^2_{\\min} + \\text{const}$ (the constant is the same for both models and cancels).\n",
    "\n",
    "**TODO 2.** Complete `aic` and `bic`, then print the table. Which model does each criterion prefer, and how strongly?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b35d4dd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def aic(chi2_min, k):\n",
    "    return ...      # TODO\n",
    "\n",
    "def bic(chi2_min, k, n=N):\n",
    "    return ...      # TODO\n",
    "\n",
    "k0, k1 = 1, 2\n",
    "d_chi2 = chi2_0 - chi2_1\n",
    "p_lrt = stats.chi2.sf(d_chi2, 1)\n",
    "print(f\"Delta chi2 = {d_chi2:.2f}  ->  p = {p_lrt:.4f}  ({stats.norm.isf(p_lrt/2):.1f} sigma, two-sided)\")\n",
    "print(f\"            M0      M1     M0 - M1\")\n",
    "print(f\"AIC     {aic(chi2_0,k0):7.2f} {aic(chi2_1,k1):7.2f} {aic(chi2_0,k0)-aic(chi2_1,k1):7.2f}\")\n",
    "print(f\"BIC     {bic(chi2_0,k0):7.2f} {bic(chi2_1,k1):7.2f} {bic(chi2_0,k0)-bic(chi2_1,k1):7.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f8d1b0",
   "metadata": {},
   "source": [
    "*Note:* for nested models differing by one parameter, $\\Delta\\mathrm{AIC} = \\Delta\\chi^2 - 2$ and $\\Delta\\mathrm{BIC} = \\Delta\\chi^2 - \\ln N$: the three numbers are the same information with different thresholds. \"Prefer $M_1$ if $\\Delta\\mathrm{AIC}>0$\" means \"if $\\Delta\\chi^2 > 2$\", i.e. $p < 0.16$; \"if $\\Delta\\mathrm{BIC}>0$\" means $\\Delta\\chi^2 > \\ln 6 = 1.79$ here. BIC is also an approximation to $-2\\ln$(evidence) — next section."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edcb8267",
   "metadata": {},
   "source": [
    "## 3. Bayesian analysis\n",
    "\n",
    "Priors (uniform, independent): $b \\sim U[0, 12]$ in both models; in $M_1$, $m \\sim U[m_{\\rm lo}, m_{\\rm hi}]$, starting with $[0, 1]$ (\"we expected a positive slope of order 1\").\n",
    "\n",
    "We compute everything on a grid in $(m, b)$. The posterior is $\\propto L(m,b)\\,\\pi(m)\\,\\pi(b)$, and the **evidence** is its integral:\n",
    "$$\n",
    "Z_1 = p(D\\mid M_1) = \\iint L(m,b)\\,\\pi(m)\\,\\pi(b)\\,dm\\,db, \\qquad Z_0 = \\int L(0,b)\\,\\pi(b)\\,db .\n",
    "$$\n",
    "The Bayes factor is $B_{10} = Z_1/Z_0$ and the posterior odds are $O_{10} = B_{10}\\times p(M_1)/p(M_0)$.\n",
    "\n",
    "To evaluate $\\chi^2$ quickly on the grid we expand the square: $\\chi^2(m,b) = S_{yy} - 2mS_{xy} - 2bS_y + m^2S_{xx} + 2mbS_x + b^2S$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1106b973",
   "metadata": {},
   "outputs": [],
   "source": [
    "M_GRID = np.linspace(-1.5, 2.5, 801);  DM = M_GRID[1] - M_GRID[0]\n",
    "B_GRID = np.linspace(0.0, 12.0, 961);  DB = B_GRID[1] - B_GRID[0]\n",
    "MM, BB = np.meshgrid(M_GRID, B_GRID, indexing=\"ij\")          # shape (n_m, n_b)\n",
    "LNORM = -0.5 * np.sum(np.log(2 * np.pi * sig**2))              # Gaussian normalisation (cancels in B10)\n",
    "B_LO, B_HI = 0.0, 12.0                                         # prior range of b\n",
    "\n",
    "def chi2_grid(y, m, b):\n",
    "    S, Sx, Sxx = np.sum(w), np.sum(w * x), np.sum(w * x * x)\n",
    "    Sy, Sxy, Syy = np.sum(w * y), np.sum(w * x * y), np.sum(w * y * y)\n",
    "    return Syy - 2 * m * Sxy - 2 * b * Sy + m**2 * Sxx + 2 * m * b * Sx + b**2 * S\n",
    "\n",
    "def log_prior_uniform(v, lo, hi):\n",
    "    return np.where((v >= lo) & (v <= hi), -np.log(hi - lo), -np.inf)\n",
    "\n",
    "def log_Z1_and_post(y, log_prior_m, grid=None):\n",
    "    # log evidence of M1 and the normalised 2-D posterior on the grid.\n",
    "    # log_prior_m: array of log pi(m) on the m grid. Wide uniform priors are fine even if they extend beyond\n",
    "    # the grid, because the likelihood is negligible there; only the normalisation 1/(hi-lo) matters.\n",
    "    # grid: optional (MM, BB, dm, db) to use a coarser grid (simulations)\n",
    "    mm, bb, dm, db = grid if grid is not None else (MM, BB, DM, DB)\n",
    "    lp = LNORM - 0.5 * chi2_grid(y, mm, bb) + log_prior_m[:, None] + log_prior_uniform(bb, B_LO, B_HI)\n",
    "    lZ = logsumexp(lp) + np.log(dm * db)\n",
    "    return lZ, np.exp(lp - logsumexp(lp))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d78bf96c",
   "metadata": {},
   "source": [
    "**TODO 3.** Write `log_Z0(y)`, the log evidence of the flat model (a 1-D integral over $b$ on `B_GRID`, same prior on $b$). Then compute $B_{10}$ for $m \\sim U[0,1]$ and the posterior odds for prior odds 1 : 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "543ddde9",
   "metadata": {},
   "outputs": [],
   "source": [
    "def log_Z0(y):\n",
    "    # TODO: log of  integral L(m=0, b) pi(b) db\n",
    "    ...\n",
    "\n",
    "lZ0 = log_Z0(y)\n",
    "lZ1, post = log_Z1_and_post(y, log_prior_uniform(M_GRID, 0, 1))\n",
    "B10 = ...            # TODO\n",
    "O10 = ...            # TODO (prior odds 1:1)\n",
    "print(f\"ln Z0 = {lZ0:.3f},  ln Z1 = {lZ1:.3f},  B10 = {B10:.1f},  posterior odds = {O10:.1f},  \"\n",
    "      f\"P(M1 | D) = {O10/(1+O10):.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42a534ef",
   "metadata": {},
   "source": [
    "Marginal posterior of the slope, and comparison with the frequentist result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32e2d87c",
   "metadata": {},
   "outputs": [],
   "source": [
    "def summarize_m(post):\n",
    "    pm = post.sum(axis=1) / DM                    # marginal posterior density of m\n",
    "    cdf = np.cumsum(pm) * DM\n",
    "    q16, q50, q84 = np.interp([0.1587, 0.5, 0.8413], cdf, M_GRID)\n",
    "    return pm, q50, q84 - q50, q50 - q16\n",
    "\n",
    "pm, med, ep, em = summarize_m(post)\n",
    "print(f\"Bayesian (m ~ U[0,1]): m = {med:.3f} +{ep:.3f} -{em:.3f}\")\n",
    "print(f\"frequentist          : m = {p1[0]:.3f} +/- {sm1:.3f}\")\n",
    "plt.plot(M_GRID, pm); plt.xlabel(\"m\"); plt.ylabel(\"p(m | D)\"); plt.xlim(-0.5, 1.5);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1847f4e",
   "metadata": {},
   "source": [
    "## 4. The effect of the prior\n",
    "\n",
    "Repeat the Bayesian analysis for wider and wider uniform priors on the slope, $[0,1]$, $[-10,10]$, $[-100,100]$, $[-10^4,10^4]$, and for one **informative** prior: a sceptic who believes slopes are small, $m \\sim N(0, 0.15)$.\n",
    "\n",
    "For each prior: $B_{10}$, $2\\ln B_{10}$, and the posterior of $m$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46eff8be",
   "metadata": {},
   "outputs": [],
   "source": [
    "priors = {\n",
    "    \"U[0, 1]\":          log_prior_uniform(M_GRID, 0, 1),\n",
    "    \"U[-10, 10]\":       log_prior_uniform(M_GRID, -10, 10),\n",
    "    \"U[-100, 100]\":     log_prior_uniform(M_GRID, -100, 100),\n",
    "    \"U[-1e4, 1e4]\":     log_prior_uniform(M_GRID, -1e4, 1e4),\n",
    "    \"N(0, 0.15) sceptic\": stats.norm.logpdf(M_GRID, 0, 0.15),\n",
    "}\n",
    "results = {}\n",
    "print(f\"{'prior on m':20s} {'B10':>9s} {'2 ln B10':>9s}   posterior m\")\n",
    "for name, lpm in priors.items():\n",
    "    lZ1_, post_ = log_Z1_and_post(y, lpm)\n",
    "    _, med_, ep_, em_ = summarize_m(post_)\n",
    "    results[name] = post_\n",
    "    print(f\"{name:20s} {np.exp(lZ1_-lZ0):9.3g} {2*(lZ1_-lZ0):9.2f}   {med_:.3f} +{ep_:.3f} -{em_:.3f}\")\n",
    "print(f\"\\nfor reference: Delta BIC = {bic(chi2_0,1)-bic(chi2_1,2):.2f},  Delta AIC = {aic(chi2_0,1)-aic(chi2_1,2):.2f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8026214d",
   "metadata": {},
   "source": [
    "Posterior contours (68% and 95% credible regions) in the $(m, b)$ plane for each prior:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f5b884e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def credible_levels(post, probs=(0.95, 0.68)):\n",
    "    # density thresholds that enclose the given probability (highest-posterior-density regions)\n",
    "    p = np.sort(post.ravel())[::-1]\n",
    "    c = np.cumsum(p)\n",
    "    return [p[np.searchsorted(c, q)] for q in probs]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(7, 6))\n",
    "colors = [\"C0\", \"C1\", \"C2\", \"C3\", \"k\"]\n",
    "widths = [3.5, 2.5, 1.6, 0.8, 1.5]          # decreasing widths, so that overlapping contours stay visible\n",
    "for (name, post_), col, lw in zip(results.items(), colors, widths):\n",
    "    ax.contour(MM, BB, post_, levels=credible_levels(post_), colors=col, linestyles=[\"--\", \"-\"], linewidths=lw)\n",
    "    ax.plot([], [], color=col, lw=lw, label=name)\n",
    "ax.plot(*p1, \"r*\", ms=11, zorder=5, label=\"best fit (ML)\")\n",
    "ax.axvline(0, color=\"gray\", lw=0.8)\n",
    "ax.set_xlim(-0.3, 1.1); ax.set_ylim(2.2, 6.6)\n",
    "ax.set_xlabel(\"m\"); ax.set_ylabel(\"b\"); ax.legend(fontsize=9)\n",
    "ax.set_title(\"posterior 68% (solid) and 95% (dashed)\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94ffa3ef",
   "metadata": {},
   "source": [
    "*If you only see some of the uniform-prior contours: they lie exactly on top of each other.*\n",
    "\n",
    "**Questions 4.**\n",
    "1. How does $B_{10}$ depend on the width of the uniform prior? Explain with the Occam factor of Lecture 3: $B_{10} \\approx \\dfrac{L_{\\max,1}}{L_{\\max,0}}\\times\\dfrac{\\sqrt{2\\pi}\\,\\sigma_m}{\\Delta m}$. Check the scaling with your numbers.\n",
    "2. Does the width of the uniform prior change the **parameter estimate** of $m$? Why not?\n",
    "3. What does the sceptic's prior change: the estimate, the Bayes factor, or both?\n",
    "4. Is there a \"right\" prior range? What would you write in a paper?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b30ab5fb",
   "metadata": {},
   "source": [
    "## 5. Simulations: how do the criteria behave?\n",
    "\n",
    "Simulate $N_{\\rm sim} = 1000$ data sets from **each** model at its best fit:\n",
    "- truth $M_0$: $y = \\hat b_0 + $ noise\n",
    "- truth $M_1$: $y = \\hat m x + \\hat b + $ noise\n",
    "\n",
    "For every fake data set compute $\\Delta\\chi^2$, $\\Delta$AIC, $\\Delta$BIC and $\\ln B_{10}$ for two priors ($U[0,1]$ and $U[-10,10]$). We use a coarser grid here so that 2000 evidence calculations take well under a minute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63c77fcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "NSIM = 1000\n",
    "m_c = np.linspace(-1.5, 2.5, 161); b_c = np.linspace(0.0, 12.0, 241)\n",
    "MMc, BBc = np.meshgrid(m_c, b_c, indexing=\"ij\")\n",
    "GRID_C = (MMc, BBc, m_c[1] - m_c[0], b_c[1] - b_c[0])\n",
    "lpm_narrow = log_prior_uniform(m_c, 0, 1)\n",
    "lpm_wide = log_prior_uniform(m_c, -10, 10)\n",
    "\n",
    "def all_criteria(y_sim):\n",
    "    _, _, c0 = fit_flat(y_sim)\n",
    "    _, _, c1 = fit_line(y_sim)\n",
    "    lz0 = log_Z0(y_sim)\n",
    "    return (c0 - c1,                                   # Delta chi2\n",
    "            aic(c0, 1) - aic(c1, 2),                   # Delta AIC  (>0 favours M1)\n",
    "            bic(c0, 1) - bic(c1, 2),                   # Delta BIC  (>0 favours M1)\n",
    "            log_Z1_and_post(y_sim, lpm_narrow, GRID_C)[0] - lz0,   # ln B10, m ~ U[0,1]\n",
    "            log_Z1_and_post(y_sim, lpm_wide, GRID_C)[0] - lz0)     # ln B10, m ~ U[-10,10]\n",
    "\n",
    "names = [\"Delta chi2\", \"Delta AIC\", \"Delta BIC\", \"ln B10 U[0,1]\", \"ln B10 U[-10,10]\"]\n",
    "sims = {}\n",
    "for truth, model_y in [(\"M0 true\", lambda: b0 + rng.normal(0, sig)),\n",
    "                       (\"M1 true\", lambda: p1[0] * x + p1[1] + rng.normal(0, sig))]:\n",
    "    sims[truth] = np.array([all_criteria(model_y()) for _ in range(NSIM)])\n",
    "obs = np.array(all_criteria(y))\n",
    "print(\"observed:\", dict(zip(names, np.round(obs, 2))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79edb7d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(1, 5, figsize=(18, 3.6))\n",
    "for j, (ax, nm) in enumerate(zip(axes, names)):\n",
    "    lo = min(sims[\"M0 true\"][:, j].min(), sims[\"M1 true\"][:, j].min())\n",
    "    hi = max(sims[\"M0 true\"][:, j].max(), sims[\"M1 true\"][:, j].max())\n",
    "    for truth, col in [(\"M0 true\", \"C0\"), (\"M1 true\", \"C3\")]:\n",
    "        ax.hist(sims[truth][:, j], bins=40, range=(lo, hi), histtype=\"step\", color=col, label=truth)\n",
    "    ax.axvline(obs[j], color=\"k\", ls=\"--\", label=\"observed\")\n",
    "    ax.set_title(nm)\n",
    "axes[0].legend(fontsize=9)\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e618dff1",
   "metadata": {},
   "source": [
    "**TODO 5.** For each criterion, compute the fraction of simulations in which $M_1$ (the slope) is preferred, for both truths. Use the decision rules:\n",
    "- $\\Delta\\chi^2$: $p < 0.05$ with 1 dof, i.e. $\\Delta\\chi^2 > 3.84$\n",
    "- AIC, BIC: $\\Delta > 0$\n",
    "- Bayes factor: $B_{10} > 3$ (\"positive\" evidence), i.e. $\\ln B_{10} > \\ln 3$\n",
    "\n",
    "NOTES:\n",
    "1. There is nothing \"deep\" about 3.\n",
    "\n",
    "2. What B = 3 means in practice: with equal prior odds, the posterior odds are 3 : 1, so P(M1 | D) = 0.75.\n",
    "\n",
    "3. p = 0.05 is weaker than it sounds. Sellke, Bayarri & Berger (2001) showed that for any reasonable prior, B₁₀ ≤ 1 / (−e · p · ln p). For p = 0.05 that gives B₁₀ ≤ 2.5. So a “significant” p = 0.05 is at best inconclusive on the Bayesian scale.\n",
    "\n",
    "https://hannig.cloudapps.unc.edu/STOR654/handouts/SellkeBayarriBerger2001.pdf\n",
    "\n",
    "Also: under $M_0$, does $\\Delta\\chi^2$ follow $\\chi^2_1$? (Compare the fraction above 3.84 with 0.05.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7600f24a",
   "metadata": {},
   "outputs": [],
   "source": [
    "thresholds = [stats.chi2.isf(0.05, 1), 0.0, 0.0, np.log(3), np.log(3)]\n",
    "print(f\"{'criterion':18s} {'P(choose M1 | M0 true)':>24s} {'P(choose M1 | M1 true)':>24s}\")\n",
    "for j, nm in enumerate(names):\n",
    "    f0 = ...    # TODO: fraction of sims[\"M0 true\"][:, j] above thresholds[j]\n",
    "    f1 = ...    # TODO: same for sims[\"M1 true\"]\n",
    "    print(f\"{nm:18s} {f0:24.3f} {f1:24.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7febd000-4713-4c21-8124-1854e321fe8a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Frequentist and information criteria (binary rule)\n",
      "rule                             P(prefer M1 | M0 true)   P(prefer M1 | M1 true)\n"
     ]
    },
    {
     "ename": "NameError",
     "evalue": "name 'sims' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[0;32mIn[13], line 13\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mrule\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m30s\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mP(prefer M1 | M0 true)\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m>24s\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mP(prefer M1 | M1 true)\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m>24s\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m     12\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m label, j, thr \u001b[38;5;129;01min\u001b[39;00m binary:\n\u001b[0;32m---> 13\u001b[0m     f0 \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39mmean(\u001b[43msims\u001b[49m[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mM0 true\u001b[39m\u001b[38;5;124m\"\u001b[39m][:, j] \u001b[38;5;241m>\u001b[39m thr)\n\u001b[1;32m     14\u001b[0m     f1 \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39mmean(sims[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mM1 true\u001b[39m\u001b[38;5;124m\"\u001b[39m][:, j] \u001b[38;5;241m>\u001b[39m thr)\n\u001b[1;32m     15\u001b[0m     \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mlabel\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m30s\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mf0\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m24.3f\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mf1\u001b[38;5;132;01m:\u001b[39;00m\u001b[38;5;124m24.3f\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n",
      "\u001b[0;31mNameError\u001b[0m: name 'sims' is not defined"
     ]
    }
   ],
   "source": [
    "# --- Section 5: how often does each criterion choose each model? ---------------------------------\n",
    "# Frequentist / information criteria: a binary rule (\"prefer M1\" or \"not\").\n",
    "#   Not preferring M1 does NOT mean choosing M0: a p-value cannot give evidence FOR M0.\n",
    "# Bayes factor: three outcomes (Kass & Raftery):\n",
    "#   favour M1 if B10 > 3, favour M0 if B10 < 1/3, inconclusive in between.\n",
    "\n",
    "print(\"Frequentist and information criteria (binary rule)\")\n",
    "binary = [(\"Delta chi2 > 3.84 (p < 0.05)\", 0, stats.chi2.isf(0.05, 1)),\n",
    "          (\"Delta AIC > 0\",               1, 0.0),\n",
    "          (\"Delta BIC > 0\",               2, 0.0)]\n",
    "print(f\"{'rule':30s} {'P(prefer M1 | M0 true)':>24s} {'P(prefer M1 | M1 true)':>24s}\")\n",
    "for label, j, thr in binary:\n",
    "    f0 = np.mean(sims[\"M0 true\"][:, j] > thr)\n",
    "    f1 = np.mean(sims[\"M1 true\"][:, j] > thr)\n",
    "    print(f\"{label:30s} {f0:24.3f} {f1:24.3f}\")\n",
    "\n",
    "print(\"\\nBayes factor (three outcomes)\")\n",
    "l3, l10 = np.log(3), np.log(10)\n",
    "print(f\"{'prior':12s} {'truth':8s} {'favour M1':>10s} {'inconclusive':>13s} {'favour M0':>10s} {'strong M0':>10s}\")\n",
    "for j, prior in [(3, \"U[0,1]\"), (4, \"U[-10,10]\")]:\n",
    "    for truth in [\"M0 true\", \"M1 true\"]:\n",
    "        lnB = sims[truth][:, j]\n",
    "        f_m1   = np.mean(lnB > l3)              # B10 > 3\n",
    "        f_inc  = np.mean(np.abs(lnB) <= l3)     # 1/3 <= B10 <= 3\n",
    "        f_m0   = np.mean(lnB < -l3)             # B10 < 1/3\n",
    "        f_m0_s = np.mean(lnB < -l10)            # B10 < 1/10  (strong evidence for M0)\n",
    "        print(f\"{prior:12s} {truth:8s} {f_m1:10.3f} {f_inc:13.3f} {f_m0:10.3f} {f_m0_s:10.3f}\")\n",
    "\n",
    "# Visual version: stacked bars of the three Bayesian outcomes\n",
    "fig, ax = plt.subplots(figsize=(7, 3.6))\n",
    "labels, fm1, finc, fm0 = [], [], [], []\n",
    "for j, prior in [(3, \"U[0,1]\"), (4, \"U[-10,10]\")]:\n",
    "    for truth in [\"M0 true\", \"M1 true\"]:\n",
    "        lnB = sims[truth][:, j]\n",
    "        labels.append(f\"{prior}\\n{truth}\")\n",
    "        fm1.append(np.mean(lnB > l3)); finc.append(np.mean(np.abs(lnB) <= l3)); fm0.append(np.mean(lnB < -l3))\n",
    "fm1, finc, fm0 = map(np.array, (fm1, finc, fm0))\n",
    "ax.bar(labels, fm1, color=\"C3\", label=\"favour M1 (B10 > 3)\")\n",
    "ax.bar(labels, finc, bottom=fm1, color=\"lightgray\", label=\"inconclusive\")\n",
    "ax.bar(labels, fm0, bottom=fm1 + finc, color=\"C0\", label=\"favour M0 (B10 < 1/3)\")\n",
    "ax.set_ylabel(\"fraction of simulations\"); ax.set_ylim(0, 1)\n",
    "ax.legend(fontsize=9, loc=\"upper center\", bbox_to_anchor=(0.5, -0.25), ncol=3)\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7dc41580",
   "metadata": {},
   "source": [
    "**Questions 5.**\n",
    "1. Which criterion makes the most \"false detections\" of a slope when $M_0$ is true? Which one misses the slope most often when $M_1$ is true? Relate this to the thresholds on $\\Delta\\chi^2$ implied by each rule.\n",
    "2. Where does the observed data set fall in each histogram?\n",
    "3. Plot $\\ln B_{10}$ (for $U[0,1]$) against $\\Delta\\chi^2$ for the simulations. What relation do you see, and why? How does the wide prior shift it?\n",
    "4. The frequentist answer is a statement about the data under $M_0$; the Bayesian answer is a probability for $M_1$ given the data and the prior. Which question did you actually want answered?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b8a34e2-b401-4d19-957a-bc462841e90f",
   "metadata": {},
   "source": [
    "## 7. The same analysis with public tools: `emcee` and `UltraNest`\n",
    "\n",
    "A grid is fine for 2 parameters, but it scales as (points per axis)^(number of parameters): useless for a\n",
    "10-parameter spectral model. In practice we use **samplers**:\n",
    "\n",
    "| tool | method | gives the posterior | gives the evidence $Z$ |\n",
    "|---|---|---|---|\n",
    "| [`emcee`](https://emcee.readthedocs.io) | MCMC (affine-invariant ensemble sampler) | yes | **no** |\n",
    "| [`UltraNest`](https://johannesbuchner.github.io/UltraNest/) | nested sampling | yes | **yes** (with an error) |\n",
    "\n",
    "Both need the same two ingredients: the **log-likelihood** and the **prior**. They take the prior in different forms:\n",
    "- `emcee` wants $\\ln[\\pi(\\theta)]$, returning $-\\infty$ outside the allowed region;\n",
    "- `UltraNest` wants a **prior transform**: a function mapping a point $u$ of the unit cube $[0,1]^k$ to parameter values\n",
    "  (for a uniform prior $\\theta = \\theta_{\\rm lo} + u(\\theta_{\\rm hi}-\\theta_{\\rm lo})$; for any other prior, the inverse CDF, `ppf`).\n",
    "\n",
    "Install once with: `pip install emcee ultranest corner`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5e11394c-485d-4d62-a1da-7923790f2488",
   "metadata": {},
   "outputs": [],
   "source": [
    "import emcee\n",
    "import ultranest\n",
    "import corner\n",
    "import logging\n",
    "\n",
    "# --- the likelihood (same as before: Gaussian errors, known sigma) ---\n",
    "def loglike_line(theta):\n",
    "    m, b = theta\n",
    "    return LNORM - 0.5 * np.sum(w * (y - m * x - b)**2)\n",
    "\n",
    "def loglike_flat(theta):\n",
    "    b = theta[0]\n",
    "    return LNORM - 0.5 * np.sum(w * (y - b)**2)\n",
    "\n",
    "# --- the priors, written once, used by both tools ---\n",
    "B_PRIOR = (0.0, 12.0)              # b ~ U[0, 12] in both models\n",
    "M_PRIORS = {                       # slope priors to compare\n",
    "    \"U[0, 1]\":            (\"uniform\", 0.0, 1.0),\n",
    "    \"U[-10, 10]\":         (\"uniform\", -10.0, 10.0),\n",
    "    \"N(0, 0.15) sceptic\": (\"normal\", 0.0, 0.15),\n",
    "}\n",
    "\n",
    "def log_prior_m(m, spec):\n",
    "    kind, a1, a2 = spec\n",
    "    if kind == \"uniform\":\n",
    "        return -np.log(a2 - a1) if a1 <= m <= a2 else -np.inf\n",
    "    return stats.norm.logpdf(m, a1, a2)\n",
    "\n",
    "def log_prior_b(b):\n",
    "    lo, hi = B_PRIOR\n",
    "    return -np.log(hi - lo) if lo <= b <= hi else -np.inf"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c18df24-b61f-4369-9547-066401c77271",
   "metadata": {},
   "source": [
    "### 7a. `emcee`: posterior of $(m, b)$ for $m \\sim U[0,1]$\n",
    "\n",
    "32 \"walkers\" start in a small ball around the maximum-likelihood fit and move together through parameter space.\n",
    "We discard the first steps (burn-in), check the autocorrelation time, and keep the rest as posterior samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b0bb070-26e1-461c-ae07-b4b3082164f6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def log_prob_line(theta, spec):\n",
    "    lp = log_prior_m(theta[0], spec) + log_prior_b(theta[1])\n",
    "    return lp + loglike_line(theta) if np.isfinite(lp) else -np.inf\n",
    "\n",
    "spec = M_PRIORS[\"U[0, 1]\"]\n",
    "ndim, nwalkers, nsteps = 2, 32, 4000\n",
    "start = p1 + 1e-3 * np.random.default_rng(1).normal(size=(nwalkers, ndim))   # p1 = (m, b) best fit\n",
    "sampler = emcee.EnsembleSampler(nwalkers, ndim, log_prob_line, args=(spec,))\n",
    "sampler.run_mcmc(start, nsteps, progress=False)\n",
    "\n",
    "tau = sampler.get_autocorr_time(quiet=True)\n",
    "burn = int(5 * tau.max())\n",
    "chain = sampler.get_chain(discard=burn, thin=max(1, int(tau.min() / 2)), flat=True)\n",
    "print(f\"autocorrelation time: {np.round(tau, 1)} steps;  burn-in = {burn};  {len(chain)} samples kept;  \"\n",
    "      f\"acceptance = {np.mean(sampler.acceptance_fraction):.2f}\")\n",
    "for i, nm in enumerate([\"m\", \"b\"]):\n",
    "    q16, q50, q84 = np.percentile(chain[:, i], [15.87, 50, 84.13])\n",
    "    print(f\"emcee   {nm} = {q50:.3f} +{q84-q50:.3f} -{q50-q16:.3f}\")\n",
    "print(f\"grid    m = {summarize_m(results['U[0, 1]'])[1]:.3f}    (Section 3)\")\n",
    "print(f\"ML fit  m = {p1[0]:.3f} +/- {sm1:.3f},  b = {p1[1]:.3f} +/- {sb1:.3f}\")\n",
    "\n",
    "fig = corner.corner(chain, labels=[\"m\", \"b\"], truths=p1, truth_color=\"C3\",\n",
    "                    quantiles=[0.1587, 0.5, 0.8413], show_titles=True, title_fmt=\".3f\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b78c6a0a-df90-4004-bc2d-3180884bfd9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(2, 1, figsize=(9, 4), sharex=True)\n",
    "raw = sampler.get_chain()\n",
    "for i, nm in enumerate([\"m\", \"b\"]):\n",
    "    axes[i].plot(raw[:, :, i], color=\"k\", alpha=0.15, lw=0.5)\n",
    "    axes[i].axvline(burn, color=\"C3\", ls=\"--\"); axes[i].set_ylabel(nm)\n",
    "axes[1].set_xlabel(\"step\"); axes[0].set_title(\"walker traces (red: end of burn-in)\");"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3eafcbe8-60bb-4c6e-bb87-68651dd05698",
   "metadata": {},
   "source": [
    "### 7b. `UltraNest`: evidence and Bayes factor\n",
    "\n",
    "Nested sampling explores the prior from the outside in and accumulates $Z = \\int L\\,\\pi\\,d\\theta$ as it goes.\n",
    "The prior enters **only** through the prior transform — so changing the prior means changing that function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82ccf6a9-9d6a-48a8-8a96-dbeb7aaf2393",
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_transform_line(spec):\n",
    "    kind, a1, a2 = spec\n",
    "    def transform(u):\n",
    "        t = np.empty_like(u)\n",
    "        t[0] = a1 + u[0] * (a2 - a1) if kind == \"uniform\" else stats.norm.ppf(u[0], a1, a2)   # m\n",
    "        t[1] = B_PRIOR[0] + u[1] * (B_PRIOR[1] - B_PRIOR[0])                                   # b\n",
    "        return t\n",
    "    return transform\n",
    "\n",
    "def transform_flat(u):\n",
    "    return B_PRIOR[0] + u * (B_PRIOR[1] - B_PRIOR[0])\n",
    "\n",
    "def run_ultranest(param_names, loglike, transform):\n",
    "    s = ultranest.ReactiveNestedSampler(param_names, loglike, transform)\n",
    "    s.logger.setLevel(logging.WARNING)                  # keep UltraNest quiet\n",
    "    return s.run(min_num_live_points=800, show_status=False, viz_callback=False)\n",
    "\n",
    "res0 = run_ultranest([\"b\"], loglike_flat, transform_flat)\n",
    "print(f\"M0 (flat):  ln Z = {res0['logz']:.3f} +/- {res0['logzerr']:.3f}   (grid: {lZ0:.3f})\")\n",
    "\n",
    "un = {}\n",
    "for name, spec in M_PRIORS.items():\n",
    "    un[name] = run_ultranest([\"m\", \"b\"], loglike_line, make_transform_line(spec))\n",
    "\n",
    "# grid values from Section 4, for comparison\n",
    "grid_lnB = {\"U[0, 1]\":            log_Z1_and_post(y, log_prior_uniform(M_GRID, 0, 1))[0] - lZ0,\n",
    "            \"U[-10, 10]\":         log_Z1_and_post(y, log_prior_uniform(M_GRID, -10, 10))[0] - lZ0,\n",
    "            \"N(0, 0.15) sceptic\": log_Z1_and_post(y, stats.norm.logpdf(M_GRID, 0, 0.15))[0] - lZ0}\n",
    "\n",
    "print(f\"\\n{'prior on m':20s} {'ln Z1 (UltraNest)':>20s} {'ln B10 UltraNest':>18s} {'ln B10 grid':>12s} {'B10':>8s} {'ncall':>8s}   m (UltraNest)\")\n",
    "for name, r in un.items():\n",
    "    lnB = r[\"logz\"] - res0[\"logz\"]\n",
    "    err = np.hypot(r[\"logzerr\"], res0[\"logzerr\"])\n",
    "    ms = r[\"samples\"][:, 0]\n",
    "    q16, q50, q84 = np.percentile(ms, [15.87, 50, 84.13])\n",
    "    print(f\"{name:20s} {r['logz']:12.3f} +/- {r['logzerr']:.3f} {lnB:10.2f} +/- {err:.2f} {grid_lnB[name]:12.2f} \"\n",
    "          f\"{np.exp(lnB):8.2f} {r['ncall']:8d}   {q50:.3f} +{q84-q50:.3f} -{q50-q16:.3f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32d7e32e-13d9-4c1c-99c8-3db87194f555",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(7, 4))\n",
    "bins = np.linspace(-0.3, 1.3, 60)\n",
    "ax.hist(chain[:, 0], bins=bins, density=True, histtype=\"stepfilled\", color=\"lightgray\", label=\"emcee, U[0,1]\")\n",
    "for (name, r), col in zip(un.items(), [\"C0\", \"C1\", \"k\"]):\n",
    "    ax.hist(r[\"samples\"][:, 0], bins=bins, density=True, histtype=\"step\", lw=1.8, color=col, label=f\"UltraNest, {name}\")\n",
    "ax.axvline(p1[0], color=\"C3\", ls=\"--\", label=\"ML fit\")\n",
    "ax.set_xlabel(\"m\"); ax.set_ylabel(\"posterior density\"); ax.legend(fontsize=9);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e63fbec-5897-4d8f-953d-1de8fa803136",
   "metadata": {},
   "source": [
    "**Questions 7.**\n",
    "1. Do `emcee`, `UltraNest` and the grid agree on the posterior of $m$? On $\\ln B_{10}$ (within the quoted error)?\n",
    "2. Why can `emcee` estimate $m$ and $b$ but not decide between $M_0$ and $M_1$?\n",
    "3. In `UltraNest` the prior is a *transform* of the unit cube. Write the transform for a log-uniform (Jeffreys) prior\n",
    "   on a positive parameter, $T \\in [T_{\\min}, T_{\\max}]$ — as in Gregory's line problem (Homework 3).\n",
    "4. Which run took more likelihood evaluations, $U[0,1]$ or $U[-10,10]$? (`r['ncall']`) Why? Relate it to the\n",
    "   information gain $H \\approx \\ln(\\text{prior width}/\\text{posterior width})$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bcb92c81-5272-4c16-b475-809c8141251e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0dbb8cb2-effd-4ed5-bd5d-4e3d8e063d8e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5777cfac-751e-42dd-aa4a-6ae95d197e92",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "43f79db7",
   "metadata": {},
   "source": [
    "## 6. Discussion — back to Gregory's spectral line (Homework 3)\n",
    "\n",
    "No new code: open your Homework 3 notebook next to this one.\n",
    "\n",
    "| | this notebook | Homework 3 (Gregory) |\n",
    "|---|---|---|\n",
    "| simple model $M_0$ | flat line, $m = 0$ | no line, $T = 0$ |\n",
    "| extra parameter | slope $m$ | line strength $T$ |\n",
    "| prior on the extra parameter | $U[0,1]$, $U[-10,10]$, … | $T \\in [0.1, 100]$ mK, uniform or Jeffreys |\n",
    "\n",
    "1. Map the two problems onto each other: which quantity plays the role of $\\Delta\\chi^2$, which of $\\sigma_m$, which of $\\Delta m$?\n",
    "2. In Homework 3 the Bayes factor depended strongly on $T_{\\max}$ for the **uniform** prior but only weakly for the **Jeffreys** prior. Use what you saw in Section 4 to explain why.\n",
    "3. Compute (on paper, from your HW3 numbers) $\\Delta\\chi^2$, ΔAIC and ΔBIC for the line. Do they agree with your Bayes factor? Which one would you trust, and why?\n",
    "4. Gregory's story had **two competing theories**, one predicting the line. How would you choose the prior odds $p(M_1)/p(M_0)$? How small must they be for the data to leave you undecided — and could a frequentist $p$-value answer that question at all?\n",
    "5. One sentence each: when would you report a $p$-value, an AIC/BIC difference, or a Bayes factor?"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
