VISERENCE

Understanding Bayesian Updating:
Estimating the Probability of Wearing Glasses

Introduction

In this post, we explore Bayesian updating through a practical example: estimating the probability of people wearing glasses. We use a Bayesian approach to update our beliefs about this probability as we observe more data. Specifically, we use the Beta distribution as a prior and update it with observed data to derive the posterior distribution.

Problem Statement

We aim to estimate the probability 𝜃 that a randomly selected person wears glasses. We are given a binary vector indicating whether each of N randomly selected people wears glasses (1) or not (0).

For example: Observations = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]

Step-by-Step Approach

Initial Setup: Uniform Prior with Beta Distribution

We start with a uniform prior distribution 𝐵(𝛼=1,𝛽=1), which reflects complete uncertainty about the probability .

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import beta

# Initial parameters for uniform Beta distribution
alpha = 1
beta_param = 1

# Function to plot the Beta distribution
def plot_beta_distribution(alpha, beta_param, title):
    theta = np.linspace(0, 1, 100)  # Create a range of theta values from 0 to 1
    distribution = beta.pdf(theta, alpha, beta_param)  # Compute the Beta probability density function for each theta
    plt.plot(theta, distribution, label=f'Beta({alpha}, {beta_param})')
    plt.xlabel('θ')
    plt.ylabel('Density')
    plt.title(title)
    plt.legend()
    plt.show()

# Plot the initial prior
plot_beta_distribution(alpha, beta_param, 'Beta(1, 1) - Initial Prior')

Updating the Beta Distribution Parameters

Next, we define a function to update the Beta distribution parameters based on new observations. This step is essential for performing Bayesian updates as we receive new observations.

# Function to update Beta distribution parameters
def update_beta(alpha, beta_param, x, N):
    alpha_updated = alpha + x  # Update alpha by adding the number of successes (people wearing glasses)
    beta_updated = beta_param + N - x  # Update beta by adding the number of failures (people not wearing glasses)
    return alpha_updated, beta_update

Visualizing the Posterior Distribution After a Single Observation

We then visualize the posterior distribution after observing a single person who wears glasses. This helps us understand how our belief about 𝜃 changes with new data.

# Observations (binary vector indicating people wearing glasses)
observations = [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
N = len(observations)  # Total number of observations

# Posterior after the first observation
alpha_prior = alpha
beta_prior = beta_param
x = sum(observations[:1])  # Sum of first observation (number of successes in the first observation)
alpha_posterior, beta_posterior = update_beta(alpha_prior, beta_prior, x, 1)  # Update the parameters
plot_beta_distribution(alpha_posterior, beta_posterior, 'Posterior after 1 observation')  # Plot the posterior

Updating the Posterior Distribution After Multiple Observations

As we continue observing more people, we update and visualize the posterior distribution after each observation. This step helps us see the effect of accumulating evidence on our belief about 𝜃.

# Posterior after the second observation
x = sum(observations[:2])  # Sum of first two observations
alpha_posterior, beta_posterior = update_beta(alpha_prior, beta_prior, x, 2)  # Update the parameters
plot_beta_distribution(alpha_posterior, beta_posterior, 'Posterior after 2 observations')  # Plot the posterior

Iterative Updates for All Observations

We iteratively update the posterior distribution for each new observation and visualize it. This step-by-step updating process mimics how we would update our beliefs in a real-world scenario as we gather more data.

# Iteratively update for all observations
alpha_current = alpha_prior
beta_current = beta_prior
for i in range(1, N + 1):
    x = sum(observations[:i])  # Sum of observations up to the i-th observation
    alpha_current, beta_current = update_beta(alpha_prior, beta_prior, x, i)  # Update the parameters
    plot_beta_distribution(alpha_current, beta_current, f'Posterior after {i} observations')  # Plot the posterior

Updating the Posterior Distribution All at Once

Finally, instead of updating iteratively, we perform a single update using all observations at once. This demonstrates that the result is the same whether we update step-by-step or all at once.

# Update the posterior all at once
x_total = sum(observations)  # Total number of successes in all observations
alpha_final, beta_final = update_beta(alpha_prior, beta_prior, x_total, N)  # Update the parameters in one step
plot_beta_distribution(alpha_final, beta_final, 'Posterior after all observations at once')  # Plot the final posterior

Why This Approach?

  • Understanding Bayesian Updating: This method illustrates the concept of Bayesian updating, where we start with a prior distribution, update it with data, and derive a posterior distribution.
  • Visualization: By plotting the distribution at each step, we can visually understand how our belief about 𝜃 evolves as more data is observed.
  • Comparison: Updating all at once versus step-by-step allows us to see that both methods yield the same final posterior distribution, reinforcing the correctness of Bayesian updating.

Insights from the Posterior Distribution

  • Posterior After One Observation: The plot shows that after observing one person wearing glasses, higher values of 𝜃 are more likely.
  • Iterative Updates: As we observe more people, the posterior distribution becomes more peaked, indicating increasing confidence in our estimate of 𝜃.
  • All-at-Once Update: The final posterior distribution is the same whether we update step-by-step or all at once, demonstrating the consistency of Bayesian updating.

Conclusion

This exercise demonstrates the power of Bayesian updating in estimating probabilities based on observed data. By starting with a prior distribution and iteratively updating it with new observations, we can refine our beliefs and make informed predictions about future data. The visualization of the posterior distribution at each step provides a clear and intuitive understanding of how our beliefs evolve as more data is observed.

Warenkorb
  • Dein Warenkorb ist leer.