Frequentist vs Baysian- A Never Ending Debate

Frequentist vs Baysian- A Never Ending Debate

19th century statistics was Bayesian while the 20th century was Frequentist, at least from the point of view of most scientific practitioners. The Bayesian-Frequentist debate reflects two different attitudes to the process of doing modeling, both looks quite legitimate.

In simple terms Bayesian statisticians are individual researchers, or a research group, trying to use all information they have to make quickest possible progress. While Frequentist statisticians draw conclusions from sample data by the emphasis on the frequency or proportion of the data only. They do not have any prior knowledge about the data. Hence, in Bayesian we have some prior knowledge while in Frequentist we don’t. You can find a more intuitive example about the difference between the two in layman terms –here.

difference

Let us now get a more statistical understanding for the two approaches.

Some useful notations-

D- Data   H- Hypothesis

Frequentist approach to statistics-

p(D/H) =  Probability of seeing the data given the (null) hypothesis.

Bayesian Approach to statistics-

p(H/D)= Probability of having a certain outcome, given the data we’ve already seen.

Frequentist

Bayesian

  • ·Data are a repeatable random sample  there involves a frequency
  • Data are observed from a realized sample (based on prior)
  • Underlying parameters(mean, covariance) remain constant during this repeatable process.
  • Parameters are unknown and described probabilistically.
  • Parameters are fixed
  • Data are fixed

bays

Prior signifies your initial belief and posterior signifies your updated belief developed after seeing the likelihood of the data with that initial belief.

As now we have a clear understanding of what these two approaches are, let us talk about what are their shortcomings. In general, a strength (weakness) of Frequentist paradigm is a weakness (strength) of Bayesian paradigm.

Bayesian is said to have involved prior beliefs, this prior can be misleading or false leading to erroneous outcomes. Suppose given a fixed set of data two different people can come up with different conclusions about the data based on their prior beliefs. This was seen as a major flaw and gives rise to Frequentist which looks objectively at the data itself.

Now Frequency interpretations are empirical, which are devised from infinite series of trials. Practically we don’t perform infinite trials this makes frequentist ambiguous and incoherent, hence some prior knowledge would help.

End Notes:-

In this article we highlighted the Differences in Bayesian and Frequentist approaches to statistics. Their intuitive understanding, advantages and shortcomings were discussed in detail. Broadly speaking, Bayesian statistics dominated 19th Century statistical practice while the 20th Century was more frequentist. What’s going to happen in the 21st Century? Food for thought.

Thinkpot : Can you suggest some examples where one is better over the other?

Advertisements

One thought on “Frequentist vs Baysian- A Never Ending Debate

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s