Module smartcore::naive_bayes
source · [−]Expand description
Supervised learning algorithms based on applying the Bayes theorem with the independence assumptions between predictors
Naive Bayes
Naive Bayes (NB) is a simple but powerful machine learning algorithm. Naive Bayes classifier is based on Bayes’ Theorem with an ssumption of conditional independence between every pair of features given the value of the class variable.
Bayes’ theorem can be written as
\[ P(y | X) = \frac{P(y)P(X| y)}{P(X)} \]
where
- \(X = (x_1,…x_n)\) represents the predictors.
- \(P(y | X)\) is the probability of class y given the data X
- \(P(X| y)\) is the probability of data X given the class y.
- \(P(y)\) is the probability of class y. This is called the prior probability of y.
- \(P(y | X)\) is the probability of the data (regardless of the class value).
The naive conditional independence assumption let us rewrite this equation as
\[ P(y | x_1,…x_n) = \frac{P(y)\prod_{i=1}^nP(x_i|y)}{P(x_1,…x_n)} \]
The denominator can be removed since \(P(x_1,…x_n)\) is constrant for all the entries in the dataset.
\[ P(y | x_1,…x_n) \propto P(y)\prod_{i=1}^nP(x_i|y) \]
To find class y from predictors X we use this equation
\[ y = \underset{y}{argmax} P(y)\prod_{i=1}^nP(x_i|y) \]
References:
Modules
Bernoulli Naive Bayes
Categorical Naive Bayes
Gaussian Naive Bayes
Multinomial Naive Bayes