- Overview
- Statistics and Machine Learning
- The Impact of Big Data
- Supervised and Unsupervised Learning
- Linear Models and Linear Regression
- Generalized Linear Models
- Generalized Additive Models
- Logistic Regression
- Enhanced Regression
- Survival Analysis
- Decision Tree Learning
- Bayesian Methods
- Neural Networks and Deep Learning
- Support Vector Machines
- Ensemble Learning
- Automated Learning
- Summary
This chapter is from the book
This chapter is from the book
This chapter is from the book
Bayesian Methods
Previously in this chapter, we discussed the value of Bayesian belief networks for exploratory analysis. There are also several techniques for prediction based on Bayesian inference; the most popular of these is the Naïve Bayes Classifier.
The Naïve Bayes Classifier is a Bayesian belief network whose structure is entirely dedicated to the characterization of a target node or response measure. Bayesian theorists call this method “naïve” because it depends on the assumption that all predictor variables are independent of one another; although this is rarely true in practical applications, Naïve Bayes performs very well versus other classifiers. The technique works with an arbitrary number of predictors; it is also computationally simple and easy to implement, which makes it a good choice to use with Big Data.
One disadvantage of Naïve Bayes is its limitation for use with categorical predictors. Some software packages address this problem by automatically converting continuous variables to categorical variables.
Enhanced versions of Naïve Bayes include Augmented Naïve Bayes and Tree Augmented Naïve Bayes, as well as Gaussian and Bernoulli Naïve Bayes. Augmented Naïve Bayes relaxes the assumption of independence among the predictor nodes; it tends to produce more accurate predictions than the generic Naïve Bayes does but requires a time-consuming unsupervised search. The Tree Augmented Naïve Bayes tends to be less accurate than Augmented Naïve Bayes, but it is computationally simpler and runs much faster.
We discussed the concept of Markov blankets earlier in this chapter in the section covering Bayesian belief networks. Although belief networks are exploratory tools, you can develop predictive models from them by designating a target node corresponding to the response measure and then determining the Markov blanket for that node. As with the Naïve Bayes Classifier, an augmented variation on this technique is available; it expands the search base in the under-lying belief network. This tends to produce better predictions but takes more time to run.