Random forests are an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Random forests correct for decision trees' habit of overfitting to their training set.

The algorithm for inducing a random forest was developed by Leo Breiman[1] and Adele Cutler,and "Random Forests" is their trademark. The method combines Breiman's "bagging" idea and the random selection of features, introduced independently by Ho and Amit and Geman in order to construct a collection of decision trees with controlled variance.

The selection of a random subset of features is an example of the random subspace method, which, in Ho's formulation, is a way to implement classification proposed by Eugene Kleinberg.

• 1 History
• 2 Algorithm
  
  • 2.1 Preliminaries: decision tree learning
  • 2.2 Tree bagging
  • 2.3 From bagging to random forests
  • 2.4 Extensions
    
• 3 Properties
  
  • 3.1 Variable importance
  • 3.2 Relationship to nearest neighbors
    
• 4 Unsupervised learning with random forests
• 5 Variants
• 6 See also
• 7 References
• 8 External links
  

Reference

• Machine Learning in Python: Essential Techniques for Predictive Analysis Wiley