Module smartcore::tree

Supervised tree-based learning methods

Classification and regression trees

Tree-based methods are simple, nonparametric and useful algorithms in machine learning that are easy to understand and interpret.

Decision trees recursively partition the predictor space \(X\) into k distinct and non-overlapping rectangular regions \(R_1, R_2,…, R_k\) and fit a simple prediction model within each region. In order to make a prediction for a given observation, \(\hat{y}\) decision tree typically use the mean or the mode of the training observations in the region \(R_j\) to which it belongs.

Decision trees suffer from high variance and often does not deliver best prediction accuracy when compared to other supervised learning approaches, such as linear and logistic regression. Hence some techniques such as Random Forests use more than one decision tree to improve performance of the algorithm.

smartcore uses CART learning technique to build both classification and regression trees.

References:

• “Classification and regression trees”, Breiman, L, Friedman, J H, Olshen, R A, and Stone, C J, 1984
• “An Introduction to Statistical Learning”, James G., Witten D., Hastie T., Tibshirani R., Chapter 8

Modules

• decision_tree_classifier
  Classification tree for dependent variables that take a finite number of unordered values.
• decision_tree_regressor
  Regression tree for for dependent variables that take continuous or ordered discrete values.