A natural extension of the decision tree algorithm is an algorithm called “Random Forest”. Random Forest is a technique that uses decision trees to analyze smaller portions of the full dataset in a process of random initialization (“bagging” – random selection of a subset of data) and voting. Different portions of data (or samples in our case) can be separated using different thresholds of various genes.

This algorithm uses multiple instances of decision trees that are applied to portions of the data one at a time. After all of the data is analyzed in this way, the tree predictions are analyzed – and majority voting is accepted. This algorithm is extremely useful for more complex patterns, where smoother borders between classes are needed. Because of this process of multiple decision tree evaluation, the algorithm can also be used to evaluate features. The more a “feature” (or in our case a gene) appears in these decision trees, the better it “performs”. Going through many trees like that, we can evaluate feature significance. This is an important point, because Random Forest can help us identify those features that are most useful for classification accuracy and impurity. We will speak about this further in the next lesson which talks about Feature Selection.