Random Forest Algorithm: When to Use & How one can Use? [With Pros & Cons]

Data Science encompasses a wide range of algorithms capable of solving problems related to classification. Random forest is usually present at the top of the classification hierarchy. Other algorithms include- Support vector machine, Naive Bias classifier, and Decision Trees.

Before learning about the Random forest algorithm, let’s first understand the basic working of Decision trees and how they can be combined to form a Random Forest. 

Decision Trees

Decision Tree algorithm falls under the category of Supervised learning algorithms. The goal of a decision tree is to predict the class or the value of the target variable based on the rules developed during the training process. Beginning from the root of the tree we compare the value of the root attribute with the data point we wish to classify and on the basis of comparison we jump to the next node.

Moving on, let’s discuss some of the important terms and their significance in dealing with decision trees.

1. Root Node: It is the topmost node of the tree, from where the division takes place to form more homogeneous nodes.
2. Splitting of Data Points: Data points are split in a manner that reduces the standard deviation after the split. 
3. Information Gain: Information gain is the reduction in standard deviation we wish to achieve after the split. More standard deviation reduction means more homogenous nodes. 
4. Entropy: Entropy is the irregularity present in the node after the split has taken place. More homogeneity in the node means less entropy. 

Read: Decision Tree Interview Questions

Need for Random forest algorithm

Decision Tree algorithm is prone to overfitting i.e high accuracy on training data and poor performance on the test data. Two popular methods of preventing overfitting of data are Pruning and Random forest. Pruning refers to a reduction of tree size without affecting the overall accuracy of the tree.

Now let’s discuss the Random forest algorithm.

One major advantage of random forest is its ability to be used both in classification as well as in regression problems. 

As its name suggests, a forest is formed by combining several trees. Similarly, a random forest algorithm combines several machine learning algorithms (Decision trees) to obtain better accuracy. This is also called Ensemble learning. Here low correlation between the models helps generate better accuracy than any of the individual predictions. Even if some trees generate false predictions a majority of them will produce true predictions therefore the overall accuracy of the model increases. 

Random forest algorithms can be implemented in both python and R like other machine learning algorithms.

When to use Random Forest and when to use the other models?

First of all, we need to decide whether the problem is linear or nonlinear. Then, If the problem is linear, we should use Simple Linear Regression in case only a single feature is present, and if we have multiple features we should go with Multiple Linear Regression. However, If the problem is non-linear, we should Polynomial Regression, SVR, Decision Tree, or Random

Forest. Then using very relevant techniques that evaluate the model’s performance such as k-Fold Cross-Validation, Grid Search, or XGBoost we can conclude the right model that solves our problem.

How do I know how many trees I should use?

For any beginner, I would advise determining the number of trees required by experimenting. It usually takes less time than actually using techniques to figure out the best value by tweaking and tuning your model. By experimenting with several values of hyperparameters such as the number of trees. Nevertheless, techniques like cover k-Fold Cross-Validation and Grid Search can be used, which are powerful methods to determine the optimal value of a hyperparameter, like here the number of trees.

Can p-value be used for Random forest?

Here, the p-value will be insignificant in the case of Random forest as they are non-linear models.

Bagging

Decision trees are highly sensitive to the data they are trained on therefore are prone to Overfitting. However, Random forest leverages this issue and allows each tree to randomly sample from the dataset to obtain different tree structures. This process is known as Bagging. 

Bagging does not mean creating a subset of the training data. It simply means that we are still feeding the tree with training data but with size N. Instead of the original data, we take a sample of size N (N data points) with replacement.

Feature Importance

Random forest algorithms allow us to determine the importance of a given feature and its impact on the prediction. It computes the score for each feature after training and scales them in a manner that summing them adds to one. This gives us an idea of which feature to drop as they do not affect the entire prediction process. With lesser features, the model will less likely fall prey to overfitting. 

Hyperparameters

The use of hyperparameters either increases the predictive capability of the model or make the model faster. 

To begin with, the n_estimator parameter is the number of trees the algorithm builds before taking the average prediction. A high value of n_estimator means increased performance with high prediction. However, its high value also reduces the computational time of the model. 

Another hyperparameter is max_features, which is the total number of features the model considers before splitting into subsequent nodes. 

Further, min_sample_leaf is the minimum number of leaves required to split the internal node. 

Lastly, random_state is used to produce a fixed output when a definite value of random_state is chosen along with the same hyperparameters and the training data.

Advantages and Disadvantages of the Random Forest Algorithm

1. Random forest is a very versatile algorithm capable of solving both classification and regression tasks. 
2. Also, the hyperparameters involved are easy to understand and usually, their default values result in good prediction.
3. Random forest solves the issue of overfitting which occurs in decision trees.
4. One limitation of Random forest is, too many trees can make the processing of the algorithm slow thereby making it ineffective for prediction on real-time data.

Also Read: Types of Classification Algorithm

Conclusion

Random forest algorithm is a very powerful algorithm with high accuracy. Its real-life application in fields of investment banking, stock market, and e-commerce websites makes them a very powerful algorithm to use. However, better performance can be achieved by using neural network algorithms but these algorithms, at times, tend to get complex and take more time to develop. 

If you’re interested to learn more about the decision tree, Machine Learning, check out IIIT-B & upGrad’s PG Diploma in Machine Learning & AI which is designed for working professionals and offers 450+ hours of rigorous training, 30+ case studies & assignments, IIIT-B Alumni status, 5+ practical hands-on capstone projects & job assistance with top firms.