Bagging Methods

Published:

Bagging, short for bootstrap aggregating, is an ensembling method that trains multiple models on slightly different versions of the same dataset. Each version is created by sampling the training data with replacement, which means some records appear more than once while others are left out. Because each model sees a different sample, the models learn slightly different solutions. At the end, their predictions are combined, usually by averaging for numeric outputs or voting for categories.

Bagging performs best with models that react strongly to small data changes, such as decision trees. Random forests use this idea at scale and show how effective it can be. By averaging predictions across models, bagging smooths out extremes and lowers the risk of overfitting.

Follow us on Facebook and LinkedIn to keep abreast of our latest news and articles