Exact implementation of RandomForest in Weka 3.7

Question

Having reviewed the original Breiman (2001) paper as well as some other board posts, I am slightly confused with the actual procedure used by WEKAs random forest implementation. None of the sources was sufficiently elaborate, many even contradict each other.

How does it work in detail, which steps are carried out?

My understanding till now:

• For each tree a bootstrap sample of the same size as the training data is created
• Only a random subset of the available features of defined size (parameter can be chosen in WEKA) is considered for each node
• Regarding the base tree learner used I found a 2006 post stating is was a modified REPTree.
• Tree is fully grown and not pruned.
• Majority vote is applied (in case of accuracy as performance metric)

My questions:

• Is the bootstrap sampling actually used?
• Is REPTree still in use or has the algorithm been changed since then?

Clarifying these issues would help me a lot!

Answer 1

To answer your questions

1. Bagging (bootstrap aggregation) is indeed used. You can view that in the code on line 529

2. It appears that RandomTree is used, as seen on line 530

This information is from Weka 3.7.5, I believe this is correct all the way back to version 3.6.8, but I have not checked the source code.