Question

I am trying to build a majority vote system for 3 Neural Networks, and I came across the concept of Bagging method. Actually, I want to use neural networks as weak learners (I know it's debatable, but some papers have tried it and I want to try it too).

For more information about the voting system I tried to construct/constructed, please read the following thread (The softmax Layer is better the the function in the thread, because the majority function only gives equivalent accuracies of the 3 NNs, but doesn't improve the overall accuracy).

I read that bagging can improve the overall accuracy of the weak learners, but as you can see I only have 3 learners and there aren't any clear information about bagging with Neural Networks. I only found after some reading that I can use ensemble learning with Neural networks by using the output of the trained NNs in a linear fashion. If I want to detail this into steps, I would write:

  1. Divide Dataset into training and validation sets
  2. From the Training sat, construct 3 bootstraps samples ?, Or I need more?
  3. Train/Develop the 3 neural Networks on the 3 bootstraps samples?
  4. Test the Neural Networks on the validation/test set
  5. How can I join decisions of the NNs by bagging?

If possible, I need your insight on these implementation steps and to know if they follow the best practices of bagging with NNs.

PS: I just started reading about bagging and boosting, so I apologize for any conceptual mistakes and contradictions I might have said.

Best regards,

No correct solution

Licensed under: CC-BY-SA with attribution
Not affiliated with datascience.stackexchange
scroll top