Question

So I'm taking N bootstrap samples and training N logistic regression classifiers on these samples. Each classifier gives me some probability of being in a binary class and then I average these N probabilities to get a final prediction.

My question is if I took the N sets of regression coefficients and averaged those and used that averaged set of coefficients in a logistic regression classifier and took the output probability as the final prediction, is this the same as taking the average of the resultant N probabilities as described in the previous paragraph?

Was it helpful?

Solution

The answer is no because the logistic function is non-linear: 1/(1+exp(-a)) + 1/(1+exp(-b)) is not equal to 1/(1+exp(-(a+b))).

But the inverse of the logistic function (also called log-odds), is linear (g(x) in this wiki page). If you are calculating the log-odds, you can average the corresponding coefficients (beta0 and beta1 in the page) in your bagging procedure.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top