Compare Coefficients of Different Regression Models
-
21-10-2020 - |
سؤال
in my project, I am using asuite of shallow and deep learning models in order to see which has the best performance on my data. However, in the pool of shallow machine learning models, I want to be able to compare the coefficients of each regression model between each other. Example: I have
- Lasso
- Ridge
- ElasticNet
- AdaBoostRegressor
- GradientBoostRegressor
I am aware that I could get the coefficients of Lasso
, Ridge
, and ElasticNet
from model.coef_
and model.intercept_
from sklearn
. However, AdaBoostRegressor
does not have this, but rather, it has weights assigned to each of the estimators. Similarly, GradientBoostingRegressor
of sklearn does not even have weights at all. How am I to compare the coefficients between each of the regressive models I am using??
المحلول
Weights across different types of models are not always comparable, so I think that it would make more sense to do this kind of comparison not across different types of model but within a single type of model varying:
- the hyper-parameters (if any),
- the set of instances (e.g. selecting different subsets randomly),
- the set of features.
I'd recommend in particular varying the instances: if the weight of a feature tends to change a lot depending on the training set it's a sign of overfitting.
If the goal is to compare the importance of some specific features for different models, it's probably more reliable to directly evaluate how the model performs with/without the feature(s).
نصائح أخرى
It's not actually possible to directly compare model coefficients. What you might do that would make more sense is to compare similar metrics.
A good start would be to learn about explainability metrics that are comparable across models : LIME, SHAP... etc. (see here : https://christophm.github.io/interpretable-ml-book/) to see how models reacts on different features.