using simple autoencoder for feature selection

Question

I am using a simple autoencoder to extract the informative features and I have multiple Q:

1. I know that the features extracted will be a linear combination of the original features so I consider that the feature that has a larger mean weight (has the highest percentage in the formation of new features) will be important so I will take that features but I don't know if this is true or not

2. the second things is that I want to apply the grid search to find the optimal hyperparameters for the model but I can't do that please if anyone can help me in this and save my life

Answer 1

1. Autoencoders normally aren't linear models. If you make them linear (i.e. you create a shallow Autoencoder with linear activations) then you get exactly a PCA result. The power of Neural Networks is their non-linearity, if you want to stick with linearity go for PCA imho.

2. Keep a Train-Validation-Test set split, and try different configurations of hyperparams checking their performance on Validation data. Alternatively there are many libraries, such as hyperopt, that let you implement more sophisticated Bayesian hyperparameter searches, but unless you want to be published at a conference or win some competition it's a bit overkill. If you're still interested, the internet is plenty of tutorials like this one.