Keras Early Stopping: Monitor 'loss' or 'val_loss'?

Question

I often use "early stopping" when I train neural nets, e.g. in Keras:

from keras.callbacks import EarlyStopping

# Define early stopping as callback
early_stopping = EarlyStopping(monitor='loss', patience=5, mode='auto', restore_best_weights=True)

# ...THE MODEL HERE...

# Call early stopping in .fit
history = model.fit_generator(..., callbacks=[early_stopping])

Question: I often wonder if it is better to monitor the loss (monitor='loss') or the validation loss (monitor='val_loss'). Are there some takeaways from the literature? What is best practice?

My intuition would be that monitoring the validation loss gives a more direct feedback of the learning process, since in case the validation loss does not increase any further after some epochs, there seems to be little more to learn.

On the other hand - when I use dropout for instance - the loss will often "lag behind" the validation loss (loss $>$ val_loss). In this case, I believe that there still is something to learn for the model, even if this does not immediately translate into improvements in the validation loss.

Answer 1

Validation loss. Overall training loss should keep decreasing so monitoring it isn't as meaningful. If your issue is noise in the validation loss, increase patience.