Comparison with transfer learning and CNN from scratch on a small dataset

Question

I am doing image classification with CNN, and I have a training set of 3200 imges and a training set of 400 images. I have used two different approches for doing this classification : Transfer learnoing and bulding a CNN from scratch.

In the case of transfer learning I get an accuracy of 85%, and in the case of building the network from scratch I get an accuracy of 89%.

But usually with a small dataset shouldn't transfer leanring have better performances?

One thing to consider first is that I used early-stopping as regularization technique, and in the case of transfer learning this stopped the training process at the epoch 18, while for the CNN from scratch I arrived at the epoch 100, which is the number of epochs I imposed for training.

Can someone help me with my doubt? Thanks in advance.

Answer 1

They have better performance for now.

If you continued to re-train your pre-trained NN you would spezialise more and more for your dataset at hand and converge to 89%.

But here is the kicker, if you used some pre-trained models for example based ImageNet dataset you would be able to generalise a lot more better. In other words for any images in the test data that were different from the ones in the train (or from the holdout/new set) you would perform better with the transfer learning.