Question

In my current project, I have only 647 rows (500 for training and 147 for testing) and I have applied the Keras Sequential model using the following code:

from keras import models
from keras import layers
from keras import regularizers
model = models.Sequential()
model.add(layers.Dense(5,activation="relu",input_shape=(train_x.shape[1],)))
model.add(layers.Dense(1,activation="sigmoid"))
from keras import optimizers
#network = model.compile(optimizer=optimizers.RMSprop(lr=0.001),loss="binary_crossentropy",metrics=["accuracy"]) 
network = model.compile(optimizer=optimizers.Adam(lr=0.05, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False),loss="binary_crossentropy",metrics=["accuracy"]) 
result = model.fit(train_x,train_y,epochs=60, batch_size=32)

Following is the result of a few Epochs:

   Epoch 1/60
   500/500 [==============================] - 1s 3ms/step - loss: 0.7253 - acc: 0.5520
   Epoch 2/60 
   500/500 [==============================] - 0s 137us/step - loss: 0.6379 - acc: 0.6640
   Epoch 3/60
   500/500 [==============================] - 0s 134us/step - loss: 0.6035 - acc: 0.6880
   Epoch 4/60
   500/500 [==============================] - 0s 158us/step - loss: 0.5852 - acc: 0.6980
   Epoch 5/60
   500/500 [==============================] - 0s 136us/step - loss: 0.5864 - acc: 0.7140
   Epoch 6/60
   500/500 [==============================] - 0s 134us/step - loss: 0.5552 - acc: 0.7240
   Epoch 7/60
   500/500 [==============================] - 0s 141us/step - loss: 0.5475 - acc: 0.7280
   Epoch 8/60
   500/500 [==============================] - 0s 164us/step - loss: 0.5340 - acc: 0.7460
   Epoch 9/60
   500/500 [==============================] - 0s 138us/step - loss: 0.5389 - acc: 0.7280
   Epoch 10/60
   500/500 [==============================] - 0s 139us/step - loss: 0.5374 - acc: 0.7540
    ===================For the simplicity I am sharing first and last few epochs result=
   Epoch 55/60
   500/500 [==============================] - 0s 161us/step - loss: 0.4947 - acc: 0.7800
   Epoch 56/60
   500/500 [==============================] - 0s 168us/step - loss: 0.5058 - acc: 0.7660
   Epoch 57/60
   500/500 [==============================] - 0s 158us/step - loss: 0.5011 - acc: 0.7700
   Epoch 58/60
   500/500 [==============================] - 0s 154us/step - loss: 0.5062 - acc: 0.7660
   Epoch 59/60
   500/500 [==============================] - 0s 156us/step - loss: 0.5040 - acc: 0.7600
   Epoch 60/60
   500/500 [==============================] - 0s 147us/step - loss: 0.4994 - acc: 0.7800

Using the above configuration (I have also tried different Neural Network architecture, the above one looks fine), I am able to achieve the best accuracy so far and which is train accuracy ~ 78 % and test accuracy ~ 72 %. I also tried with Logistic regression but in this case train accuracy ~ 65 %.

Here it looks like overfitting occurs so I tried with L2 Regularization and Dropout but none of them help to achieve the better accuracy. Unfortunately, I can't generate more data.

What should I do to achieve the better accuracy of the model (given a limited amount of data)? How can I increase the efficiency of both train and test data?

No correct solution

Licensed under: CC-BY-SA with attribution
Not affiliated with datascience.stackexchange
scroll top