Setting activation function to a leaky relu in a Sequential model

Question

I'm doing a beginner's TensorFlow course, we are given a mini-project about predicting the MNIST data set (hand written digits) and we have to finish the code such that we get a 99% accuracy (measured as 0.01 loss) in less than 10 epochs. In principle I am getting the accuracy, but the loss only reaches <0.01 at the 10th epoch (hence assignment is counted as failed). As per instructions, I'm not allowed to change the model.compile arguments, so I decided I can try to change the activation function to a leaky relu, using the code I was given. It is not as straightforward as it seems and everything I found online does not seem to work. Most suggestions are in the model.add() format, which I can't figure out how to incorporate/substitute in the code they provided us, without changing it too much (everything failed). The current code is given below:

model = tf.keras.models.Sequential([ keras.layers.Flatten(input_shape=(28,28)), keras.layers.Dense(128,activation=tf.nn.relu), keras.layers.Dense(10,activation=tf.nn.softmax) ])

Any help would be appreciated!

Answer 1

Try this, and tune alpha.

model = tf.keras.models.Sequential([ 
    keras.layers.Flatten(input_shape=(28,28)), 
    keras.layers.Dense(128,activation=tf.keras.layers.LeakyReLU(alpha=0.3)),
    keras.layers.Dense(10,activation=tf.nn.softmax)
 ])