Accuracy noise patterns during model training

Question

I'm training a logistic regression model on a small dataset. I have about 1300 samples that I split into a training and a testing set (70% and 30% respectively).

The training seems ok, however when I plot the accuracy of my model w.r.t. the epoch, some repeating noisy patterns appear at the end well after the accuracy is stabilised (after 800 epochs, see images below).

The training is done with an Adam optimizer. I'm using a learning rate of 0.04 and a weight decay of 0.07 that I found after doing a random search.

Is it something that may happen when training and is without consequence or does it reflect an issue with my data / training / software implementation ?