Understanding a Neural Network with Keras (preferably), TensorFlow or PyTorch

Question

I would like to try a technique I saw in an article a while ago where, in order to understand what each neuron is doing, you apply specific inputs to the net and see which one of them is most activated.

Is there a simple way to do it with Keras (preferably)? I can take the weights of each neuron after the NN has been trained and use forward (predict) functions to activate it, but how would I get the activation result of each individual neuron? Is there a function for that?

TensorFlow and PyTorch offer more low level functions so I could do the same thing if I programmed the NN almost from scratch, but even so, does anyone know a simpler way?

If you have tips for this task also, they would be more than welcome. I don't know if functions for this kind of task already exist, but, if they don't, I would say it would be worthwhile creating them, people on the market have complained often to me that the main reason why they are not yet using NNs is that they don't know how to understand the NN's thinking.