Does it mean if I put 0 [symmetry breaking] is disabled?
Yes
if I put a higher value does it avoid symmetry?
Yes, the best it can.
Does symmetry means 2 instances that have equivalent adjacency matrix?
I don't know what you mean by "adjacency matrix", but in any case, the answer is likely to be "not necessarily". Symmetry breaking is just a heuristic; it is implemented at a level lower than the Alloy AST, meaning that some symmetries that make sense at a high level of your domain model are not necessarily automatically detected and broken by the Alloy Analyzer.
In order to reduce the solving complexity, is there a way to specify to the solver that we are not interested in some specific signature instances permutation or relation permutation but in their architecture configuration?
I don't think that can be readily done using Alloy.