Instead of worrying about the visibility why not set a 'filter' on your grouping as you did for Details3 group? A filter by it's definition should be setting things that are set there like a 'where' clause predicate after the dataset has been evaluated. Either that or take your dataset and completely eliminate out the values you do not want and hard code your dataset to have a predicate of 'where tablix_flag = 2'
There may be more going on if your filter is set on 'Details3' group that is looking to differentiate the different values you could be grouping 'A, B, ' etc... The key is having each grouping be a different set of child values and the 2nd column appears to repeat for value 4 and 5 and column 3 on both have different values yet you mention having them be together.
I would ensure the set is simple to start and has one filter on a parent group to start of just what you want to determine. Set it's filter and if you then get expected results add a parent group to that of extra predicates.