Well, first of all, you don't need to escape the parentheses within the character classes and it's no more useful to put more than once the same character in a character class; thus your regex can become like this without any change in functionality:
\(\((((?>[^()]+)|(?R))*)\)\)
I'm not entirely sure why you're using the atomic group either; I could be wrong (and do correct me if I am), but I don't find any difference than using a standard non-capture group.
This said, you can now allow nested single bracket strings in the regex by including another possibility in the regex's inner capture:
\(\(((?:[^()]|((?R))|(\((?:[^()]|(?3))*\)))*)\)\)
1 2 3
Capture groups:
1 - gets the first big match between the outermost (( ... ))
2 - gets the inner matches and any more nested (( ... ))
3 - gets the content between single parens so that recursion is allowed in this group as well. If you don't need it, just ignore it in the array, it's there just to allow nested single parens.