The answer was pretty obvious. Just do not mention the extension you want to add. for example, instead of using:
add /r /f "\buildserver\buildresults*.pdb" /s "\fileserver\R&D\Artifacts\Symbols\" /c "Adding production symbols" /t "server symbols" /v "1.0.0.1"
use:
add /r /f "\buildserver\buildresults*.*" /s "\fileserver\R&D\Artifacts\Symbols\" /c "Adding production symbols" /t "server symbols" /v "1.0.0.1"