Your Project setup should be focussed on the "target" computer, not on compilation computer. If your target system will be 32bits, then you should reference the x86 version of the 3rd party libraries, in other way if your target computer will be 64bits then you should reference the x64 version. You shouldn't/couldn't reference to both libraries into the same Project. In any case you could mantain two parallel versions of your Project which differs just in references section. If you need to create x64 installers then you probably also would check the answer of this question: BadImageFormatException x64 Issue
AnyCPU could be useful when you don't redistribute the 3rd party DLLs, and you asume they were previouly installed on target system, and your development don't need/use any special feature of x64 set. In this case, your code will invoque the 3rd party DLL in the format they were installed.