Just to pull the various bits of information together into an answer:
Do you have .pdb files that were created when the program was built? Just rebuilding the same source doesn't work, you need to keep .pdb files for every build. – Alex Farber Feb 20 '13 at 11:05
"The debugger matches information like filename, timestamp and checksum when matching a PDB with a binary (DLL or exe)." (Saikat Sen, Codeproject article)
Following AlexFarber's hint, I copied the source code and release folder from the customer's build onto my computer (the whole build had been saved, including the .pdb and .exe files in the Release folder).
Then I copied the dump file into the release folder, and loaded the Microsoft symbols into the same folder.
Then I opened the dump file again into Visual Studio 2010.
I didn't see anything more helpful than before, which I assume is due to this - see the accepted answer on this SO question. I am using a C# and a .NET 2.0 assembly, and it seems that I can only view source code for a .NET 4.0 assembly.
I will try the WinDbg add-in called sos.dll recommended by Hans Passant on the same question next.
How to get round the need to use the pdb files from the same build:
I think you can get round this requirement for the same .pdb files if you use WinDbg. From virtualdub "If the debugger is stubborn and doesn't want to load symbols because it thinks they don't match, you can use .symopt+ 0x40 to set the "load anything" flag."
Also, if I understand correctly, you can use the chkmatch tool referred to by sergmat above, to match up the .exe and .pdb files.