One thing to keep in mind is if the check is marked "To Be Printed" (or "Print Later" in later versions of QuickBooks), you'll need to mod this to false in order for the RefNum to save. I tested using the following request (my TxnID and EditSequence will of course be different):
<?xml version="1.0" encoding="utf-8"?>
<?qbxml version="11.0"?>
<QBXML>
<QBXMLMsgsRq onError = "stopOnError">
<CheckModRq requestID = "0">
<CheckMod>
<TxnID>3CF-1389105873</TxnID>
<EditSequence>1389106557</EditSequence>
<RefNumber>12345678901</RefNumber>
<IsToBePrinted>0</IsToBePrinted>
</CheckMod>
</CheckModRq>
</QBXMLMsgsRq>
</QBXML>