Got it working:
.DATA
_180_PI dq 57.2957795130823208767981548141
.CODE
RadToDegAsm PROC rad: QWORD
LOCAL returnValue: QWORD
movsd xmm0, rad
mulsd xmm0, _180_PI
movsd returnValue, xmm0
fld return
ret
RadToDegAsm ENDP
double value is returned via floating point registers (fld) instruction.