Firstly, thank you all for your initiation and inputs to resolve this issue.
Yes, this issue appeared due to mis-understanding of communication between client and server. Server is sending response message in multiple packets - it send data in the order of header, message and tail packet at the end. so my client's - Receive method assumes that it has received full message as soon as header received first and tried de-serialize to an object and failed
When I had given a sleep statement as part of troubleshooting, it was working because it receives full message within this time.
Now, as part of resolution, I have updated Server application to buffer full message and write to network at a time.