Eventually I believe that my current implementation should work with no issues (as long as Microsoft won't change the UdpClient class implementation).
For whom it might be interesting: http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/cbdd9818-00f0-499f-a935-d8e555899d64