Generally speaking two options exist, you can read about these here -
- Windows Azure Connect let's you configure machine-to-machine VPN and is agent based
- Configuring a private network let's you create a site-to-site VPN, but it typically requires a VPN appliance. I've seen posts on the internet about people managing to configure their router as the VPN gateway but, at least at this point, this is not easy nor supported.
Also worth reading is this overview of Windows Azure Networking features that covers both (and Traffic Manager) by David Chappel