I work in Silverlight 4 as well, and I'm using the Toolkit from April 2010. It's specific for SL4, you can find it here: http://silverlight.codeplex.com/releases/view/43528
You can choose to download either the .zip or a msi. I picked the .msi, so your Toolkit will be installed in C:\Program Files (x86)\Microsoft SDKs\Silverlight\v4.0\Toolkit\{release month year}\Bin
, so VS will pick this up automatically. This allows you to add the reference through the interface in VS.
Add the xaml namespace like this:
xmlns:SLToolkit="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Input.Toolkit"
And finally, you can use the contextmenu:
<SLToolkit:ContextMenu>
</SLToolkit:ContextMenu>