You need the set the proper culture info so that the framework can know the correct decimal separator.
Imports System.Threading
Imports System.Globalization
Add the following code in the contructor (Sub New) of your start up form.
Dim ci As New CultureInfo("it-IT")
ci.NumberFormat.NumberDecimalSeparator = "."
Thread.CurrentThread.CurrentCulture = ci
Thread.CurrentThread.CurrentUICulture = ci
Here's a list of culture codes where you can find yours:
http://msdn.microsoft.com/en-us/library/ee825488%28v=cs.20%29.aspx
Edit
Another way is to subscribe to the parse event of the binding.
Dim b As New Binding("Text", TheDataSource, "TheDoubleField")
AddHandler b.Parse, Sub(s As Object, e As ConvertEventArgs) e.Value = CStr(e.Value).Replace(".", ",")
Me.TextBox1.DataBindings.Add(b)