The GetValue function of PropertyInfo returns an object, not a string - and here's the misconception in your code. That object gets converted to a string because of the + operator with the first argument there being a string (property.Name), but the CultureInfo is not applied to that string conversion.
The solution is to explicitly use the Convert.ToString(object, IFormatProvider) function, i.e.
yield return property.Name + ":" + Convert.ToString(property.GetValue(vm, System.Reflection.BindingFlags.GetProperty, null, null, System.Globalization.CultureInfo.InvariantCulture), System.Globalization.CultureInfo.InvariantCulture);