I've also been looking into how to set the string value of a custom shape data property. Just got it to work like this:
var newPropertyValue = "cool new value";
tstShape.Cells["Prop.SuperCustomPropertyName"].FormulaU = "\"" + newPropertyValue + "\"";
Disclaimer that I am no expert with Visio Automation, but it works in my circumstance. I'm using visio 2010 and studio 2010 Hopefully it helps.