PointD^ GetPoint();
Value types are not reference types. So you should not use the ^ hat here. Unfortunately this syntax is permitted in C++/CLI, it becomes a boxed value. Very wasteful. There's no direct equivalent for that in C#, beyond emulating it with ValueType as you found out.
Remove the hat to solve your problem.