P.O.D. (plain-old-data) structs are generally fairly easy to marshal.
But if you want to marshal complex C++ classes that contain things like vectors, then you'll run into trouble.
If you can write C/C++ code to transform complex classes into simpler types for calling via p/invoke then that's ok - but otherwise, run away from complex C++ classes.
However, there is another possibility.
You can use the so-called "It just works" technology to wrap C++ code using a CLI C++ class. You can mix unmanaged and CLI code in C++ - even in the same file - which can really help.
See here for more details:
http://msdn.microsoft.com/en-us/library/ms173185.aspx
http://www.windowsdevcenter.com/pub/a/dotnet/2003/03/03/mcppp2.html (this is rather old)
http://www.codeproject.com/Articles/651516/Exposing-native-to-managed-Cplusplus-CLI-vs-P-Invo