..., unsigned char **outputBuf, ...
There is a pretty serious problem with this function, it cannot reliably be called from a C program either. The caller needs to release the output buffer after using it. That requires using the exact same allocator as was used in your C code. That's very hard to guarantee in a C program and often goes wrong badly. Like when the user of your DLL doesn't use the exact same compiler version as you used. He'll use a different version of the C runtime library, one that uses his own heap. So cannot possibly release the buffer since he doesn't have a handle to the heap that you used.
It becomes next to impossible when you pinvoke, the CLR of course has absolutely no idea what C runtime version you used and is guaranteed to not use the same one that you used since it has its own private copy.
The reason you are just getting one byte is related to this problem, the pinvoke marshaller doesn't know how large the array is since it didn't create the array. That's fixable you need to apply tbe [MarshalAs(UnmanagedType.LPArray), SizeParamIndex = 3] attribute to the parameter. That tells the pinvoke marshaller that the 4th argument contains the size of the array.
You'll need to address the memory management problem, you can't leave it the way it is since you'll leak memory badly on XP and hard-crash on Vista and higher when the pinvoke marshaller tries to release the array. You need to alter the C code to use memory for the returned output buffer that is allocated on a known heap. Which requires using CoTaskMemAlloc().
Another possible fix is that you export a function that allows the caller to release the buffer. In which case you should declare the argument out IntPtr
and marshal yourself with, say, Marshal.Copy(), then release the buffer with your added function.
Or a completely different way to use the function, like having Compress() only compress but not return the data yet. With extra functions that the client code can use to discover the required buffer size and to obtain a copy of the data.