You need to enable shared memory in mono as Adrian Faciu has mentioned to make your approach work, however this is not the best approach (there's a reason it's disabled by default in the first place, even if I can't remember now exactly why).
I've used two solutions in the past:
A file-based lock. Create a known file, write the pid into that file. At startup in your app check if the file exists, and if it exists read the pid and check if there are any running processes with that pid (so that it can recover from crashes). And delete the file upon exit (in the instance that created it in the first place). The drawback is that there is a race condition at startup if several instances are launched pretty much at the same time. You can improve this with file locking, but you may have to use P/Invokes to do the proper file locking on Linux (I'm not entirely sure the managed API would do what you'd expect).
A socked-based lock. Open a known port. The advantage over the above is that you don't need to do any cleanup and there are no race conditions. The drawback is that you need a fixed/known port, and some other program might happen to use that exact port at the same time.