Here's a similar question that was posted on SO. Jon seems to echo what I said - you cannot remove a single reference from the context (AppDomain).
I ran into a code segment online that uses this piece of code:
clr.ClearProfilerData() #unload the .dll file so the file lock is released
I'm assuming you want to unload the DLL because you want to recompile it but the compiler is complaining a process is still using it? Perhaps using the method above will release it?
EDIT: Just as a follow-up, if you wanted to research into it more, try out this idea. That is, create an instance of Python in C#, load it in a custom AppDomain, and unload the custom AppDomain when finished. :)