Yes, it's a bad idea to use a mutable object as a dictionary key.
Taking a look at https://stackoverflow.com/a/7941876/400760, leads me to believe there will be unintended consequences, even with a correctly implemented GetHashCode() based on how hash based collections are typically implemented.
It should be safe to use an immutable object as a dictionary key.