What should I do when I find sensitive information in version control?

Question

Today I found what looked to be my supervisor's password in some code in version control. The password is to a database. He is very experienced and has explained before how to avoid having passwords in the source code.

How should I handle this situation? Is there a best practice about preserving evidence of a security flaw, or should I remove such data as soon as I see it?

Even if I remove the data in the current revision, there's still the revision history. Should I just alert my supervisor instead?