The problem is that you zeroed out the specter, it cannot be zeroed out completely it needs to be a smooth curve in the frequency specter in order for it to hold in the time domain. You will probably hear some kind of feedback on top of your sound and that is the time domain not cooping with the sudden stop in the frequency. Mathematically you can see it in the fft i took of the value 1 here in wolfram. 1 is an infinite long function in the time domain and it is always 1. The fft of 1 is a dirac delta function sqrt(2*pi) and that is the same as an infinite stop in the frequency band. As you see the time domain is infinite long when the frequency is infinite short. And that is why you cannot go to a complete stop in the filter you have created.
I suggest you read up on digital filter design if you want more theory. A fast fix is to accept some db ouput on the frequency you are trying to eliminate. Something like the picture here