You can use this sed probably:
sed -i.bak 's/[^a-zA-Z0-9_-]*//g' file
This removes every character that is not alpha num OR underscore or hyphen.
But I guess you need to elaborate your definition of special character better.
EDIT: As per comments you can use grep:
grep -v '[^a-zA-Z0-9_-]' file > newfile
OR else:
egrep '^[a-zA-Z0-9_-]+$' file > newfile