You’re not all that far off: your main difficulty is how to test for whether or not the file is currently hidden. (Though your bash syntax is wrong, and a Finder-based service to toggle visibility only half makes sense -- how are you going to select a file that’s invisible?) That said, this script will work [1], given that you’re running it with bash and passing the input as arguments:
for f in "$@"
do
if ( ls -lO "$f" | grep -wq hidden )
then
chflags nohidden "$f"
else
chflags hidden "$f"
fi
done
Killing the Finder afterwards is not necessary; it will notice changes to “hidden” automatically. If you create an Automator “service” workflow, you don’t need the “Get Selected Finder Items” action at the beginning; your service will be passed the selected items as input.
[1] Mostly. It won’t work correctly on files that have “hidden” in their name. Fixing this, perhaps by using stat
(1), is left as an exercise for the reader.