The issue is precedence, as explained by phlogratos. However, you can't use parenthesis as they spawn a sub-shell and you'll be exiting that shell. For this particular issue, curly braces exist. They have almost the same semantics but they spawn jobs in the current shell.
$ cat a.sh
[[ -f file ]] || { echo error; exit 0; }
echo "ok"
$ touch file
$ ./a.sh
ok
$ rm file
$ ./a.sh
error
$