Found it after I issued ls -l /proc/self/fd/
. Apparently there is a file descriptor 0
that can be used, it points to the console.
So, first we redirect back to that file descriptor:
exec >&0 2>&1
And now the log file can be safely closed:
exec {logfd}>&-
Ideal for scripts.