Put the whitelist item on top of the ignore item:
!app/design/frontend/base/default/template/mymodule/
app/design/frontend/base/default/template/
If that still doesn't work, you may have to include the template
directory itself, but ignore all files beneath it except mymodule
:
app/design/frontend/base/default/template/*
!app/design/frontend/base/default/template/mymodule/
For more information, see the gitignore man page.