Yes, you can simply choose to clone only one branch from the SVN repo:
git svn clone http://svn.example.com/project/mostRecentBranch
This will create a git repo with a single branch, which represents the history of "mostRecentBranch" from the SVN repo.
git svn
will try to also include history from before the branch's creation (i.e. the history of the branch that "mostRecentBranch" was copied/created from). However, this may not work if the branch was not created properly in SVN (i.e. if it was not created using "svn copy").