Detailed investigation of the issue, in conjunction with the very helpful support staff at GitHub, have lead us to conclude that this was a bug in the older version of TortoiseSVN that I was using when I performed the initial "update" of the external.
This we believe lead to some misconfigured meta-data in the SVN repository which was then being - inappropriately - applied by later, updated SVN clients (the branches folder would be "restored" after deletion even by an alternate, command line client such as SlikSVN).
However, after deleting the externals configuration entirely, re-committing the "external-less" SVN repository, and then re-creating the external references, the branches folder no longer appears, thus we conclude that whatever the bug was and in which-ever component it resided (TortoiseSVN or github SVN API layer) it is now fixed.
If you experience this or a similar issue with an svn:externals referencing a GitHub (or perhaps git generally) repo, then with a fully up-to-date SVN tool chain:
- delete all working copies of the external files
- delete/remove the svn:externals configuration in the SVN properties
- commit the SVN repo in this state
- then re-create your svn:externals
This should solve the problem.