OK, I got to the bottom of this. I'll list the general things to check first in case anyone ever ends up here again, then exactly what my damage was.
In general:
1) If you're not seeing stuff, turn off cull face (glDisable(GL_CULL_FACE)
); this way if you've got the handedness of your polygons the wrong way round, you'll still see something.
2) Be aware that 0,0 for OpenGL is the bottom left, but you've probably decided that 0,0 is the top left for your UI code (this can aid and abet you in giving your triangles the wrong handedness). Here point 13 shows how to avoid this.
3) If you're getting garbage on the screen, you've probably forgotten to clear your buffers.
So, specifically, I'd got some triangles the wrong way round (due to the mismatch between origins in my UI code and the OpenGL code) and, when starting to prune stuff back to the basics to debug, had accidentially commented out the glClear call.
Regarding whether it's possible to no do a matrix transformation for orthoganal code, I feel that it should be, but can't report back; cutting stuff back to basics meant that that code went and, once I'd got it working, I sure wasn't going to put it back!