From what I learned, this can not be done in Doxygen. I am now considering two authoring systems to do this:
- Doxygen to document the code.
- Open Office / Libre Office to document user manuals, which is where the bulk of the equations are.
Both authoring apps will write HTML output. And then I will combine the output with Qt Help Project system.