You can customize your html sidebar in conf.py
.
The default html sidebar consists of 4 templates:
['localtoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html']
In conf.py
you could change localtoc.html to globaltoc.html like this:
html_sidebars = { '**': ['globaltoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html'] }
Since this in the end this will be used in HTML files, this should work on ReadTheDocs.