It may really be a bug as pointed out by @tcaswell in the comment but a workaround is to use text() in axis coords:
plt.clf()
samplevalues = [100,1000,5000,10^4]
ax = plt.subplot(111)
ax.loglog(samplevalues,samplevalues,'o',color='black')
ax.text(0.5, 0.5,'hi',transform=ax.transAxes)
plt.show()
Another approach is to use figtext() but that is more cumbersome to use if there are already several plots (panels).
By the way, in the code above, I plotted the data using log-log scale directly. That is, instead of:
ax.plot(samplevalues,samplevalues,'o',color='black')
ax.set_xscale('log')
ax.set_yscale('log')
I did:
ax.loglog(samplevalues,samplevalues,'o',color='black')