Basically, is it worth-it to reduce the "size" of an image for viewing over the internet if the "size on disk" remains the same?
Yes. The "size on disk" is only relevant for evaluating disk occupancy. To measure how many bytes are transmitted to the wire, the size-on-disk is totally irrelevant, what matters is the real size. So, any reduction in real size attained by a PNG optimizer counts here.