I found this excellent blog article via Google suggestion (which said to use JPEGTRAN): http://blog.stationfour.com/automating-png-jpg-image-optimization-in-windows/
Following these steps, it is possible to add a context menu to explorer in order to reduce the filesize while keeping the quality.
FYI, my files were reduced from 25 KB to 4-8 KB. Awesome!