If-Modified-Since
The notes of If-Modified-Since warn against using something different than the contents of the Last-Modified
response header:
Note: If a client uses an arbitrary date in the If-Modified-Since header instead of a date taken from the Last-Modified header for the same request, the client should be aware of the fact that this date is interpreted in the server's understanding of time.
Your response doesn't contain a Last-Modified, but even if it did, nothing in the spec says that user agents MUST or SHOULD send If-Modified-Since, they just MAY.
If-None-Match
As to why Safari doesn't send an If-None-Match beats me; your ETag looks valid.
Enforcing Caching in general
Again in general nothing in the spec says you MUST cache, just that when you do, you MUST obey Cache-Control. So it's a bit of an asymmetric relation, you can enforce (transparent) caches to recheck with the origin server, but as an origin server, you can't enforce useragents to cache.
What can I do about it?
Safari is only partly free software. So nothing much, if adding a Last-Modified doesn't help.