Try with plot(map, raster=FALSE)
This will be slower than if raster=TRUE
was actually working.
The OpenStreetMap package uses the raster package to plot things. Some graphics devices have special code for drawing rasters like images and tiles, and the system should automatically figure out if it can use it. Sometimes though, the package passes raster=TRUE
to the plotting function to override, and if it does this then if your system doesn't have the raster capability then you get empty rasters.
I've reported things like this to the raster maintainer and changes have been made, so unless you are already upgraded to the latest versions I'd suggest doing so.