You could try setting your wallpaper using feh
.
Create a .fehbg
file in your $HOME
and put this in:
feh --bg-scale '/path/to/wallpaper.jpg'
..then save it.
Then in your autostart script just add sh $HOME/.fehbg &
then try restarting your DE. If it still persists after opening Caja
, then I am not sure about that since I am totally unfamiliar with Caja
, rather, I am not familiar with MATE
at all.