There are pretty good upgrade instructions in the docs: http://www.datastax.com/documentation/upgrade/doc/upgrade/datastax_enterprise/upgradeTo4_0.html
You will need to manually upgrade each node, and it will take two upgrades, as you have to first upgrade to the latest DSE 3.2 release (3.2.6 right now), then upgrade to DSE 4.0 (and you should go to the latest which is 4.0.3 right now)