regarding .Net Framework 4.0 Client Profile
As far as i know the .Net runtime can not be included in an Application. The PC you want to execute your Application on needs to have the proper Runtime Version installed, it doesn't matter if its running from USB or not.
Here is a good article about the topic: http://johnhaller.com/useful-stuff/dot-net-portable-apps it basically concludes that you have to chose the .Net version depending on your target OS. for Windows Vista and Windows 7 it would be .Net 2.0, for Windows 8 it would be .Net 4.0
There might be a solution using Portable.Net or Mono but thats a bit above my developer skills :)
regarding redistributables
it really depends on what kind of redists you mean, if its the visual c++ redist you only need to include the right DLL, for other redists im not sure.
Btw Apps running from USB Sticks are usually called 'portable app', if you search for dotnet portable apps you can find quite some information.