Running your example under ProcMon on my machine with MSBuild 4.5 reveals the following:
- Temporary assembly is generated by MSBuild once per the build under user %TEMP% folder. (It was actually csc.exe that created the assembly, but I think this is just a side effect on how code generation is done).
- The source code was in temporary .cs file, also under %TEMP%.
- By the end of the build all files (source files as well as output assemblies) are deleted.
In other words, you will see a perf hit on first call to the task during the build. All subsequent calls to the task will use cached assembly. After the build the cache is lost, the assembly has to be re-created again, which means that if you are after fast incremental build, you might benefit from pre-compiled DLL.