How can I force files in my project so that they always copy to output directory

Question 1

You are going to have to parse the project files and check for Content without CopyToOutputDirectory set to Always, I doubt there is another way.

That can be done using whatever scripting language you want, or you could even write a small C# tool that uses the classes from the Microsoft.Build.Evaluation namespace. Here is a possible PowerShell implementation - the hardest part is getting the regexes right. First one checks for Content without any metadata, second one for Content where CopyToOutputDirectory does not start with "A" (which I assume should be "Always", no idea how to match that whole word).

FindBadContentNodes.ps1 :

param([String]$inputDir)

Function FindBadContent()
{
  $lines = Get-Content $input
  $text = [string]::Join( "`n", $lines )
  if( $text -match "<Content Include.*/>" -Or
      $text -match "<Content Include.*`n\s*<CopyToOutputDirectory>[^A]\w*<.*" )
  {
    "Found file with bad content node"
    exit 1
  }
}

Get-ChildItem -Recurse -Include *.csproj -Path $inputDir | FindBadContent

Call this from MsBuild:

<Target Name="FindBadContentNodes">
  <Exec Command="Powershell FindBadContentNodes.ps1 -inputDir path\to\sourceDir"/>
</Target>

Note you mention or better still override these properties during the build. I'd stay away from such a solution: you're just burying the problem and relying on the CI to produce correct builds, so local builds using just VS would not be the same. Imo making the build fail is better, especially since most CI systems have a way of notifying the developper that is responsible anyway so the fix should be applied quickly.

Another possibility would be to have the CI apply the fix and then commit the changes so at least everyone has the correct version.

Question 2

IIRC there is a way in Visual Studio to set a file extension to do certain things on default, much like .config files will always set to content and copy to output directory.

So one could do the same with .sql files (and other files that they would want to be set up this way). A quick search brought me to this: http://blog.andreloker.de/post/2010/07/02/Visual-Studio-default-build-action-for-non-default-file-types.aspx

The relevant parts:

The default build action of a file type can be configured in the registry. However, instead of hacking the registry manually, we use a much better approach: pkgdef files (a good article about pkgdef files). In essence, pkdef are configuration files similar to .reg files that define registry keys and values that are automatically merged into the correct location in the real registry. If the pkgfile is removed, the changes are automatically undone. Thus, you can safely modify the registry without the danger of breaking anything – or at least, it’s easy to undo the damage.

Finally, here’s an example of how to change the default build action of a file type:

1: [$RootKey$\Projects{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}\FileExtensions.spark]

2: "DefaultBuildAction"="Content" The Guid in the key refers to project type. In this case, “{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}” means “C# projects”. A rather comprehensive list of project type guids can be found here. Although it does not cover Visual Studio 2010 explicitly, the Guids apply to the current version as well. By the way, we can use C# as the project type here, because C# based MVC projects are in fact C# projects (and web application projects). For Visual Basic, you’d use “{F184B08F-C81C-45F6-A57F-5ABD9991F28F}” instead.

$RootKey$ is in abstraction of the real registry key that Visual Studio stores the configuration under: HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\10.0_Config (Note: Do not try to manually edit anything under this key as it can be overwritten at any time by Visual Studio).

The rest should be self explanatory: this option sets the default build action of .spark files to “Content”, so those files are included in the publishing process.

All you need to do now is to put this piece of text into a file with the extension pkgdef, put it somewhere under %PROGRAMFILES(x86)%\Microsoft Visual Studio 10.0\Common7\IDE\Extensions (on 64-bit systems) or %PROGRAMFILES(x86)%\Microsoft Visual Studio 10.0\Common7\IDE\Extensions (on 32-bit systems) and Visual Studio will load and apply the settings automatically the next time it starts. To undo the changes, simply remove the files.

Finally, I’ve attached a bunch of pkgdef files that are use in production that define the “Content” default Build Action for C# and VB projects for .spark, .brail, .brailjs and .less files respectively. Download them, save them somewhere in the Extensions folder and you’re good to go.

The author also says that he built a utility to help do all of this for you:

http://tools.andreloker.de/dbag

Question 3

Expanding on @stijn answer, instead of using regex it is far easier to use native xml parsing.

Here is my proposed file, it also supports the ability to customize which files are evaluated by using a regex on the filename only.

param([String]$Path, [string]$IncludeMatch, [switch]$AllowPreserve)

Function Test-BadContentExists
{
    param (
        [parameter(Mandatory=$true,ValueFromPipeline=$true,ValueFromPipelineByPropertyName=$true)]
        [Alias("FullName")]
        [string[]]$Path,
        [string]$IncludeMatch,
        [switch]$AllowPreserve
        )
  [xml]$proj = Get-Content -Path $Path
  $ContentNodes = ($proj | Select-Xml "//Content|//n:Content" -Namespace @{n='http://schemas.microsoft.com/developer/msbuild/2003'}).Node
  if (![string]::IsNullOrEmpty($IncludeMatch)) {
    $ContentNodes = $ContentNodes | Where-Object -Property Include -Match $IncludeMatch
  }
  #remove the always nodes
  $ContentNodes = $ContentNodes | Where-Object -Property CopyToOutputDirectory -ne 'Always'
  #optionally remove the preserve nodes
  if ($AllowPreserve) {
    $ContentNodes = $ContentNodes | Where-Object -Property CopyToOutputDirectory -ne 'PreserveNewest'
  }
  if($ContentNodes)
  {
    write-output "Found file with bad content node:"
    write-output ($ContentNodes | Select-Object Include,CopyToOutputDirectory | sort Include | Out-String)

    exit 1
  }
}

[hashtable]$Options = $PSBoundParameters
[void]$Options.Remove("Path")
Get-ChildItem -Recurse -Include *.csproj -Path $Path | Test-BadContentExists @Options

and calling it, with parameter:

<Target Name="FindBadContentNodes">
  <Exec Command="Powershell FindBadContentNodes.ps1 -inputDir path\to\sourceDir -IncludeMatch '^Upgrade.*\.(sql|xml)$'"/>
</Target>

I ended up using a pre-build event instead and put this ps1 file in my solution directory so i could use it with multiple projects.

echo "Build Dir: %cd%"
echo "Sol Dir: $(SolutionDir)"
echo "Proj Dir: '$(ProjectDir)"
echo.
Powershell -NoProfile -Command "& '$(SolutionDir)\FindBadContentNodes.ps1' -Path '$(ProjectDir)' -IncludeMatch '^Upgrade.*\.(sql|xml)$'"

example build output:

1>  "Build Dir: C:\Source\RPS\MRM BI\MRMBI-Setup\MRMBI-Schema\bin\Debug"
1>  "Sol Dir: C:\Source\RPS\MRM BI\MRMBI-Setup\"
1>  "Proj Dir: 'C:\Source\RPS\MRM BI\MRMBI-Setup\MRMBI-Schema\"
1>  
1>  Found file with bad content node:
1>  
1>  Include                          CopyToOutputDirectory
1>  -------                          ----------------------
1>  Upgrades\V17.09\myfile1.sql                          
1>  Upgrades\V20.05\myfile2.sql      PreserveNewest       
1>