Profile guided optimization with MSBuild

Question

How do I build a solution in MSBuild (command line only) with PGI/PGO without adding new build configuration to projects? I've tried to add command line parameter /property:WholeProgramOptimization=PGInstrument which looks OK, it creates PGD files but does not create PGC after I ran the application. Obviously I'm missing something in MSBuild command line.

Answer 1

I spent two days scratching my head on this problem and I finally found how to do it:

First you need to build the instrumented version of your program:

msbuild.exe
    /t:Rebuild "Letter:\path\YourProject.vcxproj"
    /p:Configuration=Release
    /p:Platform=x64
    /p:WholeProgramOptimization=PGInstrument

Note that On my machine, MSBuild can be found here: C:\Program Files (x86)\MSBuild\12.0\Bin\msbuild.exe. The order of the parameters are really important. If your project were placed after the /p options, it could overwrite them.

A pgd file should have been created in your output directory. You will need its path later.

Then you need to instrument the program:

Letter:\path\OutPutDir\YourProject.exe

In this step obviously you need to add parameters to feed your program with the data it needs. If your program complains about not having access to pgort120.dll you can add a line like this one to your script: set PATH=%PATH%;C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\bin\amd64

Finally you can build the optimized version of your program:

msbuild.exe
    /t:LibLinkOnly "Letter:\path\YourProject.vcxproj"
    /p:Configuration=Release
    /p:Platform=x64
    /p:WholeProgramOptimization=PGOptimize
    /p:LinkTimeCodeGeneration=PGOptimization
    /p:ProfileGuidedDatabase="Letter:\path\OutPutDir\YourProject.pgd"

Here you need to use the address of the pgd file of the first step. Note that the target is LibLinkOnly because there is no need to recompile everything.

Hope this helps others.

Answer 2

Building on the Answer from Arnaud, without which I would never have figured this out, there are additional settings that can be useful. In my solution, I only wanted to build one project out of 20+ with PGO, which required additional flags to prevent all the referenced projects being built:

/p:BuildProjectReferences=false

Use this to stop the projects referenced by the target project being built by the Rebuild target. You need to add this to both the /t:Rebuild command and from VS 16.8.0 onwards to the /t:LibLinkOnly command.

In VS2019, you will find the pgort140.dll buried in the: Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.21.27702\bin\HostxNN\xNN folder, where xNN is x86 or x64 as appropriate.

In the case of a solution, you are likely to have a $(OutDir) output location that depends on solution settings, but that when you build a separate project ends up in the wrong place, so you will all likely need to tell the Project file where the output is with:

/p:OutDir="path to output"

In my case, with a solution file holding lots of projects and one that I need to run with PGO I ended up with a batch file (see below). In this case, at the top level I have a solution file MyApp.sln that contains many sub-projects in sub-folders. The one I want is called subproj and lives in the subprob folder with a project file called subproj.vcxproj in that folder. Each sub-project generates a DLL and there are dependencies from subproj on other sub-projects. The gist of the batch file is as follows:

REM Get the folder we are running out of, assumed to hold the solution file
SET parent=%~dp0

REM Set PLATFORM=x64, TOOLS=HOSTx64\x64 for 64-bits
SET PLATFORM=Win32
SET TOOLS=Hostx86\x86

ECHO Build %PLATFORM% Release of MyApp with PGO of the subprog project
REM Expanding Program Files (x86) causes problems so use PROGRA~2
SET VSPATH=C:\PROGRA~2\Microsoft Visual Studio\2019\Professional
REM Value for VS 16.?.? = \14.21.27702
REM Value for VS 16.3.2 = \14.22.27905\bin
REM Value for VS 16.3.3 = \14.23.28105\bin
REM Value for VS 16.4.0 = \14.24.28314\bin
REM Value for VS 16.5.0 = \14.25.28610
REM Value for VS 16.6.0 = \14.26.28801
REM Value for VS 16.7.0 = \14.27.29110
REM Value for VS 16.8.0 = \14.28.29333
SET VSTOOLS=%VSPATH%\VC\Tools\MSVC\14.21.278.29333\bin
if not exist "%VSTOOLS%" (
ECHO %VSTOOLS% Was not found. This is probably due to a new release. Please
ECHO find the new location and correct this batch file:
ECHO %parent%%me%.bat
exit /b 1
)

REM Set tools path (needed to locate the pgoNnn.dll used for PGO)
set PATH=%PATH%;%VSTOOLS%\%TOOLS%

REM MSB is the path to the MSBuild.exe command
SET MSB="%VSPATH%\MSBuild\Current\Bin\MSBuild.exe"

REM OPT is the common options shared by everything
REM /v: n=normal, m=minimal, q=quiet
SET OPT=/v:m /p:Configuration=Release /p:Platform=%PLATFORM%

REM Set where our output must go. VS likes it to end with \ for $(OutDir)
SET OUTDIR=%parent%%PLATFORM%\Release\

REM It is easier to build everything and rebuild subproj that build all the
REM sub-projects separately.
echo Build the entire solution in the Release build for the desired platform.
%MSB% MyApp.sln %OPT%

echo Now instrument the subproj
%MSB% /t:Rebuild "subproj\subproj.vcxproj" %OPT% /p:OutDir=%OUTDIR% /p:WholeProgramOptimization=PGInstrument /p:BuildProjectReferences=false

echo Run MyApp to exercise the subproj DLL as needed to generate the PGO database
%parent%%PLATFORM%\Release\MyApp.exe arguments as required...

echo Now build PGO optimized version of subproj
%MSB% /t:LibLinkOnly "subproj\subproj.vcxproj" %OPT% /p:WholeProgramOptimization=PGOptimize /p:LinkTimeCodeGeneration=PGOptimization /p:OutDir=%OUTDIR% /p:ProfileGuidedDatabase="%OUTDIR%compiler.pgd"  /p:BuildProjectReferences=false