How best to define a custom action in WiX?

Question

I have a WiX installer and a single custom action (plus undo and rollback) for it which uses a property from the installer. The custom action has to happen after all the files are on the hard disk. It seems that you need 16 entries in the WXS file for this; eight within the root, like so:

<CustomAction Id="SetForRollbackDo" Execute="immediate" Property="RollbackDo" Value="[MYPROP]"/>
<CustomAction Id="RollbackDo" Execute="rollback" BinaryKey="MyDLL" DllEntry="UndoThing" Return="ignore"/>
<CustomAction Id="SetForDo" Execute="immediate" Property="Do" Value="[MYPROP]"/>
<CustomAction Id="Do" Execute="deferred" BinaryKey="MyDLL" DllEntry="DoThing" Return="check"/>
<CustomAction Id="SetForRollbackUndo" Execute="immediate" Property="RollbackUndo" Value="[MYPROP]"/>
<CustomAction Id="RollbackUndo" Execute="rollback" BinaryKey="MyDLL" DllEntry="DoThing" Return="ignore"/>
<CustomAction Id="SetForUndo" Execute="immediate" Property="Undo" Value="[MYPROP]"/>
<CustomAction Id="Undo" Execute="deferred" BinaryKey="MyDLL" DllEntry="UndoThing" Return="check"/>

And eight within the InstallExecuteSequence, like so:

<Custom Action="SetForRollbackDo" After="InstallFiles">REMOVE&lt;>"ALL"</Custom>
<Custom Action="RollbackDo" After="SetForRollbackDo">REMOVE&lt;>"ALL"</Custom>
<Custom Action="SetForDo" After="RollbackDo">REMOVE&lt;>"ALL"</Custom>
<Custom Action="Do" After="SetForDo">REMOVE&lt;>"ALL"</Custom>
<Custom Action="SetForRollbackUndo" After="InstallInitialize">REMOVE="ALL"</Custom>
<Custom Action="RollbackUndo" After="SetForRollbackUndo">REMOVE="ALL"</Custom>
<Custom Action="SetForUndo" After="RollbackUndo">REMOVE="ALL"</Custom>
<Custom Action="Undo" After="SetForUndo">REMOVE="ALL"</Custom>

Is there a better way?

Answer 1

I came across the same problem when writing WiX installers. My approach to the problem is mostly like what Mike suggested and I have a blog post Implementing WiX custom actions part 2: using custom tables.

In short, you can define a custom table for your data:

<CustomTable Id="LocalGroupPermissionTable">
    <Column Id="GroupName" Category="Text" PrimaryKey="yes" Type="string"/>
    <Column Id="ACL" Category="Text" PrimaryKey="no" Type="string"/>
    <Row>
        <Data Column="GroupName">GroupToCreate</Data>
        <Data Column="ACL">SeIncreaseQuotaPrivilege</Data>
    </Row>
</CustomTable>

Then write a single immediate custom action to schedule the deferred, rollback, and commit custom actions:

extern "C" UINT __stdcall ScheduleLocalGroupCreation(MSIHANDLE hInstall)
{
    try {
        ScheduleAction(hInstall,L"SELECT * FROM CreateLocalGroupTable", L"CA.LocalGroupCustomAction.deferred", L"create");
        ScheduleAction(hInstall,L"SELECT * FROM CreateLocalGroupTable", L"CA.LocalGroupCustomAction.rollback", L"create");
    }
    catch( CMsiException & ) {
        return ERROR_INSTALL_FAILURE;
    }
    return ERROR_SUCCESS;
}

The following code shows how to schedule a single custom action. Basically you just open the custom table, read the property you want (you can get the schema of any custom table by calling MsiViewGetColumnInfo()), then format the properties needed into the CustomActionData property (I use the form /propname:value, although you can use anything you want).

void ScheduleAction(MSIHANDLE hInstall,
            const wchar_t *szQueryString,
            const wchar_t *szCustomActionName,
            const wchar_t *szAction)
{
    CTableView view(hInstall,szQueryString);
    PMSIHANDLE record;

    //For each record in the custom action table
    while( view.Fetch(record) ) {
        //get the "GroupName" property
        wchar_t recordBuf[2048] = {0};
        DWORD    dwBufSize(_countof(recordBuf));
        MsiRecordGetString(record, view.GetPropIdx(L"GroupName"), recordBuf, &dwBufSize);

        //Format two properties "GroupName" and "Operation" into
        //the custom action data string.
        CCustomActionDataUtil formatter;
        formatter.addProp(L"GroupName", recordBuf);
        formatter.addProp(L"Operation", szAction );

        //Set the "CustomActionData" property".
        MsiSetProperty(hInstall,szCustomActionName,formatter.GetCustomActionData());

        //Add the custom action into installation script. Each
        //MsiDoAction adds a distinct custom action into the
        //script, so if we have multiple entries in the custom
        //action table, the deferred custom action will be called
        //multiple times.
        nRet = MsiDoAction(hInstall,szCustomActionName);
    }
}

As for implementing the deferred, rollback and commit custom actions, I prefer to use only one function and use MsiGetMode() to distinguish what should be done:

extern "C" UINT __stdcall LocalGroupCustomAction(MSIHANDLE hInstall)
{
    try {
        //Parse the properties from the "CustomActionData" property
        std::map<std::wstring,std::wstring> mapProps;
        {
            wchar_t szBuf[2048]={0};
            DWORD dwBufSize = _countof(szBuf); MsiGetProperty(hInstall,L"CustomActionData",szBuf,&dwBufSize);
            CCustomActionDataUtil::ParseCustomActionData(szBuf,mapProps);
        }

        //Find the "GroupName" and "Operation" property
        std::wstring sGroupName;
        bool bCreate = false;
        std::map<std::wstring,std::wstring>::const_iterator it;
        it = mapProps.find(L"GroupName");
        if( mapProps.end() != it ) sGroupName = it->second;
        it = mapProps.find(L"Operation");
        if( mapProps.end() != it )
            bCreate = wcscmp(it->second.c_str(),L"create") == 0 ? true : false ;

        //Since we know what opeartion to perform, and we know whether it is
        //running rollback, commit or deferred script by MsiGetMode, the
        //implementation is straight forward
        if( MsiGetMode(hInstall,MSIRUNMODE_SCHEDULED) ) {
            if( bCreate )
                CreateLocalGroup(sGroupName.c_str());
            else
                DeleteLocalGroup(sGroupName.c_str());
        }
        else if( MsiGetMode(hInstall,MSIRUNMODE_ROLLBACK) ) {
            if( bCreate )
                DeleteLocalGroup(sGroupName.c_str());
            else
                CreateLocalGroup(sGroupName.c_str());
        }
    }
    catch( CMsiException & ) {
        return ERROR_INSTALL_FAILURE;
    }
    return ERROR_SUCCESS;
}

By using the above technique, for a typical custom action set you can reduce the custom action table to five entries:

<CustomAction Id="CA.ScheduleLocalGroupCreation"
              Return="check"
              Execute="immediate"
              BinaryKey="CustomActionDLL"
              DllEntry="ScheduleLocalGroupCreation"
              HideTarget="yes"/>
<CustomAction Id="CA.ScheduleLocalGroupDeletion"
              Return="check"
              Execute="immediate"
              BinaryKey="CustomActionDLL"
              DllEntry="ScheduleLocalGroupDeletion"
              HideTarget="yes"/>
<CustomAction Id="CA.LocalGroupCustomAction.deferred"
              Return="check"
              Execute="deferred"
              BinaryKey="CustomActionDLL"
              DllEntry="LocalGroupCustomAction"
              HideTarget="yes"/>
<CustomAction Id="CA.LocalGroupCustomAction.commit"
              Return="check"
              Execute="commit"
              BinaryKey="CustomActionDLL"
              DllEntry="LocalGroupCustomAction"
              HideTarget="yes"/>
<CustomAction Id="CA.LocalGroupCustomAction.rollback"
              Return="check"
              Execute="rollback"
              BinaryKey="CustomActionDLL"
              DllEntry="LocalGroupCustomAction"
              HideTarget="yes"/>

And InstallSquence table to only two entries:

<InstallExecuteSequence>
    <Custom Action="CA.ScheduleLocalGroupCreation" 
            After="InstallFiles">
        Not Installed
    </Custom>
    <Custom Action="CA.ScheduleLocalGroupDeletion" 
            After="InstallFiles">
        Installed
    </Custom>
</InstallExecuteSequence>

In addition, with a little effort most of the code can be written to be reused (such as reading from custom table, getting the properties, formatting the needed properties and set to CustomActionData properties), and the entries in the custom action table now is not application specific (the application specific data is written in the custom table), we can put custom action table in a file of its own and just include it in each WiX project.

For the custom action DLL file, since the application data is read from the custom table, we can keep application specific details out of the DLL implementation, so the custom action table can become a library and thus easier to reuse.

This is how currently I write my WiX custom actions, if anyone knows how to improve further I would very appreciate it. :)

(You can also find the complete source code in my blog post, Implementing Wix custom actions part 2: using custom tables.).

Answer 2

The WiX custom actions are a great model to follow. In this case, you only declare, with CustomAction, the immediate action, the deferred action, and the rollback action. You only schedule, with Custom, the immediate action, where the immediate action is implemented as code in a native DLL.

Then, in the immediate action's code, you call MsiDoAction to schedule the rollback and deferred actions: as they are deferred, they are written into the script at the point you call MsiDoAction rather than executed immediately. You'll need to call MsiSetProperty as well to set the custom action data.

Download the WiX source code and study how the IISExtension works, for example. WiX actions generally parse a custom table and generate the data for the deferred action's property based on that table.

Answer 3

If you have complex custom actions that need to support rollback, you might consider writing a Wix extension. Extensions typically provide authoring support (i.e. new XML tags that get mapped to MSI table entries), plus automatic scheduling of custom actions.

It's more work than just writing a custom action, but once your CAs reach a certain level of complexity, the ease-of-authoring that extensions provide can be worth it.