Configure a custom code action . Keep the following as reference.
public class PreviousVersionCheck extends CustomActionBase{
@Override
public void install(InstallerProxy proxy) throws InstallException {
boolean oldVersionRunning = isOldVersionRunning();//Logic to check if old version running.
proxy.setVariable("$OLD_VERSION_RUNNING$",oldVersionRunning)
}
}
After the custom code action , add a "Show Message Dialoge" .
Add the rule for the action as
$OLD_VERSION_RUNNING$ equals true
In the action properties, you have cancel and exit option,on click of dialouge Button..
Hope this helps..