Create a TextBlock on the header with Text="*".
Then bind its visibility with the IsModified property and a booleanToVisibility Converter
EDIT: Binding Visibility
Visibility="{Binding IsModified, Converter={StaticResource booleanToVisibilityConverter}}"
And in your Resources file
<BooleanToVisibilityConverter x:Key="booleanToVisibilityConverter"/>
EDIT2: Don't know if you need this info but just for the case. An approach to the HeaderTemplate
<Setter Property="HeaderTemplate">
<Setter.Value>
<DataTemplate>
<StackPanel>
<TextBlock Text="{Binding Content}" />
<TextBlock Text="*" Visibility="{Binding IsModified, Converter={StaticResource booleanToVisibilityConverter}}"/>
</StackPanel>
</DataTemplate>
</Setter.Value>