In WPF you can accomplish this as follows:
myRichTextBox.SelectionBrush = System.Windows.Media.Brushes.Yellow; // WPF
myRichTextBox.IsInactiveSelectionHighlightEnabled = true;
Unfortunately, the desired behavior is not possible in Windows Forms (details here). The workaround would be to use a WPF RichTextBox in the Windows Form through ElementHost.
References:
TextBoxBase.SelectionBrush Property (WPF)
TextBoxBase.IsInactiveSelectionHighlightEnabled Property (WPF)
EDIT:
Removed the WinForms solution, because SelectionBackColor
does not provide the desired behavior.