So what finally worked was I created a control that derived from ViewBox. On the ViewBox I added a "FillBrush" dependency property of type Brush
.
I then just stamp out instances of this control and fill it with Path objects that have the correct Data and bind their Fill property to the FillBrush dependency property.