Custom Panel InternalChildren order differs from bound ItemsSource

Question

I'm writing a custom ItemsControl and Panel that arranges items vertically in the order in which they were added to the ItemsSource bound to the ItemsControl. Note that this is just a prototype of the final Panel, whose arrangement will be a bit more complex. As such I'm not interested in alternate Panel suggestions.

The ItemsControl trickle-feeds the Panel items from the bound collection, so items in the collection don't all appear "simultaneously" (the Panel raises an event to say it is ready, which the ItemsControl captures to release the next item). The problem is that for some reason the ArrangeOverride on the Panel sometimes decides items should be added in the middle of the already rendered visuals, causing things to jump around.

At the moment I'm simply clicking an Add button on my test view to add items to the end of the bound ItemsSource collection. So items can be added/removed from the bound collection while this trickle feeding is happening. By the time the Panel renders these "new" items, they are being added in seemingly random locations.

I've got Trace.Writes all over the code so that I can see that the items are successfully added to the end of the collection, and to confirm that the InternalChildren are being randomly inserted in the middle. I even went as far as implementing a CollectionViewSource to enforce order on the items. Even then the InternalChildren was giving a different order to the underlying ItemsSource.

The only thing I can think of is that somehow adding items during the trickle-feeding is causing some sort of race condition, but it's all on the UI thread and I still can't understand why the order would be correct on the ItemsControl but not on the Panel.

How can I synchronise the order of the InternalChildren on the Panel with the bound ItemsControl so that the Visuals are displayed in the correct order?

Update

As requested, here's some code. There's a lot of it in the full solution so I'll try to only post the relevant bits here. As such this code won't run, but it should give you an idea. I've removed all the Trace.WriteLine code, and a lot of the additional code I don't think is important for solving the problem at hand.

I have a StaggeredReleaseCollection<T> which extends ObservableCollection<T>. Items added to the collection are held in a seperate "HeldItems" collection until they are ready to be moved into the inherited "Items" collection by way of a "Kick" method (on IFlushableCollection).

public class StaggeredReleaseCollection<T> : ObservableCollection<T>, IFlushableCollection
    {
        public event EventHandler<PreviewEventArgs> PreviewKick;
        public event EventHandler HeldItemsEmptied;

        ExtendedObservableCollection<T> _heldItems;
        ReadOnlyObservableCollection<T> _readOnlyHeldItems;

        public StaggeredReleaseCollection()
        {
            //Initialise data
            _heldItems = new ExtendedObservableCollection<T>();
            _readOnlyHeldItems = new ReadOnlyObservableCollection<T>(_heldItems);

            _heldItems.CollectionChanged += (s, e) =>
            {
                //Check if held items is being emptied
                if (e.Action == NotifyCollectionChangedAction.Remove && !_heldItems.Any())
                {
                    //Raise event if required
                    if (HeldItemsEmptied != null) HeldItemsEmptied(this, new EventArgs());
                }
            };
        }

        /// <summary>
        /// Kick's the first held item into the Items collection (if there is one)
        /// </summary>
        public void Kick()
        {
            if (_heldItems.Any())
            {
                //Fire preview event
                if (PreviewKick != null)
                {
                    PreviewEventArgs args = new PreviewEventArgs();
                    PreviewKick(this, args);
                    if (args.IsHandled) return;
                }

                //Move held item to Items
                T item = _heldItems[0];
                _heldItems.RemoveAt(0);
                Items.Add(item);

                //Notify that an item was added
                OnCollectionChanged(new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Add, item));
            }
        }
    }

I also have a VerticalStackFlushPanel which is the prototype Panel I'm building. This Panel should lay all the items vertically down it's surface. When an item is added, a Phase1 animation begins. When this is complete an event is raised so that the next item can be added.

public class VerticalStackFlushPanel : FlushPanel
{
    /// <summary>
    /// Layout vertically
    /// </summary>
    protected override Size MeasureOverride(Size availableSize)
    {
        Size desiredSize = new Size();
        for (int i = 0; i < InternalChildren.Count; i++)
        {
            UIElement uie = InternalChildren[i];
            uie.Measure(availableSize);
            desiredSize.Height += uie.DesiredSize.Height;
        }
        return desiredSize;
    }

    /// <summary>
    /// Arrange the child elements to their final position
    /// </summary>
    protected override Size ArrangeOverride(Size finalSize)
    {
        double top = 0d;
        for (int i = 0; i < InternalChildren.Count; i++)
        {
            UIElement uie = InternalChildren[i];
            uie.Arrange(new Rect(0D, top, finalSize.Width, uie.DesiredSize.Height));
            top += uie.DesiredSize.Height;
        }
        return finalSize;
    }

    public override void BeginPhase1Animation(DependencyObject visualAdded)
    {
        //Generate animation
        var da = new DoubleAnimation()
        {
            From = 0d,
            To = 1d,
            Duration = new Duration(TimeSpan.FromSeconds(1)),
        };

        //Attach completion handler
        AttachPhase1AnimationCompletionHander(visualAdded, da);

        //Start animation
        (visualAdded as IAnimatable).BeginAnimation(OpacityProperty, da);
    }

    public override void BeginPhase2Animation(DependencyObject visualAdded)
    {
        TextBlock tb = FindVisualChild<TextBlock>(visualAdded);
        if (tb != null)
        {
            //Generate animation
            var ca = new ColorAnimation(Colors.Red, new Duration(TimeSpan.FromSeconds(0.5)));
            SolidColorBrush b = new SolidColorBrush(Colors.Black);

            //Set foreground
            tb.Foreground = b;

            //Start animation
            b.BeginAnimation(SolidColorBrush.ColorProperty, ca);

            //Generate second animation
            AnimateTransformations(tb);
        }
    }
}

The abstract FlushPanel on which VerticalStackFlushPanel is based handles the raising of the Phase 1 animation event. For some reason, OnVisualChildrenChanged doesn't fire during the Kick() method the StaggeredReleaseCollection unless I explicitly raise the OnCollectionChanged event myself (maybe that's a red flag?).

public abstract class FlushPanel : Panel
{

    /// <summary>
    /// An event that is fired when phase 1 of an animation is complete
    /// </summary>
    public event EventHandler<EventArgs<object>> ItemAnimationPhase1Complete;

    /// <summary>
    /// Invoked when the <see cref="T:System.Windows.Media.VisualCollection"/> of a visual object is modified.
    /// </summary>
    /// <param name="visualAdded">The <see cref="T:System.Windows.Media.Visual"/> that was added to the collection.</param>
    /// <param name="visualRemoved">The <see cref="T:System.Windows.Media.Visual"/> that was removed from the collection.</param>
    protected override void OnVisualChildrenChanged(DependencyObject visualAdded, DependencyObject visualRemoved)
    {
        base.OnVisualChildrenChanged(visualAdded, visualRemoved);

        if (visualAdded != null && visualAdded is IAnimatable) BeginPhase1Animation(visualAdded);
    }

    /// <summary>
    /// Begin an animation for Phase 1.  Use <seealso cref="AttachPhase1AnimationCompletionHander"/> to attach the completed event handler before the animation is started.
    /// </summary>
    /// <returns>An animation that can be used to determine Phase 1 animation is complete</returns>
    public abstract void BeginPhase1Animation(DependencyObject visualAdded);

    /// <summary>
    /// Generate an animation for Phase 2
    /// </summary>
    /// <returns>An animation that can be used to determine Phase 2 animation is complete</returns>
    public abstract void BeginPhase2Animation(DependencyObject visualAdded);

    /// <summary>
    /// Attaches an animation completion handler for the Phase 1 Animation that fires an event when the animation is complete.
    /// </summary>
    /// <remarks>
    /// This event is for when this panel is used on the <see cref="StaggeredReleaseItemsControl"/>, which uses it to kick the next item onto the panel.
    /// </remarks>
    public void AttachPhase1AnimationCompletionHander(DependencyObject visualAdded, AnimationTimeline animation)
    {
        if (animation != null) animation.Completed += (s, e) =>
        {
            //Raise event
            if (ItemAnimationPhase1Complete != null) ItemAnimationPhase1Complete(this, new EventArgs<object>(visualAdded));

            //Start next phase
            BeginPhase2Animation(visualAdded);
        };
    }
}

The StaggeredReleaseItemsControl knows how to handle IFlushableCollection and FlushPanel (which StaggeredReleaseCollection<T> and VerticalStackFlushPanel are based on). If it finds instances of these at runtime, it coordinates kicking items from the StaggeredReleaseCollection<T> into the VerticalStackFlushPanel, waits for the Phase1 animation to complete, then kicks the next item etc.

It usually prevents new items from being kicked before the end of a Phase1 animation, however I've disabled that part to speed up testing.

public class StaggeredReleaseItemsControl : ItemsControl
{
    FlushPanel _flushPanel;
    IFlushableCollection _collection;

    /// <summary>
    /// A flag to track when a Phase 1 animation is underway, to prevent kicking new items
    /// </summary>
    bool _isItemAnimationPhase1InProgress;

    static StaggeredReleaseItemsControl()
    {
        DefaultStyleKeyProperty.OverrideMetadata(typeof(StaggeredReleaseItemsControl), new FrameworkPropertyMetadata(typeof(StaggeredReleaseItemsControl)));
    }

    public override void OnApplyTemplate()
    {
        _flushPanel = FindVisualChild<FlushPanel>(this);
        if (_flushPanel != null)
        {
            //Capture when Phase 1 animation is completed
            _flushPanel.ItemAnimationPhase1Complete += (s, e) =>
            {
                _isItemAnimationPhase1InProgress = false;

                //Kick collection so next item falls out (and starts it's own Phase 1 animation)
                if (_collection != null) _collection.Kick();
            };
        }
        base.OnApplyTemplate();
    }

    protected override void OnItemsSourceChanged(System.Collections.IEnumerable oldValue, System.Collections.IEnumerable newValue)
    {
        base.OnItemsSourceChanged(oldValue, newValue);

        //Grab reference to collection
        if (newValue is IFlushableCollection)
        {
            //Grab collection
            _collection = newValue as IFlushableCollection;

            if (_collection != null)
            {
                //NOTE:
                //Commented out to speed up testing
                ////Capture preview kick event
                //_collection.PreviewKick += (s, e) =>
                //{
                //    if (e.IsHandled) return;

                //    //Swallow Kick if there is already a Phase 1 animation in progress
                //    e.IsHandled = _isItemAnimationPhase1InProgress;

                //    //Set flag
                //    _isItemAnimationPhase1InProgress = true;
                //};

                //Capture held items empty event
                _collection.HeldItemsEmptied += (s, e) =>
                {
                    _isItemAnimationPhase1InProgress = false;
                };

                //Kickstart (if required)
                if (AutoKickStart) _collection.Kick();

            }
        }
    }
}

}

The Generic.xaml file puts together a standard template.

<ResourceDictionary
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:si="clr-namespace:AnimatedQueueTest2010.StaggeredItemControlTest.Controls"
>
    <!--StaggeredReleaseItemControl Style-->
    <Style TargetType="{x:Type si:StaggeredReleaseItemsControl}" BasedOn="{StaticResource {x:Type ItemsControl}}">
        <Setter Property="FontSize" Value="20" />
        <Setter Property="ItemsPanel">
            <Setter.Value>
                <ItemsPanelTemplate>
                    <si:VerticalStackFlushPanel/>
                </ItemsPanelTemplate>
            </Setter.Value>
        </Setter>
    </Style>

</ResourceDictionary>

My test view is fairly simple.

<Window 
    x:Class="AnimatedQueueTest2010.StaggeredItemControlTest.Views.StaggeredItemControlTestView"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:AnimatedQueueTest2010.StaggeredItemControlTest.Controls"
    xmlns:cm="clr-namespace:System.ComponentModel;assembly=WindowsBase"
    Title="StaggeredItemControlTestView" 
    Width="640" Height="480" 
>
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition />
            <RowDefinition Height="Auto"/>
        </Grid.RowDefinitions>

        <local:StaggeredReleaseItemsControl x:Name="ic" ItemsSource="{Binding ViewModel.Names}" />

        <StackPanel Grid.Row="1" Orientation="Horizontal">
            <StackPanel.Resources>
                <Style TargetType="Button">
                    <Setter Property="MinWidth" Value="80"/>
                    <Setter Property="MinHeight" Value="20"/>
                </Style>
            </StackPanel.Resources>
            <Button x:Name="btnKick" Content="Kick" Click="btnKick_Click"/>
            <Button x:Name="btnAdd" Content="Add" Click="btnAdd_Click"/>
        </StackPanel>

    </Grid>

</Window>

My ViewModel defines the initial state.

public class StaggeredItemControlTestViewModel : INotifyPropertyChanged
{
    public StaggeredReleaseCollection<string> Names { get; set; }

    public StaggeredItemControlTestViewModel()
    {
        Names = new StaggeredReleaseCollection<string>() { "Carl", "Chris", "Sam", "Erin" };
    }

    public event PropertyChangedEventHandler PropertyChanged;
}

And the code behind is for me to interact with it.

public partial class StaggeredItemControlTestView : Window
{
    List<string> GenesisPeople = new List<string>() { "Rob", "Mike", "Cate", "Andrew", "Dave", "Janet", "Julie" };
    Random random = new Random((int)(DateTime.Now.Ticks % int.MaxValue));

    public StaggeredItemControlTestViewModel ViewModel { get; set; }

    public StaggeredItemControlTestView()
    {
        InitializeComponent();
        ViewModel = new StaggeredItemControlTestViewModel();
        DataContext = this;
    }

    private void btnKick_Click(object sender, RoutedEventArgs e)
    {
        ViewModel.Names.Kick();
    }

    private void btnAdd_Click(object sender, RoutedEventArgs e)
    {
        //Get a random name
        //NOTE: Use a new string here to ensure it's not reusing the same object pointer
        string nextName = new string(GenesisPeople[random.Next(GenesisPeople.Count)].ToCharArray());

        //Add to ViewModel
        ViewModel.Names.Add(nextName);
    }
}

As it runs, I click the "Add" button several times, then the "Kick" button several times, and so on. As I said before, the collections are being trickle-fed the items in the correct order. However during the ArrangeOveride, the InternalChildren collection occasionally reports newly added items as being in the middle of the collection instead of the end. Given that items are only added one at a time normally, I can't understand why this is the case.

Why is InternalChildren on the Panel showing a different order to the bound StaggeredReleaseCollection<T>?

Answer 1

Eureka! I found the problem, thanks to the probing by Clemens.

The issue -is- related to why I'm having to raise OnCollectionChanged myself. On StaggeredReleaseCollection I define a new definition of Add() on this collection so that items are added to my held collection (instead of the underlying Items collection on ObservableCollection). During Kick(), I was using Items.Add(item) to move items from my held collection to the underlying collection.

The solution is to call base.Add(item) instead. Using Reflector I can see that base.Add(item) is on Collection<T> and Items.Add() is based on IList<T>. So only base.Add() contains all the notify property goodness I'm relying upon in this solution.

Aside

I'm starting to wonder if a better way would be to allow the Panel to control all this itself. If I allow items to accumulate in the normal way, perhaps I can add some properties to the visuals so that the Panel can monitor the completion of Phase 1 animations and re-arrange the next item.

That's something I'll have to explore I guess.