As soon as I posted this, I realized what might fix it. I added track by item.itemId
and it started acting as expected.
<div class="row" ng-repeat="item in items track by item.itemId | filter: searchTerms | orderBy:['Category1', 'Category2']">
<div class="rowHeader" ng-show="$index == 0 || item.Category1 != items[$index -1].Category1">
{{item.Category1}}
</div>
</div>