The answer lies in this call
mMyView = inflater.inflate(resId, this, true);
The documentation states:
Returns The root View of the inflated hierarchy. If root was supplied and attachToRoot is true, this is root; otherwise it is the root of the inflated XML file.
I was assuming the inflate call always returns the root of the inflated layout (defined by resId) but if the root parameter is provided and attachToRoot is true then it's the root of the layout the inflated layout was added to (it's new parent view) and that's root (in this case root == this).
So while
mMyView = inflater.inflate(resId, this, false);
assigns the view to mMyView that we want to remove later
mMyView = inflater.inflate(resId, this, true);
assigns the parent of that view. The call removeView(mMyView) is actually identical to removeView(this) and that obviously doesn't do anything.
This behavior is IMO very counter-intuitive and prone to errors. Why should it return the root layout instead of the inflated layout when we already have that root layout with the root parameter while we don't have a reference to the inflated layout yet?