Understanding the XLNet model for a concrete case

Question

I'm a data science student, recently I reviewed the XLNet paper and I have a doubt about it:

Imagine we are using many categories, let's say 200, can this model has problems reaching a good accuracy (only applying what the paper says), if not, why? I thought that if we combine a hierarchical classification and then apply the XLNet model it can lead to better results for many categories but I'm not sure because many examples in Kaggle and other websites use XLNet directly. Has some sense what am I saying or I didn't properly understand what XLNet does since I didn't see anyone applying this proposal for many categories?

Answer 1

There's no way to do that, XLNet is a pretrained model so the only thing we can do is tune it.