Do we really need `<unk>` tokens?

datascience.stackexchange https://datascience.stackexchange.com/questions/33444

Pregunta

I am wondering, do we really need <unk> tokens? Why do we limit our vocabulary?

Is it for speed? Accuracy?

If we disable all limitations, what do you predict happens?

No hay solución correcta

Licenciado bajo: CC-BY-SA con atribución
scroll top