Question

Because the IDF is a constant number. All value in one dimension multiply a constant number.

In SVM Linear kernel, The result will be different ?

Was it helpful?

Solution

Your initial question doesn't really make sense. You mix up two different worlds: 1) TF/IDF: features for text representation 2) SVM - Linear Kernel: The simplest approach for SVMs (indeed used for text).

The difference of TF and TF/IDF is on whether the corpus-frequencies of words are used or not. The TF/IDF is by far a better choice, independent of classifier.

Using only TF we don't really care if a word is common or not. Thus, common words like e.g. articles receive a large weight even if they contribute no real information.

In TF/IDF the more frequent a word is in the corpus, the smaller weight it receives. Thus, common words like articles receive small weights but rare words, that it is assumed to carry more information, receive larger weights.

N.B. In the above, "articles" are used as an example they should normally removed in a preprocessing step.

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top