You can build a model on one feature. I assume, that by "one feature" you mean simply one number in R
(otherwise, it would be completely "traditional" usage). However this means, that you are building a classifier in one-dimensional space, and as such - many classifiers will be redundant (as it is really a simple problem). What is more important - checking whether you can correctly classify objects using one particular dimensions does not mean that it is a good/bad feature once you use combination of them. In particular it may be the case that:
- Many features may "discover" the same phenomena in data, and so - each of them separatly can yield good results, but once combined - they won't be any better then each of them (as they simply capture same information)
- Features may be useless until used in combination. Some phenomena can be described only in multi-dimensional space, and if you are analyzing only one-dimensional data - you won't ever discover their true value, as a simple example consider four points
(0,0),(0,1),(1,0),(1,1)
such that(0,0),(1,1)
are elements of one class, and rest of another. If you look separatly on each dimension - then the best possible accuracy is0.5
(as you always have points of two different classes in exactly same points - 0 and 1). Once combined - you can easily separate them, as it is axor
problem.
To sum up - it is ok to build a classifier in one dimensional space, but:
- Such problem can be solved without "heavy machinery".
- Results should not be used as a base of feature selection (or to be more strict - this can be very deceptive).