How to understand bias parameter in LIBLINEAR?

Question

I don't understand the meaning of bias parameter in the API of LIBLINEAR. Why is it specified by user during the training? Shouldn't it be just a distance from the separating hyperplane to origin which is a parameter of the learned model?

This is from the README:

struct problem
{
    int l, n;
    int *y;
    struct feature_node **x;
    double bias;
};

If bias >= 0, we assume that one additional feature is added to the end of each data instance.

What is this additional feature?

Answer 1

Let's look at the equation for the separating hyperplane:

w_1 * x_1  + w_2 * x_2  + w_3 * x_3 + ... + w_bias * x_bias = 0

Where x are the feature values and w are the trained "weights". The additional feature x_bias is a constant, whose value is equal to the bias. If bias = 0, you will get a separating hyperplane going through the origin (0,0,0,...). You can imagine many cases, where such a hyperplane is not the optimal separator.

The value of the bias affects the margin through scaling of w_bias. Therefore the bias is a tuning parameter, which is usually determined through cross-validation similar to other parameters.