Just use the dot product starting from point B.
if (VectorOf(AC) * VectorOf(AB) < 0) {
// C is on the left of A
}
else {
if (VectorOf(BC) * VectorOf(BA) < 0) {
// C is on the right of B
}
else {
// C is between A and B
}
}
Alternatively, you can compute the projected distance, relative to vector AB :
(VectorOf(AC) * VectorOf(AB)) / (VectorOf(AB) * VectorOf(AB))
The result would be < 0, between 0 and 1, or > 1 in your three cases, as shows the math below :
C
/│
/ │
/ │
──A── H ─────B─────
The definition of the dot product is that
AC · AB = AC×AB×cos(Â) = AH×AB (signed : negative if C is left of A, positive if C is to the right).
AB · AB = AB² (positive)
The result of the division is the signed ratio AH/AB :
- 0 1 >1
────A── H ─────B─────