Difference between Anomaly Detection and Behaviour Detection

Question

In an Intrusion Detection System, There are two techniques called Anomaly Detection and Behaviour Detection. I am implementing an IDS from scratch and was checking for some signatures and from some site they were given as different types of methods for detection. What is the basic difference in them? As it seems to me that both are same and therefore same signatures should be able to detect these kind of attacks.

Example for Anomaly detection as given on the site: Detecting a function call that is not part of the normal profile

Example for Behaviour detection as given on the site: Searching for any remote invocation of cmd.exe.

Now as it seems to me that both are kind of same things i.e. deviation from normal behaviour so why they were characterised as different methods??

Answer 1

There is indeed a difference between anomaly-based and behavioral detection. Before exploring the two, I would like to point out that the intrusion detection community uses two additional styles: misused-based (aka. signature-based) and specification-based detection, yet these are unrelated to your question.

Anomaly-based detection

Definition: A two-step approach that involves first training a system with data to establish some notion of normality and then use the established profile on real data to flag deviations.

Example: look at some features of benign URLs, e.g., their length, character distribution, etc., to find define what a "normal" URL looks like. With this notion of normality, you would then flag URLs that are too far off the normal URL length or have too many abnormal characters in them.

Pros:

• Can detect potentially a wide range of novel attacks

Cons:

• May miss known attacks
• May miss novel attacks if they don't stick out along the observed dimension
• High fals positive rate (see the base rate fallacy)
• Purity of training data (i.e., absence of attacks)

Behavior-based detection

Definition: Looking for evidence of compromise rather than the attack itself.

Example: monitor the shell history for unset HISTFILE, a command that typically only attackers enter after compromising a machine.

Pros:

• May detect a wide range of novel attacks
• Low false positives
• Can be cheap to deploy and monitor

Cons:

• Post-facto, attack already occurred
• Easy to evade once known

Answer 2

Indeed "anomaly-based detection" and "behavior-based detection" are not different. Behavior detection is usually found in vendors' datasheets, and they refer to the communication patterns (and their features) they observe/feed the detection engine with.

Answer 3

The two main types of IDS are signature-based and anomaly-based. The difference is simple: signature-based IDS rely on a database of known attacks, while anomaly-based observe the behavior of the network, profile the normal behavior, and in the case of any anomalies, these anomalies cause deviations on which it alerts.