Anomaly Detection API Object

Schema to customize anomaly detection for an individual metric

    {
      "config": {
        "holdfor": 1,
        "learning": 100,
        "minimum_deviation": 0.3,
        "samples": 250,
        "sigma": 4,
        "stateful": true,
        "symmetric": false,
        "type": "adaptive",
        "vector": 0
      },
      "detector": "adaptive",
      "fqm": "moog:integration:rest",
      "metric": "ELB:BackendConnectionErrors",
      "uuid": "integration::metrics"
    }

config settings

holdfor

integer

The number of anomalous data points to hold for until generating an event. Suppose Hold for = 1. When a metric generates an anomaly, the detector holds for one more anomaly before it generates an event.

This window is 1 by default for most supported metrics. You might want to increase this number for a specific metric in the following corner case:

  • You want to reduce the "noise" for a metric that generates a lot of repeat anomalies. This can happen if a metric has a very short polling cycle and moves frequently between normal and anomalous range. This can cause the detector to generate a series of repetitive anomalies that say, in essence, "this metric is constantly switching between normal and anomalous range."

NOTE: If you increase the Hold For window, the detector might miss some anomalies that do not generate enough consecutive data points to trigger an anomaly event.

Consider the following metric, which switches between anomalous and normal states every 2 minutes or so. When hold-for and reset-hold-for are both set to the default of 1, this results in a spurt of anomalies.

You might decide that this is normal behavior, and that you only want to generate anomalies when the metric is in anomalous state for 3 minutes or more. In this case, set hold-for and reset-hold-for to 2 or higher.

learning

integer

The number of datums to collect before anomaly detection begins.

minimum_deviation

float

The minimum possible deviation used to calculate anomalies, based on the historic range of values. This setting is useful for metric data sets with very narrow ranges.

Consider the following metric, where the value remains within 1.0 and 1.05 nearly all the time. If the range of values is very narrow, even tiny deviations can result in "false-positive" anomalies.

With the minimum deviation set to 0.3, only values outside of 0.3 times the mean (plus or minus) are considered for anomaly detection.

samples

integer

Number of datums to keep in memory to determine the threshold ranges.

sigma

integer

The number of standard deviations from the norm to determine if a data point is anomalous. You might want to change this setting in the following cases:

  • A metric changes frequently and over a wide range, which causes the detector to flag non-anomalous data points as "false-positive" anomalies. In this case you might want to increase the number of deviations.

  • A metric changes very little and a data point outside the norm, even by a small amount, indicates an anomaly. In this case you might want to decrease the number of deviations.

NOTE: Changing the number of deviations can affect anomaly detection dramatically. As with any change to an anomaly detector, you should closely monitor the metric after you apply the change to ensure that you are getting the detection behavior you want.

Consider the following metric, where the number of deviations is set to 4.

If we lower the number of deviations to 2, more data points are now considered anomalous.

If we raise the number of deviations to 8, fewer data points are now considered anomalous.

stateful

boolean

When True, generate an anomaly only when the metric changes state: when it enters an anomalous state, when its value changes significantly while in an anomalous state, or when it returns to a normal state.

When False, generate an anomaly for every anomalous data point.

See Stateful.

symmetric

boolean

By default, the Adaptive detector calculates upper and lower sigma values to determine anomalies above and below the metric mean. Enable this to calculate the same sigma value for anomalies in both directions.

type

string

In some cases, you can change the anomaly engine for a specific metric. See Anomaly Detectors.

vector

signed integer

(Threshold detector only) Consider Low Threshold or High Threshold only, or both High and Low thresholds to identify anomalies.

ValueDescription
-1Below threshold
 0Above or below threshold
 1Above threshold

detector

string

In some cases, you can change the anomaly engine for a specific metric. See Anomaly Detectors.

fqm

string

The fully-qualified name of the managed object associated with a metric. The full name is in the format namespace:managed-object:archive.

You can view the FQM at the top of a metric chart in the UI:

metric

string

The metric name.

uuid

string

The universally-unique ID of a collector. The UUID is in the format namespace:managed-object:archive.

You can view the UUID in the UI page for the collector.