Documentation

AlertNode

The alert node triggers an event of varying severity levels and passes the event to event handlers. The criteria for triggering an alert is specified via a lambda expression. See AlertNode.Info, AlertNode.Warn, and AlertNode.Crit below.

Constructor

Chaining method Description
alert ( ) Create an alert node, which can trigger alerts.

To dynamically construct a custom HTTP body or URL, use an alert template. For example, httpPost('localhost/?host={{ index .Tags "host"}}&cpu={{ index .Tags "cpu" }}').

Property methods

Setters description
alerta ( ) Send the alert to Alerta.
all ( ) Indicates an alert should trigger only if all points in a batch match the criteria. Does not apply to stream alerts.
bigPanda ( ) Send the alert to BigPanda.
crit ( value ast.LambdaNode) Filter expression for the CRITICAL alert level. An empty value indicates the level is invalid and is skipped.
critReset ( value ast.LambdaNode) Filter expression for resetting the CRITICAL alert level to lower level.
details ( value string) Template for constructing a detailed HTML message for the alert. The same template data is available as the AlertNode.Message property, in addition to a Message field that contains the rendered Message value.
discord ( ) Send the alert to Discord.
durationField ( value string) Optional field key to add the alert duration to the data. The duration is always in units of nanoseconds.
email ( to ...string) Email the alert data.
exec ( executable stringargs ...string) Execute a command whenever an alert is triggered and pass the alert data over STDIN in JSON format.
flapping ( low float64high float64) Perform flap detection on the alerts. The method used is similar method to Nagios: https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/3/en/flapping.html
hipChat ( ) Send the alert to HipChat.
history ( value int64) Number of previous states to remember when computing flapping levels and checking for state changes. Minimum value is 2 in order to keep track of current and previous states.
id ( value string) Template for constructing a unique ID for a given alert.
idField ( value string) Optional field key to add to the data, containing the alert ID as a string.
idTag ( value string) Optional tag key to use when tagging the data with the alert ID.
info ( value ast.LambdaNode) Filter expression for the INFO alert level. An empty value indicates the level is invalid and is skipped.
inhibit ( category stringequalTags ...string) Inhibit other alerts in a category. The equal tags provides a list of tags that must be equal in order for an alert event to be inhibited.
infoReset ( value ast.LambdaNode) Filter expression for resetting the INFO alert level to lower level.
kafka ( ) Send the alert to a Kafka cluster.
levelField ( value string) Optional field key to add to the data, containing the alert level as a string.
levelTag ( value string) Optional tag key to use when tagging the data with the alert level.
log ( filepath string) Log JSON alert data to file. One event per line. Must specify the absolute path to the log file. It will be created if it does not exist. Example: stream
message ( value string) Template for constructing a meaningful message for the alert.
messageField ( value string) Optional field key to add to the data, containing the alert message.
mqtt ( topic string) Send alert to an MQTT broker
noRecoveries ( ) Do not send recovery alerts.
opsGenie ( ) Send alert to OpsGenie using OpsGenie’s v1 API. (Deprecated)
opsGenie2 ( ) Send alert to OpsGenie using OpsGenie’s v2 API.
pagerDuty ( ) Send the alert to PagerDuty using PagerDuty’s v1 API. (Deprecated)
pagerDuty2 ( ) Send the alert to PagerDuty using PagerDuty’s v2 API.
post ( urls ...string) HTTP POST JSON alert data to a specified URL.
pushover ( ) Send the alert to Pushover.
quiet ( ) Suppresses all error logging events from this node.
sensu ( ) Send the alert to Sensu.
slack ( ) Send the alert to Slack.
snmpTrap ( trapOid string) Send the alert using SNMP traps.
stateChangesOnly ( maxInterval ...time.Duration) Only sends events where the state changed. Each different alert level OK, INFO, WARNING, and CRITICAL are considered different states.
talk ( ) Send the alert to Talk.
tcp ( address string) Send JSON alert data to a specified address over TCP.
telegram ( ) Send the alert to Telegram.
topic ( value string) Topic specifies the name of an alert topic to which, alerts will be published. Alert handlers can be configured per topic, see the API documentation.
victorOps ( ) Send alert to VictorOps.
warn ( value ast.LambdaNode) Filter expression for the WARNING alert level. An empty value indicates the level is invalid and is skipped.
warnReset ( value ast.LambdaNode) Filter expression for resetting the WARNING alert level to lower level.

Chaining methods

Alert, Barrier, Bottom, ChangeDetect, Combine, Count, CumulativeSum, Deadman, Default, Delete, Derivative, Difference, Distinct, Ec2Autoscale, Elapsed, Eval, First, Flatten, GroupBy, HoltWinters, HoltWintersWithFit, HttpOut, HttpPost, InfluxDBOut, Join, K8sAutoscale, KapacitorLoopback, Last, Max, Mean, Median, Min, Mode, MovingAverage, Percentile, Sample, Shift, Sideload, Spread, StateCount, StateDuration, Stats, Stddev, Sum, SwarmAutoscale, Top, Trickle, Union, Where, Window


Available event handlers

Different event handlers can be configured for each AlertNode. Some handlers like Email, HipChat, Sensu, Slack, OpsGenie, VictorOps, PagerDuty, Telegram and Talk have a configuration option, global, that indicates that all alerts implicitly use the handler.

Handler Description
Alerta Post alert message to Alerta.
BigPanda Post alert message to BigPanda.
Discord Post alert message to Discord channel.
email Send and email with alert data.
exec Execute a command passing alert data over STDIN.
HipChat Post alert message to HipChat room.
Kafka Send alert to a Apache Kafka cluster.
log Log alert data to file.
MQTT Post alert message to MQTT.
OpsGenie v1 Send alert to OpsGenie using their v1 API. (Deprecated)
OpsGenie v2 Send alert to OpsGenie using their v2 API.
PagerDuty v1 Send alert to PagerDuty using their v1 API. (Deprecated)
PagerDuty v2 Send alert to PagerDuty using their v2 API.
post HTTP POST data to a specified URL.
Pushover Send alert to Pushover.
Sensu Post alert message to Sensu client.
Slack Post alert message to Slack channel.
SNMPTrap Trigger SNMP traps.
Talk Post alert message to Talk client.
tcp Send data to a specified address via raw TCP.
Telegram Post alert message to Telegram client.
VictorOps Send alert to VictorOps.

Alert event data

Each event that gets sent to a handler contains the following alert data:

Name Description
ID The ID of the alert, user defined.
Message The alert message, user defined.
Details The alert details, user defined HTML content.
Time The time the alert occurred.
Duration The duration of the alert in nanoseconds.
Level One of OK, INFO, WARNING or CRITICAL.
Data influxql.Result containing the data that triggered the alert.
Recoverable Indicates whether the alert is auto-recoverable. Determined by the .noRecoveries() property.

Events are sent to handlers if the alert is in a state other than ‘OK’ or the alert just changed to the ‘OK’ state from a non ‘OK’ state (a.k.a. the alert recovered). Using the AlertNode.StateChangesOnly property events will only be sent to handlers if the alert changed state.

It is valid to configure multiple alert handlers, even with the same type.

Example:

stream
    .groupBy('service')
  |alert()
    .id('kapacitor/{{ index .Tags "service" }}')
    .message('{{ .ID }} is {{ .Level }} value:{{ index .Fields "value" }}')
    .info(lambda: "value" > 10)
    .warn(lambda: "value" > 20)
    .crit(lambda: "value" > 30)
    .post("http://example.com/api/alert")
    .post("http://another.example.com/api/alert")
    .tcp("exampleendpoint.com:5678")
    .email('oncall@example.com')

Each expression maintains its own state. The order of execution for the expressions is not considered to be deterministic. For each point an expression may or may not be evaluated. If no expression is true then the alert is considered to be in the OK state.

Kapacitor supports alert reset expressions. This way when an alert enters a state, it can only be lowered in severity if its reset expression evaluates to true.

Example:

stream
  |from()
    .measurement('cpu')
    .where(lambda: "host" == 'serverA')
    .groupBy('host')
  |alert()
    .info(lambda: "value" > 60)
    .infoReset(lambda: "value" < 50)
    .warn(lambda: "value" > 70)
    .warnReset(lambda: "value" < 60)
    .crit(lambda: "value" > 80)
    .critReset(lambda: "value" < 70)

For example, given the following values, the corresponding alert states are:

Value Alert State
61 INFO
73 WARNING
64 WARNING
85 CRITICAL
62 INFO
56 INFO
47 OK

Available Statistics:

  • alerts_triggered: Total number of alerts triggered
  • oks_triggered: Number of OK alerts triggered
  • infos_triggered: Number of Info alerts triggered
  • warns_triggered: Number of Warn alerts triggered
  • crits_triggered: Number of Crit alerts triggered

Properties

Property methods modify state on the calling node. They do not add another node to the pipeline, and always return a reference to the calling node. Property methods are marked using the . operator.

Alerta

Send the alert to Alerta. Detailed configuration options and setup instructions are provided in the Alerta Event Handler article.

Example kapacitor.conf

[alerta]
  enabled = true
  url = "https://alerta.yourdomain"
  token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"
  environment = "Production"
  origin = "Kapacitor"

Example TICKscript

stream
  |alert()
    .alerta()
      .resource('Hostname or service')
      .event('Something went wrong')

All

Indicates an alert should trigger only if all points in a batch match the criteria. Does not apply to stream alerts.

alert.all()

BigPanda

Send the alert to BigPanda. Detailed configuration options and setup instructions are provided in the BigPanda Event Handler article.

Example kapacitor.conf

[bigpanda]
  enabled = true
  url = "https://api.bigpanda.io/data/v2/alerts"
  token = "BigPanda-API-auth-bearer-token"
  app-key = "BigPanda-integration-app-key"

Example TICKscript

stream
  |alert()
    .bigPanda()
      .appKey('fc39458f98e91eb0310258c3b725d643')
      .primaryProperty('device')
      .secondaryProperty('sensor_name')
      .topic('bigpanda-topic')

Category

Category places this alert in a named category. Categories are used to inhibit alerts.

alert.category(value string)

Crit

Filter expression for the CRITICAL alert level. An empty value indicates the level is invalid and is skipped.

alert.crit(value ast.LambdaNode)

CritReset

Filter expression for resetting the CRITICAL alert level to lower level.

alert.critReset(value ast.LambdaNode)

Details

Template for constructing a detailed HTML message for the alert. The same template data is available as the AlertNode.Message property, in addition to a Message field that contains the rendered Message value.

The intent is that the Message property be a single line summary while the Details property is a more detailed message possibly spanning multiple lines, and containing HTML formatting.

This template is rendered using the html/template package in Go so safe and valid HTML is generated.

The json method is available within the template to convert any variable to a valid JSON string.

xExample:

|alert()
  .id('{{ .Name }}')
  .details('''
<h1>{{ .ID }}</h1>
<b>{{ .Message }}</b>
Value: {{ index .Fields "value" }}
''')
  .email()

Default: {{ json . }}

alert.details(value string)

Discord

Send the alert to Discord. Detailed configuration options and setup instructions are provided in the Discord Event Handler article.

Example kapacitor.conf

[discord]
  enabled = true
  url = "https://discordapp.com/api/webhooks/xxxxxxxxxxxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

Example TICKscript

stream
  |alert()
    .discord()
      .embedTitle('Alert!')

DurationField

Optional field key to add the alert duration to the data. The duration is always in units of nanoseconds.

alert.durationField(value string)

Email

Email the alert data to specified “To” email addresses. Detailed configuration options and setup instructions are provided in the Email Event Handler article.

Example kapacitor.conf

[smtp]
  enabled = true
  host = "localhost"
  port = 25
  username = "xxxx"
  password = "xxxx"
  from = "kapacitor@example.com"
  to = ["oncall@example.com"]
  global = true
  state-changes-only = true

Example TICKscript

|alert()
  .id('{{ .Name }}')
  // Email subject
  .message('{{ .ID }}:{{ .Level }}')
  // Email body as HTML
  .details('''
<h1>{{ .ID }}</h1>
<b>{{ .Message }}</b>
Value: {{ index .Fields "value" }}
''')
  .email()

Exec

Execute a command whenever an alert is triggered and pass the alert data over STDIN in JSON format. Detailed usage instructions and examples are provided in the Exec Event Handler article.

// Pattern
alert.exec(executable string, args ...string)

// Example
alert.exec('/usr/bin/python', 'myscript.py')

Flapping

Perform flap detection on the alerts. The method used is similar method to Nagios flapping

Each different alerting level is considered a different state. The low and high thresholds are inverted thresholds of a percentage of state changes. Meaning that if the percentage of state changes goes above the high threshold, the alert enters a flapping state. The alert remains in the flapping state until the percentage of state changes goes below the low threshold. Typical values are low: 0.25 and high: 0.5. The percentage values represent the number state changes over the total possible number of state changes. A percentage change of 0.5 means that the alert changed state in half of the recorded history, and remained the same in the other half of the history.

// Pattern
alert.flapping(low float64, high float64)

// Example
alert.flapping(0.25, 0.5)

Kafka

Send the alert to an Apache Kafka cluster. Detailed configuration options and setup instructions are provided in the Kafka Event Handler article.

Example kapacitor.conf

[[kafka]]
  enabled = true
  id = "localhost"
  timeout = 10s

Example TICKscript

stream
  |alert()
    .kafka()
      .cluster('kafka-cluster')
      .kafkaTopic('kafka-topic-name')

History

Number of previous states to remember when computing flapping levels and checking for state changes. Minimum value is 2 in order to keep track of current and previous states.

Default: 21

// Pattern
alert.history(value int64)

// Example
alert.history(21)

HipChat

Send the alert to HipChat. Detailed configuration options and setup instructions are provided in the HipChat Event Handler article.

Example kapacitor.conf

[hipchat]
  enabled = true
  url = "https://orgname.hipchat.com/v2/room"
  room = "4189212"
  token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"

Example TICKscript

stream
  |alert()
    .hipChat()
      .room('Kapacitor')

Id

Template for constructing a unique ID for a given alert.

Available template data

Data Name Description
Name Measurement name.
TaskName The name of the task
Group Concatenation of all group-by tags of the form [key=value,]+. If no groupBy is performed equal to literal ’nil’.
Tags Map of tags. Use `{{ index .Tags “key” }}`` to get a specific tag value.
ServerInfo Information about the running server. Available nested fields are Hostname, ClusterID and ServerID.

Default: {{ .Name }}:{{ .Group }}

// Pattern
alert.id(value string)

// Example
alert.id('{{ .Name }}:{{ .Group }}')

Example: ID template using name and group

stream
  |from()
    .measurement('cpu')
    .groupBy('cpu')
  |alert()
    .id('kapacitor/{{ .Name }}/{{ .Group }}')

Resulting ID: kapacitor/cpu/cpu=cpu0

Example: ID template using a tag

stream
  |from()
    .measurement('cpu')
    .groupBy('service')
  |alert()
    .id('kapacitor/{{ index .Tags "service" }}')

Resulting ID: kapacitor/authentication

Example: ID template using multiple tags

stream
  |from()
    .measurement('cpu')
    .groupBy('service', 'host')
  |alert()
    .id('kapacitor/{{ index .Tags "service" }}/{{ index .Tags "host" }}')

Resulting ID: kapacitor/authentication/auth001.example.com

IdField

Optional field key to add to the data, containing the alert ID as a string.

// Pattern
alert.idField(value string)

// Example
alert.idField('id')

IdTag

Optional tag key to use when tagging the data with the alert ID.

// Pattern
alert.idTag(value string)

// Example
alert.idTag('alertID')

Info

Filter expression for the INFO alert level. An empty value indicates the level is invalid and is skipped.

// Pattern
alert.info(value ast.LambdaNode)

// Example
alert.info(lambda: 'usage_idle' < 60)

InfoReset

Filter expression for resetting the INFO alert level to lower level.

// Pattern
alert.infoReset(value ast.LambdaNode)

// Example
alert.infoReset(lambda: 'usage_idle' > 60)

Inhibit

Inhibit other alerts in a category. The equal tags provides a list of tags that must be equal in order for an alert event to be inhibited.

The following two TICKscripts demonstrate how to use the inhibit feature:

    //cpu_alert.tick
stream
  |from()
    .measurement('cpu')
    .groupBy('host')
  |alert()
    .category('system_alerts')
    .crit(lambda: "usage_idle" < 10.0)
//host_alert.tick
stream
  |from()
    .measurement('uptime')
    .groupBy('host')
  |deadman(0.0, 1m)
    .inhibit('system_alerts', 'host')

The deadman is a type of alert node and can be used to inhibit all alerts in the system_alerts category when triggered. The host argument to the inhibit function says that the host tag must be equal between the CPU alert and the host alert in order for it to be inhibited. This has the effect of the deadman alerts only inhibiting CPU alerts for hosts that are currently dead.

alert.inhibit(category string, equalTags ...string)

Kafka

Send the alert to a Kafka topic. Detailed setup and usage instructions are provided in the Kafka Event Handler article.

Example: kapacitor.conf

[[kafka]]
  enabled = true
  id = "default"
  brokers = ["localhost:9092"]

Example: TICKscript

stream
 |alert()
    .kafka()
      .cluster('default')
      .kafkaTopic('alerts')

LevelField

Optional field key to add to the data, containing the alert level as a string.

// Pattern
alert.levelField(value string)

// Example
alert.levelField('INFO')

LevelTag

Optional tag key to use when tagging the data with the alert level.

// Pattern
alert.levelTag(value string)

// Example
alert.levelTag('level')

Log

Log JSON alert data to file. Detailed setup and usage instructions are provided in the Log Event Handler article.

Example TICKscript

stream
  |alert()
    .log('/tmp/alert')
      .mode(0644)

Message

Template for constructing a meaningful message for the alert.

Available template data

Data Name Description
ID The ID of the alert.
Name Measurement name.
TaskName The name of the task
Group Concatenation of all group-by tags of the form 1. If no groupBy is performed equal to literal ’nil’.
Tags Map of tags. Use {{ index .Tags "key" }} to get a specific tag value.
Level Alert Level - INFO, WARNING, CRITICAL.
Fields Map of fields. Use {{ index .Fields "key" }} to get a specific field value.
Time The time of the point that triggered the event. Default format is YYYY-MM-DD 00:00:00 +0000 UTC
Duration The duration of the alert.

Default: {{ .ID }} is {{ .Level }}

Example:

stream
  |from()
    .measurement('cpu')
    .groupBy('service', 'host')
  |alert()
    .id('{{ index .Tags "service" }}/{{ index .Tags "host" }}')
    .message('{{ .ID }} is {{ .Level}} value: {{ index .Fields "value" }}')

Resulting Message: authentication/auth001.example.com is CRITICAL value:42

MessageField

Optional field key to add to the data containing the alert message.

// Pattern
alert.messageField(value string)

// Example
alert.messageField('message')

MQTT

Send alert to an MQTT broker. Detailed configuration options and usage instructions are provided in the MQTT Event Handler article.

Example kapacitor.conf

[[mqtt]]
  enabled = true
  name = "localhost"
  default = true
  url = "tcp://localhost:1883"
  client-id = "kapacitor"
  username = "myusername"
  password = "mysupersecretpassw0rd"

Example TICKscript

stream
  |alert()
    .mqtt('topic')
      .brokerName('localhost')

NoRecoveries

Do not send recovery alerts. Sets recoverable alert data field to false.

alert.noRecoveries()

OpsGenie v1

Send alert to OpsGenie using OpsGenie’s v1 API. Detailed configuration options and setup instructions are provided in the OpsGenie v1 Event Handler article.

Example kapacitor.conf

[opsgenie]
  enabled = true
  api-key = "xxxxx"
  teams = ["everyone"]
  recipients = ["jim", "bob"]

Example TICKscript

stream
  |alert()
    .opsGenie()

OpsGenie v2

Send alert to OpsGenie using OpsGenie’s v2 API. Detailed configuration options and setup instructions are provided in the OpsGenie v2 Event Handler article.

Example kapacitor.conf

[opsgenie2]
  enabled = true
  api-key = "xxxxx"
  teams = ["everyone"]
  recipients = ["jim", "bob"]

Example TICKscript

stream
  |alert()
    .opsGenie2()

PagerDuty v1

Send the alert to PagerDuty using PagerDuty’s v1 API. Detailed configuration options and setup instructions are provided in the PagerDuty v1 Event Handler article.

Example kapacitor.conf

[pagerduty]
  enabled = true
  service-key = "xxxx"
  url = "https://events.pagerduty.com/generic/2010-04-15/create_event.json"

Example TICKscript

stream
  |alert()
    .pagerDuty()

PagerDuty v2

Send the alert to PagerDuty using PagerDuty’s v2 API. Detailed configuration options and setup instructions are provided in the PagerDuty v2 Event Handler article.

Example kapacitor.conf

[pagerduty2]
  enabled = true
  routing-key = "xxxx"
  url = "https://events.pagerduty.com/v2/enqueue"

Example TICKscript

stream
  |alert()
    .pagerDuty2()

Post

HTTP POST JSON alert data to a specified URL. Detailed configuration options and setup instructions are provided in the Post Event Handler article.

Example TICKscript

stream
  |alert()
    .post('http://example.com')
      .captureResponse()

Pushover

Send the alert to Pushover. Detailed configuration options and setup instructions are provided in the Pushover Event Handler article.

Example kapacitor.conf

[pushover]
  enabled = true
  token = "9hiWoDOZ9IbmHsOTeST123ABciWTIqXQVFDo63h9"
  user_key = "Pushover"

Example TICKscript

stream
  |alert()
    .pushover()
      .sound('siren')
      .user_key('other user')
      .device('mydev')
      .title('mytitle')
      .uRL('myurl')
      .URLTitle('mytitle')

Quiet

Suppress all error logging events from this node.

alert.quiet()

Sensu

Send the alert to Sensu. Detailed configuration options and setup instructions are provided in the Sensu Event Handler article.

*Example kapacitor.conf

[sensu]
  enabled = true
  url = "http://sensu:3030"
  source = "Kapacitor"
  handlers = ["sns","slack"]

Example TICKscript

stream
  |alert()
    .sensu()
      .handlers('sns','slack')

Slack

Send the alert to Slack. Detailed configuration options and setup instructions are provided in the Slack Event Handler article.

Example kapacitor.conf

[slack]
  enabled = true
  url = "https://hooks.slack.com/services/xxxxxxxxx/xxxxxxxxx/xxxxxxxxxxxxxxxxxxxxxxxx"
  channel = "#general"

Example TICKscript

stream
  |alert()
    .slack()
      .channel('#alerts')
      .channel('@jsmith')

SnmpTrap

Send the alert using SNMP traps. Detailed configuration options and setup instructions are provided in the SNMP Trap Event Handler article.

Example kapacitor.conf

[snmptrap]
  enabled = true
  addr = "127.0.0.1:9162"
  community = "public"

Example TICKscript

stream
  |alert()
    .snmpTrap('1.3.6.1.2.1.1')
      .data('1.3.6.1.2.1.1.6', 'i', '{{ index .Field "value" }}')
      .data('1.3.6.1.2.1.1.7', 's', '{{ .Message }}')

StateChangesOnly

Only sends events where the state changed. Each different alert level OK, INFO, WARNING, and CRITICAL are considered different states.

Example

stream
  |from()
    .measurement('cpu')
  |window()
    .period(10s)
    .every(10s)
  |alert()
    .crit(lambda: "value" > 10)
    .stateChangesOnly()
    .slack()

If the “value” is greater than 10 for a total of 60s, then only two events will be sent. First, when the value crosses the threshold, and second, when it falls back into an OK state. Without stateChangesOnly, the alert would have triggered 7 times: 6 times for each 10s period where the condition was met and once more for the recovery.

An optional maximum interval duration can be provided. An event will not be ignored (aka trigger an alert) if more than the maximum interval has elapsed since the last alert.

Example

stream
  // ...
  |alert()
    .crit(lambda: "value" > 10)
    .stateChangesOnly(10m)
    .slack()

TCP

Send JSON alert data to a specified address over TCP. Detailed usage instructions are provided in the TCPEvent Handler article.

// Pattern
alert.tcp(address string)

// Example
alert.tcp('127.0.0.1:7777')

Telegram

Send the alert to Telegram. Detailed configuration options and setup instructions are provided in the Telegram Event Handler article.

Example kapacitor.conf

[telegram]
  enabled = true
  token = "123456789:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  chat-id = "xxxxxxxxx"
  parse-mode = "Markdown"
	disable-web-page-preview = true
	disable-notification = false

Example TICKscript

stream
  |alert()
    .telegram()
      .chatId('xxxxxxx')

Topic

Topic specifies the name of an alert topic to which alerts will be published. Alert handlers can be configured per topic. See the API documentation.

// Pattern
alert.topic(value string)

// Example
alert.topic('cpu')

VictorOps

Send alert to VictorOps. Detailed configuration options and setup instructions are provided in the VictorOps Event Handler article.

Example kapacitor.conf

[victorops]
  enabled = true
  api-key = "xxxxx"
  routing-key = "everyone"

Example TICKscript

stream
  |alert()
    .victorOps()
      .routingKey('team_rocket')

Warn

Filter expression for the WARNING alert level. An empty value indicates the level is invalid and is skipped.

// Pattern
alert.warn(value ast.LambdaNode)

// Example
alert.warn(lambda: 'usage_idle' < 20)

WarnReset

Filter expression for resetting the WARNING alert level to lower level.

// Pattern
alert.warnReset(value ast.LambdaNode)

// Example
alert.warnReset(lambda: 'usage_idle' > 20)

Chaining Methods

Chaining methods create a new node in the pipeline as a child of the calling node. They do not modify the calling node. Chaining methods are marked using the | operator.

Alert

Create an alert node, which can trigger alerts.

alert|alert()

Returns: AlertNode

Barrier

Create a new Barrier node that emits a BarrierMessage periodically.

One BarrierMessage will be emitted every period duration.

alert|barrier()

Returns: BarrierNode

Bottom

Select the bottom num points for field and sort by any extra tags or fields.

alert|bottom(num int64, field string, fieldsAndTags ...string)

Returns: InfluxQLNode

ChangeDetect

Create a new node that only emits new points if different from the previous point.

alert|changeDetect(field string)

Returns: ChangeDetectNode

Combine

Combine this node with itself. The data is combined on timestamp.

alert|combine(expressions ...ast.LambdaNode)

Returns: CombineNode

Count

Count the number of points.

alert|count(field string)

Returns: InfluxQLNode

CumulativeSum

Compute a cumulative sum of each point that is received. A point is emitted for every point collected.

alert|cumulativeSum(field string)

Returns: InfluxQLNode

Deadman

Helper function for creating an alert on low throughput, a.k.a. deadman’s switch.

  • Threshold: trigger alert if throughput drops below threshold in points/interval.
  • Interval: how often to check the throughput.
  • Expressions: optional list of expressions to also evaluate. Useful for time of day alerting.

Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |deadman(100.0, 10s)
    //Do normal processing of data
    data...

The above is equivalent to this example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |stats(10s)
            .align()
        |derivative('emitted')
            .unit(10s)
            .nonNegative()
        |alert()
            .id('node \'stream0\' in task \'{{ .TaskName }}\'')
            .message('{{ .ID }} is {{ if eq .Level "OK" }}alive{{ else }}dead{{ end }}: {{ index .Fields "emitted" | printf "%0.3f" }} points/10s.')
            .crit(lambda: "emitted" <= 100.0)
    //Do normal processing of data
    data...

The id and message alert properties can be configured globally via the ‘deadman’ configuration section.

Since the AlertNode is the last piece it can be further modified as usual. Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    data
        |deadman(100.0, 10s)
            .slack()
            .channel('#dead_tasks')
    //Do normal processing of data
    data...

You can specify additional lambda expressions to further constrain when the deadman’s switch is triggered. Example:

    var data = stream
        |from()...
    // Trigger critical alert if the throughput drops below 100 points per 10s and checked every 10s.
    // Only trigger the alert if the time of day is between 8am-5pm.
    data
        |deadman(100.0, 10s, lambda: hour("time") >= 8 AND hour("time") <= 17)
    //Do normal processing of data
    data...
alert|deadman(threshold float64, interval time.Duration, expr ...ast.LambdaNode)

Returns: AlertNode

Default

Create a node that can set defaults for missing tags or fields.

alert|default()

Returns: DefaultNode

Delete

Create a node that can delete tags or fields.

alert|delete()

Returns: DeleteNode

Derivative

Create a new node that computes the derivative of adjacent points.

alert|derivative(field string)

Returns: DerivativeNode

Difference

Compute the difference between points independent of elapsed time.

alert|difference(field string)

Returns: InfluxQLNode

Distinct

Produce batch of only the distinct points.

alert|distinct(field string)

Returns: InfluxQLNode

Ec2Autoscale

Create a node that can trigger autoscale events for a ec2 autoscalegroup.

alert|ec2Autoscale()

Returns: Ec2AutoscaleNode

Elapsed

Compute the elapsed time between points.

alert|elapsed(field string, unit time.Duration)

Returns: InfluxQLNode

Eval

Create an eval node that will evaluate the given transformation function to each data point. A list of expressions may be provided and will be evaluated in the order they are given. The results are available to later expressions.

alert|eval(expressions ...ast.LambdaNode)

Returns: EvalNode

First

Select the first point.

alert|first(field string)

Returns: InfluxQLNode

Flatten

Flatten points with similar times into a single point.

alert|flatten()

Returns: FlattenNode

GroupBy

Group the data by a set of tags.

Can pass literal * to group by all dimensions. Example:

    |groupBy(*)
alert|groupBy(tag ...interface{})

Returns: GroupByNode

HoltWinters

Compute the Holt-Winters (/influxdb/v1/query_language/functions/#holt-winters) forecast of a data set.

alert|holtWinters(field string, h int64, m int64, interval time.Duration)

Returns: InfluxQLNode

HoltWintersWithFit

Compute the Holt-Winters (/influxdb/v1/query_language/functions/#holt-winters) forecast of a data set. This method also outputs all the points used to fit the data in addition to the forecasted data.

alert|holtWintersWithFit(field string, h int64, m int64, interval time.Duration)

Returns: InfluxQLNode

HttpOut

Create an HTTP output node that caches the most recent data it has received. The cached data is available at the given endpoint. The endpoint is the relative path from the API endpoint of the running task. For example, if the task endpoint is at /kapacitor/v1/tasks/<task_id> and endpoint is top10, then the data can be requested from /kapacitor/v1/tasks/<task_id>/top10.

alert|httpOut(endpoint string)

Returns: HTTPOutNode

HttpPost

Creates an HTTP Post node that POSTS received data to the provided HTTP endpoint. HttpPost expects 0 or 1 arguments. If 0 arguments are provided, you must specify an endpoint property method.

alert|httpPost(url ...string)

Returns: HTTPPostNode

InfluxDBOut

Create an influxdb output node that will store the incoming data into InfluxDB.

alert|influxDBOut()

Returns: InfluxDBOutNode

Join

Join this node with other nodes. The data is joined on timestamp.

alert|join(others ...Node)

Returns: JoinNode

K8sAutoscale

Create a node that can trigger autoscale events for a kubernetes cluster.

alert|k8sAutoscale()

Returns: K8sAutoscaleNode

KapacitorLoopback

Create an kapacitor loopback node that will send data back into Kapacitor as a stream.

alert|kapacitorLoopback()

Returns: KapacitorLoopbackNode

Last

Select the last point.

alert|last(field string)

Returns: InfluxQLNode

Max

Select the maximum point.

alert|max(field string)

Returns: InfluxQLNode

Mean

Compute the mean of the data.

alert|mean(field string)

Returns: InfluxQLNode

Median

Compute the median of the data.

Note: This method is not a selector. If you want the median point, use .percentile(field, 50.0).

alert|median(field string)

Returns: InfluxQLNode

Min

Select the minimum point.

alert|min(field string)

Returns: InfluxQLNode

Mode

Compute the mode of the data.

alert|mode(field string)

Returns: InfluxQLNode

MovingAverage

Compute a moving average of the last window points. No points are emitted until the window is full.

alert|movingAverage(field string, window int64)

Returns: InfluxQLNode

Percentile

Select a point at the given percentile. This is a selector function, no interpolation between points is performed.

alert|percentile(field string, percentile float64)

Returns: InfluxQLNode

Sample

Create a new node that samples the incoming points or batches.

One point will be emitted every count or duration specified.

alert|sample(rate interface{})

Returns: SampleNode

Shift

Create a new node that shifts the incoming points or batches in time.

alert|shift(shift time.Duration)

Returns: ShiftNode

Sideload

Create a node that can load data from external sources.

alert|sideload()

Returns: SideloadNode

Spread

Compute the difference between min and max points.

alert|spread(field string)

Returns: InfluxQLNode

StateCount

Create a node that tracks number of consecutive points in a given state.

alert|stateCount(expression ast.LambdaNode)

Returns: StateCountNode

StateDuration

Create a node that tracks duration in a given state.

alert|stateDuration(expression ast.LambdaNode)

Returns: StateDurationNode

Stats

Create a new stream of data that contains the internal statistics of the node. The interval represents how often to emit the statistics based on real time. This means the interval time is independent of the times of the data points the source node is receiving.

alert|stats(interval time.Duration)

Returns: StatsNode

Stddev

Compute the standard deviation.

alert|stddev(field string)

Returns: InfluxQLNode

Sum

Compute the sum of all values.

alert|sum(field string)

Returns: InfluxQLNode

SwarmAutoscale

Create a node that can trigger autoscale events for a Docker swarm cluster.

alert|swarmAutoscale()

Returns: SwarmAutoscaleNode

Top

Select the top num points for field and sort by any extra tags or fields.

alert|top(num int64, field string, fieldsAndTags ...string)

Returns: InfluxQLNode

Trickle

Create a new node that converts batch data to stream data.

alert|trickle()

Returns: TrickleNode

Union

Perform the union of this node and all other given nodes.

alert|union(node ...Node)

Returns: UnionNode

Where

Create a new node that filters the data stream by a given expression.

alert|where(expression ast.LambdaNode)

Returns: WhereNode

Window

Create a new node that windows the stream by time.

NOTE: Window can only be applied to stream edges.

alert|window()

Returns: WindowNode


Was this page helpful?

Thank you for your feedback!


The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

New capabilities, including faster query performance and management tooling advance the InfluxDB v3 product line. InfluxDB Clustered is now generally available.

InfluxDB v3 performance and features

The InfluxDB v3 product line has seen significant enhancements in query performance and has made new management tooling available. These enhancements include an operational dashboard to monitor the health of your InfluxDB cluster, single sign-on (SSO) support in InfluxDB Cloud Dedicated, and new management APIs for tokens and databases.

Learn about the new v3 enhancements


InfluxDB Clustered general availability

InfluxDB Clustered is now generally available and gives you the power of InfluxDB v3 in your self-managed stack.

Talk to us about InfluxDB Clustered