Manage Kapacitor subscriptions
Kapacitor is tightly integrated with InfluxDB through the use of InfluxDB subscriptions, local or remote endpoints to which all data written to InfluxDB is copied. Kapacitor subscribes to InfluxDB allowing it to capture, manipulate, and act on your data.
How Kapacitor subscriptions work
Kapacitor allows you to manipulate and act on data as it is written into InfluxDB. Rather than querying InfluxDB for data (except when using the BatchNode), all data is copied to your Kapacitor server or cluster through an InfluxDB subscription. This reduces the query load on InfluxDB and isolates overhead associated with data manipulation to your Kapacitor server or cluster.
On startup, Kapacitor checks for a subscription in InfluxDB with a name matching the Kapacitor server or cluster ID.
This ID is stored inside of /var/lib/kapacitor/.
If the ID file doesn’t exist on startup, Kapacitor creates it.
If a subscription matching the Kapacitor ID doesn’t exist in InfluxDB, Kapacitor
creates a new subscription in InfluxDB.
This process ensures that when Kapacitor stops, it reconnects to the same subscription
on restart as long as the contents of /var/lib/kapacitor/ remain intact.
InfluxDB user must have admin privileges
The InfluxDB user used to create subscriptions for Kapacitor must have
admin privileges.
Configure the InfluxDB user to use with the [influxdb].username setting
in your Kapacitor configuration file.
The directory in which Kapacitor stores its ID can be configured with the
data-dir root configuration option
in the kapacitor.conf.
Kapacitor IDs in containerized or ephemeral filesystems
In containerized environments, filesystems are considered ephemeral and typically
do not persist between container stops and restarts.
If /var/lib/kapacitor/ is not persisted, Kapacitor will create a new InfluxDB subscription
on startup, resulting in unnecessary “duplicate” subscriptions.
You will then need to manually drop the unnecessary subscriptions.
To avoid this, persist the /var/lib/kapacitor directory.
Many persistence strategies are available and which to use depends on your
specific architecture and containerization technology.
Configure Kapacitor subscriptions
Kapacitor subscription configuration options are available under the [[influxdb]] section in the kapacitor.conf.
Below is an example of subscription-specific configuration options followed by a description of each.
Example Kapacitor subscription configuration
[[influxdb]]
# ...
disable-subscriptions = false
subscription-mode = "server"
subscription-protocol = "http"
subscriptions-sync-interval = "1m0s"
# ...
[influxdb.subscriptions]
my_database1 = [ "default", "longterm" ]
[influxdb.excluded-subscriptions]
my_database2 = [ "default", "shortterm" ]
disable-subscriptions
Set to true to disable all subscriptions.
subscription-mode
Defines the subscription mode of Kapacitor. Available options:
"server""cluster"
The default setting is cluster.
subscription-protocol
Defines which protocol to use for subscriptions. Available options:
"udp""http""https"
The default setting is http.
[influxdb.subscriptions]
Defines a set of databases and retention policies to subscribe to.
If empty, Kapacitor will subscribe to all databases and retention policies except for those listed in
[influxdb.excluded-subscriptions].
[influxdb.subscriptions]
# Pattern:
db_name = <list of retention policies>
# Example:
my_database = [ "default", "longterm" ]
[influxdb.excluded-subscriptions]
Defines a set of databases and retention policies to exclude from subscriptions.
[influxdb.excluded-subscriptions]
# Pattern:
db_name = <list of retention policies>
# Example:
my_database = [ "default", "longterm" ]
Only one of [influxdb.subscriptions] or [influxdb.excluded-subscriptions]
need be defined. They essentially fulfill the same purpose in different ways,
but specific use cases do lend themselves to one or the other.
Troubleshooting
View the Kapacitor server or cluster ID
There are two ways to view your Kapacitor server or cluster ID:
-
View the contents of
/var/lib/kapacitor/server.idor/var/lib/kapacitor/cluster.id.The location of ID files depends on your operating system and the
data-dirsetting in yourkapacitor.conf. -
Run the following command:
kapacitor stats generalThe server and cluster IDs are included in the output.
Duplicate Kapacitor subscriptions
Duplicate Kapacitor subscriptions are often caused by the contents of /var/lib/kapacitor
not persisting between restarts as described above.
The solution is to ensure the contents of this director are persisted.
Any duplicate Kapacitor subscriptions already created will need to be manually removed.
Was this page helpful?
Thank you for your feedback!
Support and feedback
Thank you for being part of our community! We welcome and encourage your feedback and bug reports for Kapacitor and this documentation. To find support, use the following resources: