Manage the node lifecycle

Every InfluxDB 3 Core server process registers itself as a node in the catalog—the metadata store that tracks databases, tables, and nodes. The catalog is the source of truth for a node’s identity and state, and it persists in object storage—independently of the process, its container, or its host. Understanding how a node moves through its states helps you restart, upgrade, scale, and decommission InfluxDB 3 Core safely.

Node states

The catalog records one of the following states for each node:

StateDescription
runningThe node started and registered itself in the catalog
stoppingA graceful stop was requested, but the node hasn’t acknowledged it yet
stoppedThe node acknowledged its final snapshot and completed shutdown
removingThe node is marked for permanent removal from the cluster

Core uses running and stopped

InfluxDB 3 Core runs a single node and doesn’t provide node management commands, so a InfluxDB 3 Core node only ever reads as running or stopped. The stopping and removing states apply to InfluxDB 3 Enterprise clusters, where an operator can stop and remove individual nodes.

Node identity has two parts:

  • Node ID: The name you assign with --node-id. It identifies the node across restarts.
  • Instance ID: A UUID that InfluxDB 3 Core generates the first time a node ID registers. A node that restarts with the same node ID reuses its existing instance ID.

Lifecycle overview

stateDiagram-v2 [*] --> running: influxdb3 serve running --> stopped: SIGTERM or SIGINT stopped --> running: influxdb3 serve (same node ID)

Register a node

When you start a server with influxdb3 serve, the node registers itself in the catalog and enters the running state:

influxdb3 serve \
  --node-id 
NODE_ID
\
--object-store file \ --data-dir ~/.influxdb3

Replace NODE_ID with a unique identifier for the node.

Registration is how a node claims its node ID. If the node ID already exists in the catalog, InfluxDB 3 Core applies the re-registration rules before accepting the node.

Stop a node

Stop InfluxDB 3 Core by signaling the process—for example, press Ctrl-c in the foreground, run systemctl stop influxdb3, or stop the container. SIGTERM and SIGINT both start a graceful shutdown.

During a graceful shutdown, the node does the following:

  1. Stops accepting writes.
  2. Flushes the write-ahead log (WAL) buffer to object storage.
  3. Waits for an in-progress snapshot to finish.
  4. Marks itself stopped in the catalog.

Because the final flush writes buffered data to the WAL in object storage, acknowledged writes survive the shutdown. When the node restarts, WAL replay restores any writes that weren’t yet captured in a snapshot.

Give the process time to shut down

A graceful shutdown isn’t instantaneous. If your init system, container runtime, or orchestrator sends SIGKILL before the flush completes, the node can’t finish its final flush. For guidance on timeouts, see Deploy with an orchestrator.

Re-register a node

Whether a node ID can be claimed again depends on the current state of the node in the catalog:

Current stateCan register again?
stoppedYes—any instance can take over the node ID
runningOnly the same instance ID (an idempotent retry or a restart)

Because a node restarting with the same node ID reuses its existing instance ID, an ordinary restart always satisfies these rules—even if the node still reads as running after an ungraceful stop.

Deploy with an orchestrator

Helm, Kubernetes, and Ansible deployments drive the node lifecycle on your behalf. The following guidance keeps orchestrated restarts on the graceful path.

Kubernetes and Helm

Give each node a stable node ID. Use a StatefulSet so pod names are stable and ordinal-based, and derive --node-id from the pod name.

The official InfluxDB 3 Core Helm chart does this already—it runs a StatefulSet and sets --node-id=$(POD_NAME) from metadata.name.

A Deployment generates a new random pod name on every rollout, which registers a new node in the catalog on each restart and leaves the old entries behind.

Set a termination grace period that fits your WAL. Kubernetes sends SIGTERM, waits terminationGracePeriodSeconds (default 30), and then sends SIGKILL. A node that’s still flushing when SIGKILL arrives stops ungracefully. Set terminationGracePeriodSeconds well above your observed shutdown time:

spec:
  template:
    spec:
      terminationGracePeriodSeconds: 300

The Helm chart doesn’t set a grace period

The InfluxDB 3 Core Helm chart doesn’t set terminationGracePeriodSeconds, so pods inherit the Kubernetes default of 30 seconds. For nodes with a large WAL, raise it in your values.yaml overrides and confirm the shutdown completes in the pod logs.

Restarts reuse the same node. A helm upgrade or kubectl rollout restart terminates and recreates the pod with the same name, so the node re-registers with its existing node ID and replays its WAL.

Ansible and systemd

Let systemd send SIGTERM. The systemd default KillSignal is SIGTERM, which starts a graceful shutdown—don’t override it with SIGKILL.

Raise TimeoutStopSec. If the node doesn’t exit within TimeoutStopSec, systemd escalates to SIGKILL. Set it above your observed shutdown time:

[Service]
KillSignal=SIGTERM
TimeoutStopSec=300

Wait for the node to return to running. Confirm the node re-registered and finished WAL replay before you send traffic to it again.

Never use kill -9. Ad hoc kill -9, docker kill, and force-stopped containers all skip the final flush.

Verify node state

InfluxDB 3 Core doesn’t provide node management commands, but you can query the system.nodes table in the _internal database to see the node’s registered state:

influxdb3 query \
  --database _internal \
  --token 
AUTH_TOKEN
\
"SELECT node_id, mode, state, updated_at FROM system.nodes"

Replace AUTH_TOKEN with a token that has permission to query the _internal database.

Troubleshoot node lifecycle issues

The node didn’t shut down cleanly

If the process was killed before it finished flushing, restart it with the same node ID and object store configuration. WAL replay restores acknowledged writes that weren’t yet captured in a snapshot.

A new node appears after each restart

Each restart registered a new node ID. Check that your deployment assigns a stable --node-id—see Deploy with an orchestrator.


Was this page helpful?

Thank you for your feedback!