Documentation

Enable trace logging

Learn how to enable trace logging to help you identify performance bottlenecks and troubleshoot problems in queries.

When you enable trace logging for a query, InfluxDB propagates your trace ID through system processes and collects additional log information. InfluxDB Support can then use the trace ID that you provide to filter, collate, and analyze log information for the query run. The tracing system follows the OpenTelemetry traces model for providing observability into a request.

Avoid unnecessary tracing

Only enable tracing for a query when you need to request troubleshooting help from InfluxDB Support. To manage resources, InfluxDB has an upper limit for the number of trace requests. Too many traces can cause InfluxDB to evict log information.

To enable tracing for a query, include the influx-trace-id header in your query request.

Syntax

Use the following syntax for the influx-trace-id header:

influx-trace-id: TRACE_ID:1112223334445:0:1

In the header value, replace the following:

  • TRACE_ID: a unique string, 8-16 bytes long, encoded as hexadecimal (32 maximum hex characters). The trace ID should uniquely identify the query run.
  • :1112223334445:0:1: InfluxDB constant values (required, but ignored)

Example

The following examples show how to create and pass a trace ID to enable query tracing in InfluxDB:

Use the InfluxDBClient3 InfluxDB Python client and pass the headers argument in the query() method.

from influxdb_client_3 import InfluxDBClient3
import secrets

def use_flightcalloptions_trace_header():
  print('# Use FlightCallOptions to enable tracing.')
  client = InfluxDBClient3(token=f"
DATABASE_TOKEN
"
,
host=f"cluster-id.influxdb.io", database=f"
DATABASE_NAME
"
)
# Generate a trace ID for the query: # 1. Generate a random 8-byte value as bytes. # 2. Encode the value as hexadecimal. random_bytes = secrets.token_bytes(8) trace_id = random_bytes.hex() # Append required constants to the trace ID. trace_value = f"{trace_id}:1112223334445:0:1" # Encode the header key and value as bytes. # Create a list of header tuples. headers = [((b"influx-trace-id", trace_value.encode('utf-8')))] sql = "SELECT * FROM home WHERE time >= now() - INTERVAL '30 days'" influxql = "SELECT * FROM home WHERE time >= -90d" # Use the query() headers argument to pass the list as FlightCallOptions. client.query(sql, headers=headers) client.close() use_flightcalloptions_trace_header()

Use the InfluxDBClient3 InfluxDB Python client and flight.ClientMiddleware to pass and inspect headers.

Tracing response header

With tracing enabled and a valid trace ID in the request, InfluxDB’s DoGet action response contains a header with the trace ID that you sent.

Trace response header syntax
trace-id: TRACE_ID

Inspect Flight response headers

To inspect Flight response headers when using a client library, pass a FlightClientMiddleware instance. that defines a middleware callback function for the onHeadersReceived event (the particular function name you use depends on the client library language).

The following example uses Python client middleware that adds request headers and extracts the trace ID from the DoGet response headers:

import pyarrow.flight as flight

class TracingClientMiddleWareFactory(flight.ClientMiddleware):
  # Defines a custom middleware factory that returns a middleware instance.
    def __init__(self):
        self.request_headers = []
        self.response_headers = []
        self.traces  = []

    def addRequestHeader(self, header):
        self.request_headers.append(header)

    def addResponseHeader(self, header):
        self.response_headers.append(header)

    def addTrace(self, traceid):
        self.traces.append(traceid)

    def createTrace(self, traceid):
      # Append InfluxDB constants to the trace ID.
      trace = f"{traceid}:1112223334445:0:1"

      # To the list of request headers,
      # add a tuple with the header key and value as bytes.
      self.addRequestHeader((b"influx-trace-id", trace.encode('utf-8')))

    def start_call(self, info):
        return TracingClientMiddleware(info.method, self)

class TracingClientMiddleware(flight.ClientMiddleware):
  # Defines middleware with client event callback methods.
    def __init__(self, method, callback_obj):
        self._method = method
        self.callback = callback_obj

    def call_completed(self, exception):
      print('callback: call_completed')
      if(exception):
        print(f"  ...with exception: {exception}")

    def sending_headers(self):
      print('callback: sending_headers: ', self.callback.request_headers)
      if len(self.callback.request_headers) > 0:
        return dict(self.callback.request_headers)

    def received_headers(self, headers):
      self.callback.addResponseHeader(headers)
      # For the DO_GET action, extract the trace ID from the response headers.
      if str(self._method) == "FlightMethod.DO_GET" and "trace-id" in headers:
          trace_id = headers["trace-id"][0]
          self.callback.addTrace(trace_id)

from influxdb_client_3 import InfluxDBClient3
import secrets

def use_middleware_trace_header():
  print('# Use Flight client middleware to enable tracing.')

  # Instantiate the middleware.
  res = TracingClientMiddleWareFactory()

  # Instantiate the client, passing in the middleware instance that provides
  # event callbacks for the request.
  client = InfluxDBClient3(token=f"
DATABASE_TOKEN
"
,
host=f"cluster-id.influxdb.io", database=f"
DATABASE_NAME
"
,
flight_client_options={"middleware": (res,)}) # Generate a trace ID for the query: # 1. Generate a random 8-byte value as bytes. # 2. Encode the value as hexadecimal. random_bytes = secrets.token_bytes(8) trace_id = random_bytes.hex() res.createTrace(trace_id) sql = "SELECT * FROM home WHERE time >= now() - INTERVAL '30 days'" client.query(sql) client.close() assert trace_id in res.traces[0], "Expect trace ID in DoGet response." use_middleware_trace_header()

Replace the following:

  • DATABASE_NAME: your InfluxDB Cloud Dedicated database
  • DATABASE_TOKEN: a database token with sufficient permissions to the specified database

Store or log your query trace ID to ensure you can provide it to InfluxDB Support for troubleshooting.

After you run your query with tracing enabled, do the following:

  • Remove the tracing header from subsequent runs of the query (to avoid unnecessary tracing).
  • Provide the trace ID in a request to InfluxDB Support.

Retrieve system information for a query

If you enable trace logging for a query, the trace-id appears in the system.queries table.


Was this page helpful?

Thank you for your feedback!


Introducing InfluxDB Clustered

A highly available InfluxDB 3.0 cluster on your own infrastructure.

InfluxDB Clustered is a highly available InfluxDB 3.0 cluster built for high write and query workloads on your own infrastructure.

InfluxDB Clustered is currently in limited availability and is only available to a limited group of InfluxData customers. If interested in being part of the limited access group, please contact the InfluxData Sales team.

Learn more
Contact InfluxData Sales

The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Flux is going into maintenance mode and will not be supported in InfluxDB 3.0. This was a decision based on the broad demand for SQL and the continued growth and adoption of InfluxQL. We are continuing to support Flux for users in 1.x and 2.x so you can continue using it with no changes to your code. If you are interested in transitioning to InfluxDB 3.0 and want to future-proof your code, we suggest using InfluxQL.

For information about the future of Flux, see the following: