XPath Protocol Buffers input data format

Use the xpath_protobuf input data format to parse Protocol Buffers data into Telegraf metrics using XPath 1.0 expressions.

xpath_protobuf is one of the formats provided by the Telegraf XPath parser. It shares its configuration options and query syntax with the XPath JSON input data format. See that page for the complete option reference, the document-tree mapping, and query examples. This page covers the additional settings required for Protocol Buffers.

Configuration

[[inputs.file]]
  files = ["example.dat"]
  data_format = "xpath_protobuf"

  ## Protocol-buffer definition file(s)
  xpath_protobuf_files = ["sparkplug_b.proto"]

  ## Name of the protocol-buffer message type to use in a fully qualified form.
  xpath_protobuf_type = "org.eclipse.tahu.protobuf.Payload"

  ## List of paths to use when looking up imported protocol-buffer
  ## definition files.
  # xpath_protobuf_import_paths = ["."]

  ## Number of (header) bytes to ignore before parsing the message.
  # xpath_protobuf_skip_bytes = 0

  ## Keep native data types instead of converting everything to strings.
  # xpath_native_types = false

  ## Print the internal document when debug logging is enabled.
  ## Especially useful for binary formats like protocol buffers.
  # xpath_print_document = false

  [[inputs.file.xpath]]
    # metric_selection = "/metrics/*"
    [inputs.file.xpath.fields]
      value = "number(value)"

Protocol Buffers settings

xpath_protobuf_files

The names of the protocol-buffer definition files (.proto).

Type: array of strings
Default: None; required

xpath_protobuf_type

The top-level message type to use for deserializing the data, in fully qualified form. Usually, this is constructed from the package name and the message name in the protocol-buffer definition file as <package name>.<message name>.

Type: string
Default: None; required

xpath_protobuf_import_paths

Paths to search for imported protocol-buffer definition files. If your .proto file imports other definitions with the import statement, add the directories containing those files.

Type: array of strings
Default: ["."] (the current working directory when starting Telegraf)

For example, if A.proto in /data/my_proto_files imports B.proto from the same directory:

syntax = "proto3";

package foo;

import "B.proto";

message Measurement {
    ...
}

use the following settings:

[[inputs.file]]
  files = ["example.dat"]

  data_format = "xpath_protobuf"
  xpath_protobuf_files = ["A.proto"]
  xpath_protobuf_type = "foo.Measurement"
  xpath_protobuf_import_paths = [".", "/data/my_proto_files"]

xpath_protobuf_skip_bytes

The number of bytes to skip before parsing the protocol-buffer message. Useful when the raw data has a header, for example a message-length header or a gRPC header.

Type: integer
Default: 0

Known headers and the corresponding values:

Byte-array fields

Protocol-buffer messages often encode data as byte arrays. By default, byte-array fields convert to strings. Use fields_bytes_as_hex or fields_bytes_as_base64 to encode them as hex or base64 strings instead.

Because Protocol Buffers is a binary format, use xpath_print_document = true with debug logging to inspect the parsed document and work out your queries.