Documentation

InfluxDB Clustered

Partition templates

Use partition templates to define the patterns used to generate partition keys. A partition key uniquely identifies a partition and is used to name the partition Parquet file in the Object store.

A partition template consists of 1-8 template parts—dimensions to partition data by. There are three types of template parts:

  • tag: An InfluxDB tag to partition by.
  • tag bucket: An InfluxDB tag and number of “buckets” to group tag values into. Data is partitioned by the tag bucket rather than each distinct tag value.
  • time: A Rust strftime date and time string that specifies the time interval to partition data by. The smallest unit of time included in the time part template is the interval used to partition data.

A partition template can include up to 7 total tag and tag bucket parts and only 1 time part.

Restrictions

Template part size limit

Each template part is limited to 200 bytes in length. Anything longer will be truncated at 200 bytes and appended with #.

Partition key size limit

With the truncation of template parts, the maximum length of a partition key is 1,607 bytes (1.57 KiB).

Reserved keywords

The following reserved keywords cannot be used in partition templates:

  • time

Reserved Characters

If used in template parts, non-ASCII characters and the following reserved characters must be percent encoded:

  • |: Partition key part delimiter
  • !: Null or missing partition key part
  • ^: Empty string partition key part
  • #: Key part truncation marker
  • %: Required for unambiguous reversal of percent encoding

Tag part templates

Tag part templates consist of a tag key to partition by. Generated partition keys include the unique tag value specific to each partition.

Tag bucket part templates

Tag bucket part templates consist of a tag key to partition by and the number of “buckets” to partition tag values into–for example:

customerID,500

Values of the customerID tag are bucketed into 500 distinct “buckets.” Each bucket is identified by the remainder of the tag value hashed into a 32bit integer divided by the specified number of buckets:

hash(tagValue) % N

Generated partition keys include the unique tag bucket identifier specific to each partition.

Supported number of tag buckets: 1-1,000

Tag buckets should be used to partition by high cardinality tags or tags with an unknown number of distinct values.

Time part templates

Time part templates use Rust strftime date and time formatting syntax to specify time format in partition keys. The smallest unit of time included in the time part template is the interval used to partition data.

Avoid partitioning by less than one day

We do not recommend using time intervals less than one day to partition data. This can result in over-partitioned data and may hurt query performance.

The following is adapted from the Rust strftime source code.

Date specifiers

Variable Example Description
%Y 2001 The full proleptic Gregorian year, zero-padded to 4 digits. chrono supports years from -262144 to 262143. Note: years before 1 BCE or after 9999 CE, require an initial sign (+/-).
%C 20 The proleptic Gregorian year divided by 100, zero-padded to 2 digits. 1
%y 01 The proleptic Gregorian year modulo 100, zero-padded to 2 digits. 1
%m 07 Month number (01–12), zero-padded to 2 digits.
%b Jul Abbreviated month name. Always 3 letters.
%B July Full month name. Also accepts corresponding abbreviation in parsing.
%h Jul Same as %b.
%d 08 Day number (01–31), zero-padded to 2 digits.
%e 8 Same as %d but space-padded. Same as %_d.
%a Sun Abbreviated weekday name. Always 3 letters.
%A Sunday Full weekday name. Also accepts corresponding abbreviation in parsing.
%w 0 Sunday = 0, Monday = 1, …, Saturday = 6.
%u 7 Monday = 1, Tuesday = 2, …, Sunday = 7. (ISO 8601)
%U 28 Week number starting with Sunday (00–53), zero-padded to 2 digits. 2
%W 27 Same as %U, but week 1 starts with the first Monday in that year instead.
%G 2001 Same as %Y but uses the year number in ISO 8601 week date. 3
%g 01 Same as %y but uses the year number in ISO 8601 week date. 3
%V 27 Same as %U but uses the week number in ISO 8601 week date (01–53). 3
%j 189 Day of the year (001–366), zero-padded to 3 digits.
%D 07/08/01 Month-day-year format. Same as %m/%d/%y.
%x 07/08/01 Locale’s date representation (e.g., 12/31/99).
%F 2001-07-08 Year-month-day format (ISO 8601). Same as %Y-%m-%d.
%v 8-Jul-2001 Day-month-year format. Same as %e-%b-%Y.

Time specifiers

Variable Example Description
%H 00 Hour number (00–23), zero-padded to 2 digits.
%k 0 Same as %H but space-padded. Same as %_H.
%I 12 Hour number in 12-hour clocks (01–12), zero-padded to 2 digits.
%l 12 Same as %I but space-padded. Same as %_I.
%P am am or pm in 12-hour clocks.
%p AM AM or PM in 12-hour clocks.
%M 34 Minute number (00–59), zero-padded to 2 digits.
%S 60 Second number (00–60), zero-padded to 2 digits. 4
%f 26490000 Number of nanoseconds since last whole second. 5
%.f .026490 Decimal fraction of a second. Consumes the leading dot. 5
%.3f .026 Decimal fraction of a second with a fixed length of 3.
%.6f .026490 Decimal fraction of a second with a fixed length of 6.
%.9f .026490000 Decimal fraction of a second with a fixed length of 9.
%3f 026 Decimal fraction of a second like %.3f but without the leading dot.
%6f 026490 Decimal fraction of a second like %.6f but without the leading dot.
%9f 026490000 Decimal fraction of a second like %.9f but without the leading dot.
%R 00:34 Hour-minute format. Same as %H:%M.
%T 00:34:60 Hour-minute-second format. Same as %H:%M:%S.
%X 00:34:60 Locale’s time representation (e.g., 23:13:48).
%r 12:34:60 AM Locale’s 12 hour clock time. (e.g., 11:11:04 PM). Falls back to %X if the locale does not have a 12 hour clock format.

Time zone specifiers

Variable Example Description
%Z ACST Local time zone name. Skips all non-whitespace characters during parsing. Identical to %:z when formatting. 6
%z +0930 Offset from the local time to UTC (with UTC being +0000).
%:z +09:30 Same as %z but with a colon.
%::z +09:30:00 Offset from the local time to UTC with seconds.
%:::z +09 Offset from the local time to UTC without minutes.
%#z +09 Parsing only: Same as %z but allows minutes to be missing or present.

Date and time specifiers

Variable Example Description
%c Sun Jul 8 00:34:60 2001 Locale’s date and time (e.g., Thu Mar 3 23:05:25 2005).
%+ 2001-07-08T00:34:60.026490+09:30 ISO 8601 / RFC 3339 date & time format. 7
%s 994518299 UNIX timestamp, the number of seconds since 1970-01-01 00:00 UTC. 8

Special specifiers

Variable Example Description
%t Literal tab (\t).
%n Literal newline (\n).
%% Literal percent sign.

It is possible to override the default padding behavior of numeric specifiers %?. This is not allowed for other specifiers and results in the BAD_FORMAT error.

Modifier Description
%-? Suppresses any padding including spaces and zeroes. (e.g. %j = 012, %-j = 12)
%_? Uses spaces as a padding. (e.g. %j = 012, %_j = 12)
%0? Uses zeroes as a padding. (e.g. %e = 9, %0e = 09)

Notes:

  1. %C, %y: This is floor division, so 100 BCE (year number -99) will print -1 and 99 respectively. ↩︎ ↩︎

  2. %U: Week 1 starts with the first Sunday in that year. It is possible to have week 0 for days before the first Sunday. ↩︎

  3. %G, %g, %V: Week 1 is the first week with at least 4 days in that year. Week 0 does not exist, so this should be used with %G or %g. ↩︎ ↩︎ ↩︎

  4. %S: It accounts for leap seconds, so 60 is possible. ↩︎

  5. %f, %.f:
    %f and %.f are notably different formatting specifiers.
    %f counts the number of nanoseconds since the last whole second, while %.f is a fraction of a second.
    Example: 7μs is formatted as 7000 with %f, and formatted as .000007 with %.f. ↩︎ ↩︎

  6. %Z: Since chrono is not aware of timezones beyond their offsets, this specifier only prints the offset when used for formatting. The timezone abbreviation will NOT be printed. See this issue for more information.

    Offset will not be populated from the parsed data, nor will it be validated. Timezone is completely ignored. Similar to the glibc strptime treatment of this format code.

    It is not possible to reliably convert from an abbreviation to an offset, for example CDT can mean either Central Daylight Time (North America) or China Daylight Time. */ ↩︎

  7. %+: Same as %Y-%m-%dT%H:%M:%S%.f%:z, i.e. 0, 3, 6 or 9 fractional digits for seconds and colons in the time zone offset.

    This format also supports having a Z or UTC in place of %:z. They are equivalent to +00:00.

    Note that all T, Z, and UTC are parsed case-insensitively.

    The typical strftime implementations have different (and locale-dependent) formats for this specifier. While Chrono’s format for %+ is far more stable, it is best to avoid this specifier if you want to control the exact output. ↩︎

  8. %s: This is not padded and can be negative. For the purpose of Chrono, it only accounts for non-leap seconds so it slightly differs from ISO C strftime behavior. ↩︎

Was this page helpful?

Thank you for your feedback!

© 2024 InfluxData, Inc.

What is your InfluxDB cluster URL?

Enter your InfluxDB cluster URL and we’ll update code examples for you.

Enter cluster URL

Thank you for your feedback!

Let us know what we can do better:

Thank you!

No thanks

The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more