Documentation

Migrate data from InfluxDB 1.x to InfluxDB Cloud Serverless

To migrate data from an InfluxDB 1.x OSS or Enterprise instance to InfluxDB Cloud Serverless powered by the v3 storage engine, export the data as line protocol and write the exported data to a bucket in your InfluxDB Cloud Serverless organization. Because full data migrations will likely exceed your organizations’ limits and adjustable quotas, migrate your data in batches.

All write requests are subject to your InfluxDB Cloud Serverless organization’s rate limits and adjustable quotas.

Before you migrate

Before you migrate from InfluxDB 1.x to InfluxDB Cloud Serverless, there are schema design practices supported by the TSM storage engine that are not supported in the InfluxDB v3 storage engine. Specifically, InfluxDB v3 enforces the following schema restrictions:

  • You can’t use duplicate names for tags and fields
  • Measurements can contain up to 200 columns where each column represents time, a field, or a tag.

For more information, see Schema restrictions.

If your schema does not adhere to these restrictions, you must update your schema before migrating to InfluxDB Cloud Serverless.

Fix duplicate tag and field names

Fix measurements with more than 200 total columns

Tools to use

The migration process uses the following tools:

InfluxDB 1.x and 2.x CLIs are unique

If both the InfluxDB 1.x and 2.x influx CLIs are installed in your $PATH, rename one of the the binaries to ensure you’re executing commands with the correct CLI.

Migrate data

  1. Export data from your InfluxDB 1.x instance as line protocol.

    Use the InfluxDB 1.x influx_inspect export utility to export data as line protocol and store it in a file. Include the following:

    • (Required) -lponly flag to export line protocol without InfluxQL DDL or DML.
    • (Required) -out flag with a path to an output file. Default is ~/.influxdb/export. Any subsequent export commands without the output file defined will overwrite the existing export file.
    • -compress flag to use gzip to compress the output.
    • -datadir flag with the path to your InfluxDB 1.x data directory. Only required if the data directory is at a non-default location. For information about default locations, see InfluxDB OSS 1.x file system layout or InfluxDB Enterprise 1.x file system layout.
    • -waldir flag with the path to your InfluxDB 1.x wal directory. Only required if the wal directory is at a non-default location. For information about default locations, see InfluxDB OSS 1.x file system layout or InfluxDB Enterprise 1.x file system layout.
    • -database flag with a specific database name to export. By default, all databases are exported.
    • -retention flag with a specific retention policy to export. By default, all retention policies are exported.
    • -start flag with an RFC3339 timestamp that defines the earliest time to export. Default is 1677-09-20T16:27:54-07:44.
    • -end flag with an RFC3339 timestamp that defines the latest time to export. Default is 2262-04-11T16:47:16-07:00.

    We recommend exporting each database and retention policy combination separately to easily write the exported data into corresponding InfluxDB Cloud Serverless buckets.

    Export all data in a database and retention policy to a file
    influx_inspect export \
      -lponly \
      -database example-db \
      -retention example-rp \
      -out path/to/export-file.lp
    
    View more export command examples:

    Export all data to a file

    Export all data to a compressed file

    Export data within time bounds to a file

    Export a database and all its retention policies to a file

    Export a specific database and retention policy to a file

    Export all data from non-default data and wal directories

  2. Create InfluxDB Cloud Serverless buckets for each InfluxDB 1.x database and retention policy combination. InfluxDB Cloud Serverless combines InfluxDB 1.x databases and retention policies into buckets–named locations for time series data with specified retention periods.

    View example 1.x databases and retention policies as InfluxDB Cloud buckets

    Use the InfluxDB 2.x influx CLI or the InfluxDB Cloud Serverless user interface (UI) to create a bucket.

    Use the influx bucket create command to create a new bucket.

    Provide the following:

    influx bucket create \
      --name example-db/autogen \
      --retention 7d
    
    1. Go to cloud2.influxdata.com in a browser to log in and access the InfluxDB UI.

    2. Navigate to Load Data > Buckets using the left navigation bar.

    1. Click + Create bucket.

    2. Provide a bucket name (for example: example-db/autogen) and select a retention period. Supported retention periods depend on your InfluxDB Cloud Serverless plan.

    3. Click Create.

  3. Write the exported line protocol to your InfluxDB Cloud Serverless organization.

    Use the InfluxDB 2.x CLI to write data to InfluxDB Cloud Serverless. While you can use the /api/v2/write API endpoint to write data directly, the influx write command lets you define the rate at which data is written to avoid exceeding your organization’s rate limits.

    Use the influx write command and include the following:

    • InfluxDB Cloud Serverless connection and authentication credentials
    • -b, --bucket flag to identify the target bucket.
    • -f, --file flag with the path to the line protocol file to import.
    • -rate-limit flag with a rate limit that matches your InfluxDB Cloud organization’s write rate limit.
    • --compression flag to identify the compression type of the import file. Options are none or gzip. Default is none.

    Authentication credentials

    The examples below assume your InfluxDB host, organization, and token are provided by either the active influx CLI configuration or by environment variables (INFLUX_HOST, INFLUX_ORG, and INFLUX_TOKEN). If you do not have a CLI configuration set up or the environment variables set, include these required credentials for each command with the following flags:

    • --host: InfluxDB host
    • -o, --org or --org-id: InfluxDB organization name or ID
    • -t, --token: InfluxDB API token
    influx write \
      --bucket example-db/autogen \
      --file path/to/export-file.lp \
      --rate-limit "300 MB / 5 min"
    
    influx write \
      --bucket example-db/autogen \
      --file path/to/export-file.lp.gzip \
      --rate-limit "300 MB / 5 min" \
      --compression gzip
    

    Repeat for each export file and target bucket.


Was this page helpful?

Thank you for your feedback!


The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Flux is going into maintenance mode and will not be supported in InfluxDB 3.0. This was a decision based on the broad demand for SQL and the continued growth and adoption of InfluxQL. We are continuing to support Flux for users in 1.x and 2.x so you can continue using it with no changes to your code. If you are interested in transitioning to InfluxDB 3.0 and want to future-proof your code, we suggest using InfluxQL.

For information about the future of Flux, see the following:

InfluxDB Cloud Serverless