Documentation

Query CSV data sources

Use csv.from() and experimental csv.from() to query CSV data with Flux. Query a CSV string, CSV file, or CSV data from a URL. Import the csv or experimental/csv package.

CSV parsing modes

csv.from() supports two CSV parsing modes:

  • annotations: (Default) Use CSV annotations to determine column data types and table grouping.
  • raw: Parse all columns as strings and use the first row as the header row and all subsequent rows as data.

When using the annotations parsing mode, CSV data must include all annotation rows (#datatype, #group, and #default).

Results structure

The structure of results returned by csv.from() depends on the parsing mode used.

  • annotations: csv.from() returns a stream of tables grouped by columns defined as true in the #group annotation row.
  • raw: csv.from()returns a stream of tables with no grouping (all rows are in a single table). All data is formatted as strings.

Examples

If just getting started, use the Flux REPL or the InfluxDB Data Explorer to execute Flux queries.


Query an annotated CSV string

  1. Import the csv package.
  2. Use csv.from() and the csv parameter to specify the annotated CSV string to query.

Query

import "csv"

csvData =
    "
#group,false,false,true,true,true,false,false
#datatype,string,long,string,string,string,long,double
#default,_result,,,,,,
,result,table,dataset,metric,sensorID,timestamp,value
,,0,air-sensors,humidity,TLM0100,1627049400000000000,34.79
,,0,air-sensors,humidity,TLM0100,1627049700000000000,34.65
,,1,air-sensors,humidity,TLM0200,1627049400000000000,35.64
,,1,air-sensors,humidity,TLM0200,1627049700000000000,35.67
,,2,air-sensors,temperature,TLM0100,1627049400000000000,71.84
,,2,air-sensors,temperature,TLM0100,1627049700000000000,71.87
,,3,air-sensors,temperature,TLM0200,1627049400000000000,74.10
,,3,air-sensors,temperature,TLM0200,1627049700000000000,74.17
"

csv.from(csv: csvData)

Results

dataset metric sensorID timestamp value
air-sensors humidity TLM0100 1627049400000000000 34.79
air-sensors humidity TLM0100 1627049700000000000 34.65
dataset metric sensorID timestamp value
air-sensors humidity TLM0200 1627049400000000000 35.64
air-sensors humidity TLM0200 1627049700000000000 35.67
dataset metric sensorID timestamp value
air-sensors temperature TLM0100 1627049400000000000 71.84
air-sensors temperature TLM0100 1627049700000000000 71.87
dataset metric sensorID timestamp value
air-sensors temperature TLM0200 1627049400000000000 74.10
air-sensors temperature TLM0200 1627049700000000000 74.17

Query a raw CSV string

  1. Import the csv package.

  2. Use csv.from() and provide the following parameters:

    • csv: CSV string to query
    • mode: raw

Query

import "csv"

csvData =
    "
dataset,metric,sensorID,timestamp,value
air-sensors,humidity,TLM0100,1627049400000000000,34.79
air-sensors,humidity,TLM0100,1627049700000000000,34.65
air-sensors,humidity,TLM0200,1627049400000000000,35.64
air-sensors,humidity,TLM0200,1627049700000000000,35.67
air-sensors,temperature,TLM0100,1627049400000000000,71.84
air-sensors,temperature,TLM0100,1627049700000000000,71.87
air-sensors,temperature,TLM0200,1627049400000000000,74.10
air-sensors,temperature,TLM0200,1627049700000000000,74.17
"

csv.from(csv: csvData, mode: "raw")

Results

When using the raw CSV parsing mode, all columns values are strings.

dataset metric sensorID timestamp value
air-sensors humidity TLM0100 1627049400000000000 34.79
air-sensors humidity TLM0100 1627049700000000000 34.65
air-sensors humidity TLM0200 1627049400000000000 35.64
air-sensors humidity TLM0200 1627049700000000000 35.67
air-sensors temperature TLM0100 1627049400000000000 71.84
air-sensors temperature TLM0100 1627049700000000000 71.87
air-sensors temperature TLM0200 1627049400000000000 74.10
air-sensors temperature TLM0200 1627049700000000000 74.17

Query CSV data from a file

  1. Import the csv package.
  2. Use csv.from() and the file parameter to query CSV data from a file.

Flux must have access to the file system

To query CSV data from a file, Flux must have access to the filesystem. If Flux does not have access to the file system, the query will return an error similar to:

failed to read file: filesystem service is uninitialized

If using InfluxDB Cloud or InfluxDB OSS, the Flux process does not have access to the filesystem.

Query

import "csv"

csv.from(file: "/path/to/example.csv")

/path/to/example.csv

#group,false,false,true,true,true,false,false
#datatype,string,long,string,string,string,long,double
#default,_result,,,,,,
,result,table,dataset,metric,sensorID,timestamp,value
,,0,air-sensors,humidity,TLM0100,1627049400000000000,34.79
,,0,air-sensors,humidity,TLM0100,1627049700000000000,34.65
,,1,air-sensors,humidity,TLM0200,1627049400000000000,35.64
,,1,air-sensors,humidity,TLM0200,1627049700000000000,35.67
,,2,air-sensors,temperature,TLM0100,1627049400000000000,71.84
,,2,air-sensors,temperature,TLM0100,1627049700000000000,71.87
,,3,air-sensors,temperature,TLM0200,1627049400000000000,74.10
,,3,air-sensors,temperature,TLM0200,1627049700000000000,74.17

Results

dataset metric sensorID timestamp value
air-sensors humidity TLM0100 1627049400000000000 34.79
air-sensors humidity TLM0100 1627049700000000000 34.65
dataset metric sensorID timestamp value
air-sensors humidity TLM0200 1627049400000000000 35.64
air-sensors humidity TLM0200 1627049700000000000 35.67
dataset metric sensorID timestamp value
air-sensors temperature TLM0100 1627049400000000000 71.84
air-sensors temperature TLM0100 1627049700000000000 71.87
dataset metric sensorID timestamp value
air-sensors temperature TLM0200 1627049400000000000 74.10
air-sensors temperature TLM0200 1627049700000000000 74.17

Query CSV data from a URL

  1. Import the experimental/csv package.
  2. Use the experimental csv.from() function and url parameter to specify the URL to query.

The experimental csv.from() function does not support multiple parsing modes and only works with annotated CSV.

import "experimental/csv"

csv.from(url: "https://example.com/example.csv")

To use the parsing modes available in csv.from():

  1. Import the csv and experimental/http packages.
  2. Use http.get() to fetch the CSV data.
  3. Use string() to convert the response body to a string.
  4. Use csv.from() to parse the CSV data and return results.
import "csv"
import "experimental/http"

url = "https://example.com/example.csv"
csvData = string(v: http.get(url: url).body)

csv.from(csv: csvData, mode: "raw")

Was this page helpful?

Thank you for your feedback!


Introducing InfluxDB Clustered

A highly available InfluxDB 3.0 cluster on your own infrastructure.

InfluxDB Clustered is a highly available InfluxDB 3.0 cluster built for high write and query workloads on your own infrastructure.

InfluxDB Clustered is currently in limited availability and is only available to a limited group of InfluxData customers. If interested in being part of the limited access group, please contact the InfluxData Sales team.

Learn more
Contact InfluxData Sales

The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Flux is going into maintenance mode and will not be supported in InfluxDB 3.0. This was a decision based on the broad demand for SQL and the continued growth and adoption of InfluxQL. We are continuing to support Flux for users in 1.x and 2.x so you can continue using it with no changes to your code. If you are interested in transitioning to InfluxDB 3.0 and want to future-proof your code, we suggest using InfluxQL.

For information about the future of Flux, see the following: