Documentation

Use pandas to analyze data

Use pandas, the Python data analysis library, to process, analyze, and visualize data stored in an InfluxDB Cloud Dedicated database.

pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

Install prerequisites

The examples in this guide assume using a Python virtual environment and the InfluxDB v3 influxdb3-python Python client library. For more information, see how to get started using Python to query InfluxDB.

Installing influxdb3-python also installs the pyarrow library that provides Python bindings for Apache Arrow.

Install pandas

To use pandas, you need to install and import the pandas library.

In your terminal, use pip to install pandas in your active Python virtual environment:

pip install pandas

Use PyArrow to convert query results to pandas

The following steps use Python, influxdb3-python, and pyarrow to query InfluxDB and stream Arrow data to a pandas DataFrame.

  1. In your editor, copy and paste the following code to a new file–for example, pandas-example.py:

    # pandas-example.py
    
    from influxdb_client_3 import InfluxDBClient3
    import pandas
    
    # Instantiate an InfluxDB client configured for a database
    client = InfluxDBClient3(
      "https://cluster-id.a.influxdb.io",
      database="
    DATABASE_NAME
    "
    ,
    token="
    DATABASE_TOKEN
    "
    )
    # Execute the query to retrieve all record batches in the stream # formatted as a PyArrow Table. table = client.query( '''SELECT * FROM home WHERE time >= now() - INTERVAL '90 days' ORDER BY time''' ) client.close() # Convert the PyArrow Table to a pandas DataFrame. dataframe = table.to_pandas() print(dataframe)
  2. Replace the following configuration values:

    • DATABASE_NAME: the name of the InfluxDB database to query
    • DATABASE_TOKEN: an InfluxDB database token with read permission on the specified database
  3. In your terminal, use the Python interpreter to run the file:

    python pandas-example.py
    

The example calls the following methods:

View example results

Next, use pandas to analyze data.

Use pandas to analyze data

View data information and statistics

The following example shows how to use pandas DataFrame methods to transform and summarize data stored in InfluxDB Cloud Dedicated.

# pandas-example.py

from influxdb_client_3 import InfluxDBClient3
import pandas

# Instantiate an InfluxDB client configured for a database
client = InfluxDBClient3(
  "https://cluster-id.a.influxdb.io",
  database="
DATABASE_NAME
"
,
token="
DATABASE_TOKEN
"
)
# Execute the query to retrieve all record batches in the stream # formatted as a PyArrow Table. table = client.query( '''SELECT * FROM home WHERE time >= now() - INTERVAL '90 days' ORDER BY time''' ) client.close() # Convert the PyArrow Table to a pandas DataFrame. dataframe = table.to_pandas() # Print information about the results DataFrame, # including the index dtype and columns, non-null values, and memory usage. dataframe.info() # Calculate descriptive statistics that summarize the distribution of the results. print(dataframe.describe()) # Extract a DataFrame column. print(dataframe['temp']) # Print the DataFrame in Markdown format. print(dataframe.to_markdown())

Replace the following configuration values:

  • DATABASE_NAME: The name of the InfluxDB database to query.
  • DATABASE_TOKEN: An InfluxDB database token with read permission on the specified database.

Downsample time series

The pandas library provides extensive features for working with time series data.

The pandas.DataFrame.resample() method downsamples and upsamples data to time-based groups–for example:

# pandas-example.py

...

# Use the `time` column to generate a DatetimeIndex for the DataFrame
dataframe = dataframe.set_index('time')

# Print information about the index
print(dataframe.index)

# Downsample data into 1-hour groups based on the DatetimeIndex
resample = dataframe.resample("1H")

# Print a summary that shows the start time and average temp for each group
print(resample['temp'].mean())

View example results

For more detail and examples, see the pandas documentation.


Was this page helpful?

Thank you for your feedback!


The future of Flux

Flux is going into maintenance mode. You can continue using it as you currently are without any changes to your code.

Read more

InfluxDB v3 enhancements and InfluxDB Clustered is now generally available

New capabilities, including faster query performance and management tooling advance the InfluxDB v3 product line. InfluxDB Clustered is now generally available.

InfluxDB v3 performance and features

The InfluxDB v3 product line has seen significant enhancements in query performance and has made new management tooling available. These enhancements include an operational dashboard to monitor the health of your InfluxDB cluster, single sign-on (SSO) support in InfluxDB Cloud Dedicated, and new management APIs for tokens and databases.

Learn about the new v3 enhancements


InfluxDB Clustered general availability

InfluxDB Clustered is now generally available and gives you the power of InfluxDB v3 in your self-managed stack.

Talk to us about InfluxDB Clustered