Work with dynamic types

A dynamic type is a wrapper for data whose type is not known until runtime. Dynamic types help when working with data from external sources (like JSON) that support types that do not have an equivalent Flux type.

• Parse JSON into Flux types
• Dynamic type syntax
• Reference values in a dynamic type
• Operate on dynamic types

Parse JSON into Flux types

The primary (but not exclusive) use case for dynamic types is converting JSON data into native Flux types; specifically Flux arrays and records.

Because of strict typing rules in Flux, JSON data doesn’t always gracefully parse into native Flux types. The most common reason is that Flux arrays require all children to be the same type. JSON arrays allow elements to be different types.

With Flux records (the corollary of a JSON object), the properties in the record are part of the record’s type. If you have a JSON array of objects where the objects’ schemas are different, the array cannot be parsed as a Flux array because the types of the records in the array do not match.

["three", 2, true]

[
    {"first-name": "John", "last-name": "Doe", "age": 42},
    {"first-name": "Jane", "last-name": "Doe"},
    {"first-name": "James", "last-name": "Doe", "age": 15}
]

The dynamic type provides a way to resolve the differences between JSON and Flux types.

Dynamic type syntax

Flux does not provide a literal syntax for dynamic types. To cast a value to a dynamic type:

1. Import the experimental/dynamic package.
2. Use dynamic.dynamic() to convert a value to a dynamic type.

import "experimental/dynamic"

dynamic.dynamic(v: "Example string")

// Returns dynamic(Example string)

Reference values in a dynamic type

Use bracket and dot notation to reference values in dynamic records and arrays. Use the exists operator to check if a dynamic type contains a non-null value.

• Reference values in a dynamic record
• Reference values in a dynamic array
• Ensure a dynamic type contains a non-null value

Reference values in a dynamic record

import "experimental/dynamic"

record = {one: 1, two: 2, three: 3}
dynamicRecord = dynamic.dynamic(v: record)

dynamicRecord.one
// Returns dynamic(1)

dynamicRecord["two"]
// Returns dynamic(2)

Reference values in a dynamic array

import "experimental/dynamic"

arr = ["a", "b", "c"]
dynamicArr = dynamic.asArray(v: dynamic.dynamic(v: arr))

dynamicArr[0]
// Returns dynamic(a)

Ensure a dynamic type contains a non-null value

Use the exists operator to check if a dynamic type contains a non-null value. If you attempt to access members of a null dynamic value, Flux returns an error. exists lets you guard against errors caused by attempting to access members in null dynamic values.

import "experimental/dynamic"
import "internal/debug"

dynamicRecord = dynamic.dynamic(v: {one: 1, two: 2, three: 3})
dynamicNull = dynamic.dynamic(v: debug.null())

exists dynamicRecord
// Returns true

exists dynamicNull
// Returns false

dynamicRecord.one
// Returns dynamic(1)

dynamicRecord.four
// Returns dynamic(<null>)

dynamicNull.one
// Error: cannot access property "one" on value of type invalid

Operate on dynamic types

• Convert dynamic types to Flux types

  • Basic types
  • Composite types

• Include dynamic types in a table

• Convert a JSON array to a Flux table

• Encode a dynamic type as JSON

  • Encode a dynamic record as JSON
  • Encode a dynamic array of different basic types as JSON

Convert dynamic types to Flux types

Basic types

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: "string")

string(v: dynamicValue)
// Returns "string"

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: "12")

int(v: dynamicValue)
// Returns 12

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: "12")

uint(v: dynamicValue)
// Returns 12

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: "12.1")

float(v: dynamicValue)
// Returns 12.1

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: "true")

bool(v: dynamicValue)
// Returns true

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: 3000000000)

duration(v: dynamicValue)
// Returns 3s

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: 1640995200000000000)

time(v: dynamicValue)
// Returns 2022-01-01T00:00:00.000000000Z

import "experimental/dynamic"

dynamicValue = dynamic.dynamic(v: "Hello world!")

bytes(v: dynamicValue)
// Returns 0x48656c6c6f20776f726c6421

import "experimental/dynamic"
import "regexp"

dynamicValue = dynamic.dynamic(v: "^[abc][123]{1,}")
stringValue = string(v: dynamicValue)

regexp.compile(v: stringValue)
// Returns /^[abc][123]{1,}/

Composite types

Dynamic composite types will almost always come from working with a byte-encoded JSON string. All the examples below assume this.

["3", 2, true]

import "array"
import "experimental/dynamic"

json = jsonBytes // Byte-encoded JSON above
parsed = dynamic.jsonParse(data: json)

parsed
    |> dynamic.asArray()
    |> array.map(fn: (x) => int(v: x))

// Returns [3, 2, 1]

[
    {"first-name": "John", "last-name": "Doe", "age": 42},
    {"first-name": "Jane", "last-name": "Doe"},
    {"first-name": "James", "last-name": "Doe", "age": 15}
]

import "array"
import "experimental/dynamic"

json = jsonBytes // Byte-encoded JSON above
parsed = dynamic.jsonParse(data: json)

parsed
    |> dynamic.asArray()
    |> array.map(
        fn: (x) => ({
            fname: string(v: x["first-name"]),
            lname: string(v: x["last-name"]),
            age: int(v: x.age)
        }),
    )

// Returns [
//     {age: 42, fname: John, lname: Doe},
//     {age: <null>, fname: Jane, lname: Doe},
//     {age: 15, fname: James, lname: Doe}
// ]

{"first-name": "John", "last-name": "Doe", "age": 42}

import "experimental/dynamic"

json = jsonBytes // Byte-encoded JSON above
parsed = dynamic.jsonParse(data: json)

newRecord = {
    fname: string(v: parsed["first-name"]),
    lname: string(v: parsed["last-name"]),
    age: int(v: parsed.age)
}

// newRecord returns {age: 42, fname: John, lname: Doe}

Check the type of a value inside of a dynamic type

Use dynamic.isType() to check the type of a value inside of a dynamic type.

The following example uses the http/requests package and the Fruityvice API to return information about apples as a JSON object.

{
    "genus": "Malus",
    "name": "Apple",
    "id": 6,
    "family": "Rosaceae",
    "order": "Rosales",
    "nutritions": {
        "carbohydrates": 11.4,
        "protein": 0.3,
        "fat": 0.4,
        "calories": 52,
        "sugar": 10.3
    }
}

import "experimental/dynamic"

response = requests.get(url: "https://www.fruityvice.com/api/fruit/apple")
parsed = dynamic.jsonParse(data: response.body)

dynamic.isType(v: parsed.genus, type: "string")
// Returns true

dynamic.isType(v: parsed.nutritions, type: "array")
// Returns false

dynamic.isType(v: parsed.nutritions, type: "object")
// Returns true

Include dynamic types in a table

Dynamic types are not supported in Flux tables. To include a dynamic value in a Flux table, cast the dynamic type to a Flux basic type.

The following example uses array.from() to build and ad hoc table using dynamic types. Each dynamic type must be cast to a Flux basic type when defining the row record.

import "array"
import "experimental/dynamic"

dynamicString = dynamic.dynamic(v: "one")
dynamicInt = dynamic.dynamic(v: 1)
dynamicFloat = dynamic.dynamic(v: 1.0)
dynamicBool = dynamic.dynamic(v: true)

array.from(
    rows: [
        {
            string: string(v: dynamicString),
            int: int(v: dynamicInt),
            float: float(v: dynamicFloat),
            bool: bool(v: dynamicBool),
        },
    ],
)

Convert a JSON array to a Flux table

1. Import the following packages:

   • array
   • experimental/dynamic

2. Use dynamic.jsonParse() to parse the the JSON response body returned from requests.get() into a Flux dynamic type. Define a variable to capture the parsed JSON. The example below uses the parsed variable.

3. Use dynamic.asArray() to convert the dynamic type returned by dynamic.jsonParse() to a dynamic array. Define a variable to capture the dynamic array. The example below uses the fruit variable.

4. Use array.map() to do the following:

   • Iterate over each dynamic record in the dynamic array and flatten nested records. Flux tables do not support nested records (the corollary of a JSON object).
   • Explicitly cast dynamic values in each dynamic record to a Flux basic type.

The following example uses the http/requests package and the Fruityvice API to return information about various fruit as JSON and then structure the returned data as a Flux table.

import "array"
import "experimental/dynamic"
import "http/requests"

response = requests.get(url: "https://www.fruityvice.com/api/fruit/all")
parsed = dynamic.jsonParse(data: response.body)
fruit = dynamic.asArray(v: parsed)

fruit_flat =
    fruit
        |> array.map(
            fn: (x) =>
                ({
                    name: string(v: x.name),
                    cals: int(v: x.nutritions.calories),
                    carbs: float(v: x.nutritions.carbohydrates),
                    fat: float(v: x.nutritions.fat),
                    protein: float(v: x.nutritions.protein),
                    sugar: float(v: x.nutritions.sugar),
                }),
        )

array.from(rows: fruit_flat)

Encode a dynamic type as JSON

Encode a dynamic record as JSON

1. Import the experimental/dynamic package.
2. Use dynamic.dynamic() to convert the record to a dynamic type.
3. Use dynamic.jsonEncode() to convert the dynamic record into a byte-encoded JSON string.

import "experimental/dynamic"

dynamicRecord = dynamic.dynamic(v: {one: 1, two: 2, three: 3})

dynamic.jsonEncode(v: dynamicRecord)

// Returns the following byte-encoded JSON string:
// {"one":1,"three":3,"two":2}

Encode a dynamic array of different basic types as JSON

Build a dynamic array where each element is a different basic type. This might be necessary if you’re send JSON data to an API that expects an array of values where each value is a different type. Values in a Flux array must be the same type, so you must build a dynamic array of dynamic types:

1. Import the experimental/dynamic package.
2. Use dynamic.dynamic()
   to convert the array to a dynamic type.
3. Use dynamic.dynamic() to convert all values in the array to dynamic types.
4. Use dynamic.jsonEncode() to convert the dynamic array into a byte-encoded JSON string.

import "experimental/dynamic"

arr =
    dynamic.dynamic(
        v: [
            dynamic.dynamic(v: "three"),
            dynamic.dynamic(v: 2),
            dynamic.dynamic(v: true)
        ],
    )

dynamic.jsonEncode(v: arr)

// Returns the following byte-encoded JSON string:
// ["three", 2, true]