๐Ÿ’ผWorkloads

Fine-tune your workloads to optimize costs and efficiency of your environment.

Understand how your environment changes over time, what affects your overall spend and spot anomalies related to your Warehouse, orchestration and analytical usage.

Workloads is the newest addition to Alvin's metadata analysis toolkit. It allows you to segment queries running within your data warehouse into distinct groups and zoom into each group's metrics with a specialized lens.

They are grouped by analytical queries from your connected BI environment, dbt models materializing over time, users and service accounts. We also separate Query Fingerprints โ€“ itโ€™s queries without the literals (stream, numbers, parameters) or limits, to get to the base structure of queries and changes that happen to those patterns over time. It allows you to understand what drives your Warehouse costs at an aggregated level.

Query Fingerprints

To understand what drives your Warehouse costs at an aggregated level, we've introduced Query Fingerprints โ€“ workload here is understood as the truncated query.

In simple terms, it's queries without the literals (strings, numbers, parameters) or limits. The goal is to get to the base structure of queries and changes that happen to those patterns over time.

In this Details tab, there will always be only one unique query version, cost and usage overview, and the SQL. On the bottom right, wherever applicable, you will see the available insights about the query performance (like slot contention for BigQuery or high Warehouse load for Snowflake).

Users

Here you can group costs over time by association with different users and service accounts.

You can add and remove columns in this view to your liking.

Contrary to the Fingerprints, in the Users tab you will see multiple queries associated with one user.

The detailed query view will change depending on the query type.

Dashboards

Here you can easily see which of your dashboards are driving the most cost.

After opening the detailed view, you will be able to see all the query versions issued by this dashboard. You can inspect each query version to peek at the cost chart, the SQL as well as available optimization insights.

Dashboard Elements

In case of multiple dashboard elements you can group the queries for specific charts to understand the granular cost drivers.

Dbt Models

This integrations works twofold: directly from your Data Warehouse and through the dbt integration.

For any given model, you will automatically see all past query versions and cost differences happening alongside the changes.

If you integrate your dbt environment you will additionally be able to see the file paths of the model and other additional context, however the core funcionality itself is not reliant on it.

Last updated