🌵Pipeline Pruner ᴮᴱᵀᴬ

What models do we actually use?

A common occurrence for the beneficiaries of dbt-led data democratisation is that the models pile up quickly, and are neither owned nor deleted. Who is using what, and what is what for remains tribal knowledge. Alvin is the first tool on the market brining clarity to trimming your pipelines using the comprehensive understanding of your usage data & your lineage graph – within and outside of dbt projects!

How does it work?

Pipeline Pruner identifies redundant costs and busywork. This means pipeline sub-branches that are being computed, but have no usage downstream, like tables writing to other tables without any actual analytical or value-driving usage.

You run the report simply by clicking on the big pink button. We're working towards improving the loading time without compromising the accuracy. As of now, depending on the size of your environment, it can take up to a couple of minutes.

Results

The Report will provide you with with a summary of all Tables and Branches we recommend you review and let go of.

You will see a list sorted descending by size measured in node count, as well as estimated potential savings. They're calculated based on the build costs of the table(s) involved.

Branch Details

Clicking on any branch will pull out a summary of all the entities that are connected but don’t have any usage and visualise the hierarchy with the ability to collapse and expand for easier browsing.

You can also look at the lineage in the standard graph format. We will show it automatically only for the smaller branches for which it's easily readable. You can choose to load it for the bigger ones if you find it useful.

Taking action

As next steps we recommend you take on to exploring the entities through search, and double check the impact of depreciation with Impact Analysis.