Causal intelligence for BigQuery, dbt, and Iceberg REST
Every tool sees symptoms.NaviLake finds the cause — and the fix.
BigQuery knows the bill. dbt knows the model. An Iceberg REST catalog knows snapshots and properties. Neither explains the other. NaviLake sits on top — traces a finding back to the table, the job, and the model — and tells you what to change. Live today on BigQuery and dbt. Iceberg REST is in-product — nine health rules, an estate view, and grant-gated reversible property writes (default off), proven on our synthetic tenant. Spark is next. We do not host your catalog.
No migration. No agent required. We never read table rows.
Illustrative chain — BigQuery + dbt
2 systems correlated · 3 events in chain
Confidence84%
dbtBigQuery2 layers correlated
dbtRoot cause
fct_order_events — full refresh, no partition filter
BigQueryPruning never runs
Table is partitioned by event_date — the model never filters on it
BigQueryWhat you feel
Every nightly run scans the full history
Pattern
Full-table scan each run
Table
1 partitioned table
Job
Nightly dbt job
First fix
Add a partition filter
Example only — a dbt model with no partition filter, and the BigQuery full-scan that follows. Not a live finding and not a dollar claim.
Each connector you add changes what we can see.
A single connector shows you one layer. Two connectors unlock cross-system causality. BigQuery, dbt, and Iceberg REST are in the product today — and each connector you add compounds what NaviLake can trace back to its origin.
Not just tables — the jobs that write them, the models that schedule them, and the queries that read them. Every finding says what happened, what the evidence is, and what to change. Dollars only when we can attribute them.
A model·Cost
The model looked fine. Every run scanned the whole table.
A dbt model refreshed nightly without a partition filter on a date-partitioned BigQuery table. Each run succeeded — but scanned the full history while downstream dashboards only needed the last 30 days.
Full-table scan on every nightly rundbtBigQuerylive
A table·Cost
The table is still billed. Nobody has queried it in 90 days.
Writes stopped, then reads stopped. BigQuery still charges for the bytes. No dbt model points at it. The finding is the table, the last job, and a copy-paste drop — not a storage-reclaim percentage we cannot prove.
Unused table — last read 90+ days agoBigQuerydbtlive
A pipeline·Trust
Nothing failed. Everything was wrong.
A model stopped writing to its table. No job errored, no test failed, every status stayed green — while three executive dashboards kept serving numbers that had frozen eleven days earlier.
3 dashboards serving stale datadbtBigQuerylive
The whole estate·Control
Nobody could say who owned a third of the data.
Not one table — hundreds. No model, no owner, no recent reads. Nothing was looking across the estate at once, so nothing fired.
Unused and unowned tables, one inboxBigQuerydbtlive
dbt sees your models. BigQuery sees your bills. Neither sees the full picture on its own. Not one of them can tell you a model's refresh pattern is causing full-table scans — because that fact lives in the relationship between transformation and warehouse.
Every tool in your stack only sees its own layer.
Your warehouse tool knows what's expensive. Your catalog knows what exists. None of them talk to each other — so the engineer becomes the integration layer, manually stitching root causes together over and over.
× Single-layer tools
See cost, but not why
See models, but not the warehouse scan they cause
See lineage, but not whether anything still reads it
2–4 hours of manual stitching per incident
NaviLake
Sees cost and the upstream model
Ties a full-scan job back to the dbt refresh that caused it
Cross-references lineage with real query history
Correlated and ranked automatically
Live today
Correlated together — findings from one layer enrich the other.
BigQuery
Usage, cost, query history, and query fingerprints — full-scan and cache analysis included.
dbt
Model-to-table mapping with confidence scoring, lineage, and run cost attribution.
Iceberg REST
Catalog health from REST metadata — nine ICE rules, estate view, BQ↔Iceberg joins. Grant-gated reversible property writes (default off). Compact / expire / orphan stay recommend-only. Orphan detection needs a storage LIST. Proven on our synthetic tenant, not claimed on every customer catalog. Not a migration product. Spark is not in this product yet.
Coming next
Each new connector compounds the intelligence of every connector before it.
Spark
Job-to-table attribution and compute tuning. Compact / expire / orphan execution. Not in the product today.
The insight isn't in any one layer. It's in between them.
dbt doesn't know which queries downstream will run. BigQuery sees a successful job. Neither connects the model's refresh pattern to the scan cost.
NaviLake correlates across the boundaries to catch what slips between the cracks.
Illustrative example — BigQuery + dbt
critical
warehouse.sales.order_events
Daily dbt refresh triggers full-table scans — partition pruning never kicks in
warehouse×transformation2 layers correlated
Transformation layer
dbt model order_events does a full refresh daily with no partition filter
Warehouse layer
BigQuery table partitioned by event_date — but the model never filters on it
Query pattern
Downstream BI queries filter by date range — they inherit the full scan cost from the upstream model
Root Cause
dbt model refreshes the entire table nightly without an event_date partition filter. BigQuery can't prune partitions, so every run scans the full history.
Access pattern:Downstream jobs filter by date; the table is unpartitioned
Symptom:Every query scans the full table
First fix:Partition on the filter column — stay in BigQuery
Format-move advice is on the roadmap — a labelled heuristic (stay / look closer), not a migration product and not a dollar promise.
Confidence
81%
Fix in place first. Move only when the evidence says so.
NaviLake does not migrate your tables. Live recommendations are copy-paste fixes on BigQuery and dbt — partition filters, unused tables, model-to-table cost. Iceberg REST adds catalog health and, if an administrator grants it, reversible table-property writes. Table-format "stay or look closer" advice comes later, labelled as a heuristic with the unknowns listed.
Your team already knows the data platform is slow and expensive. Now you can show the evidence — table, job, and model — without stitching tools by hand.
First scan
Surfaces unused tables, orphaned pipelines, and storage patterns as soon as BigQuery, dbt, or Iceberg REST is connected.
2–4 hours
Time saved per incident — root cause surfaced instantly instead of manually stitched together across tools.
First sync
Time from a connected BigQuery project to a first finding — after jobs are in, not a public self-serve timer.
Every finding that carries a dollar has a confidence score and the evidence behind it. Findings without a defensible number show the pattern and omit the $. You and your team look at the same record.
This isn't a tool that adds to your stack of dashboards nobody checks. It's the thing that explains why the other dashboards say what they say.
Recommend only
NaviLake never runs DDL or DML in your project. Fixes are copy-paste for your team. Read-only IAM on BigQuery: metadata and jobs, never table rows.
→ 34 tables unused for 90+ days → 12 dbt models with no downstream readers → 3 tables flagged for missing partition filters
Prioritized by impact — with evidence for each."
*Illustrative example based on typical deployment
Connect BigQuery. Add dbt. Iceberg REST is a connect step too.
1
Connect BigQuery
A reader service account — metadataViewer, jobUser, resourceViewer. Never dataViewer. Never table rows.
2
Add dbt
Point us at your manifest (file or GCS). That unlocks model-to-table mapping and run attribution.
3
Read the causal chain
What happened, which job and model, what to change. Iceberg REST is a connect step in-product today — health rules plus optional grant-gated property writes. Spark is still next.
Metadata-only by default. Your team approves every write.
We never read table rows. Warehouse and compact/expire/orphan work stay recommend-only until you run them. The one catalog write we offer is a grant-gated, reversible Iceberg property change — default off.
Metadata-only
We read schema, statistics, and job history — never your actual data files. No write access to BigQuery. Iceberg property writes need an explicit org grant.
Copy-paste fixes
Most recommendations are text and SQL for your team to run. NaviLake never executes against BigQuery. Compact / expire / orphan stay recommend-only; you run them on your compute.
Evidence frozen on the finding
Each recommendation keeps the evidence that produced it. An approved Iceberg property change stores the inverse and can be reverted. Warehouse rollback stays your team's change process.
Least-privilege BigQuery
metadataViewer + jobUser + resourceViewer. We never ask for dataViewer — that role reads table rows. Iceberg compact / expire / orphan stays recommend-only until execution on customer compute. Property writes are a separate, default-off grant.
Not a cost dashboard. Not a catalog.
We didn't start by picking tools. We started by picking a problem.
"Every data team we've ever been part of had the same failure mode: five excellent tools, each confident about its own layer, none of them agreeing on the full picture. We didn't want to build another excellent single-layer tool. We wanted to build the layer that sits above all of them — one that gets more valuable every time you connect something new to it, not less.
BigQuery and dbt are where we started because that's where the pain is sharpest today. Iceberg REST intelligence is in the product now — health rules, an estate view, and grant-gated reversible property writes, proven on our synthetic tenant. Spark is next, and it won't be where we stop."
Design-partner access. No public price list yet.
BigQuery + dbt is live today. Iceberg REST is in-product — health rules and an estate view, proven on our synthetic tenant, not claimed on every customer catalog. We will tell you what a paid seat looks like after you have seen findings on your own jobs — not before. There is no 50-table free SKU, no Growth/Scale grid, and no Iceberg health plan on a price card we do not sell.