Querying Multiple Data Sources Without Building an ETL Layer First

The default response to "I need to run analytics across these three data sources" is to build an ETL pipeline. Ingest everything into a central warehouse, normalize the schemas, and then query from there. That sequence makes sense for some situations. For many others, it adds four to six weeks of infrastructure work before you can answer a single analytics question.

This article covers the cases where skipping ETL entirely is the right call, the techniques that make it practical, and the limits you will hit when cross-source federation is not the right answer.

Why the ETL instinct kicks in, and when it is wrong

The ETL instinct exists for good historical reasons. Analytical queries over normalized, co-located data are fast. Join performance is predictable. You can add indexes. You own the schema. Once data is in your warehouse, you control the data lifecycle.

But that reasoning assumes query performance and data lifecycle control are the bottleneck. For a lot of analytics work, especially early in a product or during an exploratory analysis phase, the actual bottleneck is time to first result. You want to know if customer churn correlates with support ticket volume. You have churn data in Snowflake and support tickets in PostgreSQL. The ETL approach says: write the ingestion pipeline, run it, validate the data, then ask the question. That is a week of work minimum, often more.

The federation approach says: join across Snowflake and PostgreSQL directly and get an answer today. It will not be as fast as a co-located query, but for a one-time exploratory question it does not need to be.

Where the ETL instinct goes wrong is treating every multi-source analytics need as if it has production-dashboard latency requirements. Most do not. Exploratory queries, ad-hoc reports, and infrequent cross-system reconciliation all have latency tolerances measured in seconds or minutes, not milliseconds.

The technical options for querying without ETL

Database federation via foreign data wrappers

PostgreSQL's foreign data wrapper (FDW) mechanism lets you register an external data source as a pseudo-table. You write a SQL query that looks like it is joining two local tables; the FDW translates the relevant predicates into the remote source's query language and fetches the required rows.

FDWs are production-quality for certain access patterns. Reading small filtered result sets from a remote PostgreSQL or MySQL instance via postgres_fdw or mysql_fdw works well. The limitations appear when you need to scan large tables remotely or when the remote source does not support predicate pushdown. A query like SELECT * FROM remote_orders WHERE customer_id = 12345 pushes the filter down and is fast. A query that requires the FDW to fetch all rows and join them locally can turn a three-second query into a three-minute one.

The practical scope for FDWs is intra-organization queries where you have reasonable assumptions about the size of filtered result sets. Joining a local customer table to a remote orders table on customer ID is viable. Joining two large unfiltered tables across a WAN link is not.

Query federation layers

Dedicated query federation tools like Trino (formerly PrestoSQL), Apache Arrow Flight, and similar systems take the FDW concept further. They provide a SQL query layer that can route subqueries to multiple backend systems, each with a native connector, and merge the results.

The key capability these systems add over FDWs is cost-based optimization across multiple sources. The query planner can decide which predicates to push down to which source and where to perform joins based on statistics about the remote tables. When the statistics are accurate and the planner makes good decisions, federation query performance approaches co-located performance for filtered queries.

The limitation is operational complexity. Running a Trino cluster for internal analytics is a real infrastructure commitment. It is the right call for an analytics platform team that needs to provide cross-source access to many users. It is often not the right call for a small data engineering team that needs to answer a handful of cross-source questions per week.

Application-layer federation

For many analytics needs, the simplest approach is to run two queries and join them in application code. Query Snowflake for the churn data. Query PostgreSQL for the support tickets. Load both result sets into a DataFrame. Join them in pandas or in a lightweight in-process engine like DuckDB.

This approach gets dismissed as "not a real analytics solution," but for exploratory work it is often the most pragmatic choice. It scales with the size of the result sets, not the size of the source tables. It has no infrastructure dependencies. It is debuggable by anyone who can read Python.

The limitation is that it only works when the result sets are small enough to fit in memory after filtering. A cross-source join of two 100-row aggregation results works fine in pandas. A cross-source join of two 10-million-row tables does not.

What Nava Labs does in this space

The approach we have built into Nava Labs is cross-source SQL execution without requiring the user to think about where each table lives. When you connect a PostgreSQL source and a BigQuery source, you can write a single SQL query that references tables from both, and the query layer handles predicate pushdown to each backend, local join execution where necessary, and result merging.

Consider a team connecting their production PostgreSQL database to their BigQuery analytics warehouse. They want to join real-time transactional data in Postgres against historical aggregates in BigQuery. With Nava Labs, that query runs directly without moving data. The query planner pushes filters down to each backend so we are not transferring entire tables across the wire.

The performance story is honest: a cross-source join over large unfiltered tables will be slower than a co-located query on the same data in a warehouse. We do not hide that tradeoff. What we optimize for is eliminating the setup cost. The first cross-source query should take minutes to get working, not weeks.

When ETL is actually the right answer

We want to be direct about when you should build the ETL pipeline instead of using federation.

If a query will run on a schedule with latency requirements under a second, co-location is the right choice. Query federation adds network latency on every execution, and that compounds with query frequency.

If the same cross-source join is the foundation of a production dashboard that runs every five minutes, the overhead of setting up the pipeline pays for itself quickly. The break-even on ETL infrastructure investment is roughly proportional to query frequency and latency sensitivity.

If the source data volumes require full table scans on both sides of a join, federation will struggle. You need the data co-located and indexed. Federation is for filtered access patterns, not full-table analytics.

And if data governance requires that all production data flow through an audited ingestion layer before it appears in any analytics surface, ETL is not optional. Some organizations have regulatory requirements around data provenance that make federation, even read-only federation, non-compliant.

A decision framework

Before defaulting to ETL, ask three questions. First, how often will this query run and what is the acceptable latency? If the answer is "a few times a week and seconds are fine," federation is worth trying. Second, what is the size of the filtered result sets you will be joining? If you can push meaningful predicates to each source, the join size is manageable. Third, is this exploratory or production? Exploratory queries almost never need the full ETL treatment.

If the answer to any of these pushes toward ETL, build it. The goal is not to avoid ETL permanently. The goal is to not spend four weeks building infrastructure before you know whether the underlying question is worth answering.