The QA Dashboard for ChatAEC needed to be live in a week. The pipeline already ran on Postgres. The question was whether to reach for a warehouse.
We didn't. Here's why — and where the approach breaks.
The setup
postgres ──▶ qa.qa_v1_* views ──▶ postgrest ──▶ caddy ──▶ react
The operational database holds everything the QA pipeline produces: raw LLM outputs, citation checks, confidence scores, hallucination flags. Instead of moving that data into a warehouse, we wrote SQL views that compute the metrics the dashboard needs.
PostgREST auto-generates a REST API from the qa schema. The SQL view is the endpoint. Adding a new dashboard metric means writing a SQL view — no API code, no migration, no deploy.
Caddy handles authentication (Bearer token) at the proxy layer. The React frontend is a thin consumer of the PostgREST endpoints.
What we lose at scale
This approach has three breaking points:
1. Cross-source joins. The Iceberg lakehouse holds historical document data that the QA schema can't see. As long as all QA metrics are computable from Postgres alone, this doesn't matter. The moment we need "hallucination rate by document vintage," we need Trino.
2. Analytical query complexity. SQL views on an operational database compete with the pipeline's write load. A badly-written analytical query can take out the pipeline. We mitigate this with materialized views refreshed on pipeline checkpoints — but it's a real risk at higher data volumes.
3. Historical backfills. Reprocessing six months of QA data to add a new metric means running the new view against the operational table. With a warehouse, that would be a Trino query over Iceberg. On Postgres it's a table scan with consequences.
What we got
- 4 days to v1. No new infrastructure, no new credentials, no new monitoring surface.
- New signal → live in < 1 hour. Write the SQL view, reload PostgREST, deploy the React component. Done.
- Zero new services to operate. Postgres and Caddy were already running.
- The operational database as documentation. The views make the QA schema self-describing in a way that a warehouse mapping wouldn't.
For a one-engineer team with a one-week deadline, those tradeoffs were obvious. The warehouse conversation will happen when we start joining across Iceberg — not before.
The citation drift view
CREATE OR REPLACE VIEW qa.qa_v1_citation_drift AS
SELECT
run_id,
document_id,
COUNT(*) FILTER (WHERE citation_found = false) AS missing_citations,
COUNT(*) FILTER (WHERE citation_text_match < 0.8) AS drifted_citations,
COUNT(*) AS total_citations,
ROUND(
COUNT(*) FILTER (WHERE citation_found = true AND citation_text_match >= 0.8)::numeric
/ NULLIF(COUNT(*), 0) * 100, 1
) AS accuracy_pct
FROM qa.citation_checks
GROUP BY run_id, document_id;
The view is cheap enough to query live (materialized hourly for the aggregate panel, live for the per-run drill-down).