How a read-only analytics API went from tens of seconds to low single-digit seconds at p99, and how the storage behind it shrank by roughly three quarters.

A read-heavy analytics API was spending its time moving data, not computing. The fix was architectural, not a tune-up: embed DuckDB inside the Rust service and query local columnar files instead of a remote database.

What changed

  • Parquet snapshots on object storage, pulled to local disk on a schedule — no network round trips on the query path.
  • DuckDB's vectorized engine does aggregations and joins far faster than row-by-row fetching.
  • The storage footprint dropped by roughly three quarters thanks to columnar compression.

What it cost

  • Reads are only as fresh as the last snapshot — fine for analytics, wrong for transactional data.
  • Memory must be budgeted explicitly; an embedded engine shares the process.

The lesson: when reads dominate and data changes in batches, moving the engine to the data beats tuning the network path. Measure p99 before and after — the averages will undersell the win.