Hum, the duckdb + lambda benchmark seems to very slow with python. From my (limited) experience using Rust with Duckdb is quite fast and a lot cheaper.
Loved this post. Would be awesome if you did more cost / performance comparisons with ETL tools and data transformation / integration libraries for pipelines ( dlthub, polars, airbyte ) for example.
Great benchmark!
Made me think about recent discussions I had with Kestra users which are using Kubernetes + DuckDB (and a bit of Kestra) to manage such problem.
Not the serverless way indeed, but same idea of using DuckDB as the query engine for low latency requests ✅
Hum, the duckdb + lambda benchmark seems to very slow with python. From my (limited) experience using Rust with Duckdb is quite fast and a lot cheaper.
yes good point, havn't try deploying duckdb with rust.
Loved this post. Would be awesome if you did more cost / performance comparisons with ETL tools and data transformation / integration libraries for pipelines ( dlthub, polars, airbyte ) for example.
Thanks Ibrahim. yes indeed, great idea for a future post :=
Are people using duck db in production?
I have not used it so far, but hesrd great things about it.
Been using MPP like Spark.
seems to : https://www.linkedin.com/feed/update/urn:li:activity:7168545397244608512?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7168545397244608512%2C7168642664278536193%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287168642664278536193%2Curn%3Ali%3Aactivity%3A7168545397244608512%29