Excellent. I'm quite new to Iceberg and just starting to experiment with it, but this was one of my first thoughts -- why isn't there a simple one-file option for the catalog to live with the actual data.
Thanks, Julien, for that simple solution to use a lakehouse-ready catalog!
Since it uses PyIceberg under the hood, I guess it could work with PySpark too. Are you planning to add support for Spark/PySpark?
My use case would be to connect Spark to both Unity Catalog, on one hand, and Boring Catalog, on the other hand, if that is possible at all (not sure that Spark allows to connect to several catalogs in parallel).
We have a lakehouse with AWS S3 and Databricks, we are migrating the data lake catalog from AWS Glue to Unity Catalog, and there are still plenty of projects on S3 but not in the official zones of the data lake; Boring Catalog would be an awesome solution to provide those projects with a catalog.
Excellent. I'm quite new to Iceberg and just starting to experiment with it, but this was one of my first thoughts -- why isn't there a simple one-file option for the catalog to live with the actual data.
Awesome! Feel free to share any feedback as you explore—fresh eyes are super valuable for spotting friction.
Definitely! But for me it will get interesting once geospatial support lands in ;-)
I make some test with it and honestly you did a great job!
Nice, super cool to hear!
Wow I have been waiting for something like this local setup. Can’t wait to try to out. Thank you.
Love this! Awesome way to use pyiceberg :)
Thanks, Julien, for that simple solution to use a lakehouse-ready catalog!
Since it uses PyIceberg under the hood, I guess it could work with PySpark too. Are you planning to add support for Spark/PySpark?
My use case would be to connect Spark to both Unity Catalog, on one hand, and Boring Catalog, on the other hand, if that is possible at all (not sure that Spark allows to connect to several catalogs in parallel).
We have a lakehouse with AWS S3 and Databricks, we are migrating the data lake catalog from AWS Glue to Unity Catalog, and there are still plenty of projects on S3 but not in the official zones of the data lake; Boring Catalog would be an awesome solution to provide those projects with a catalog.