This would bring so much ease to devops with tranditional databases like pg. I looked, there doesn't seem to be an equivilant. Maybe it's possible to use sqlmesh but I don't think it would be ideal.
Thank you for this, I was working within a web base environments which could replicate a specific instance for their branch which was great. However I did not know how to implement this with data. Your article was very insightful. However I am wondering how you are supposed to handle data ingestion from different data sources with environments. How would you do it ?
You mention “ When a developer begins a new branch, a zero-copy clone of the database is created, allowing them to experiment freely with actual production data.” I haven’t found a way to do that with DevOps. We use Snowflake and every time we create a new branch, we have to manually clone any database we need. Could you please share how you implement this feature?
I usually do it in GitLab/GitHub with a step triggered by the creation of a new branch. This step runs a Snowflake CLONE command via snowsql to create a new database based on the name of the new branch.
Feel from DM me if want to provide more info on your setup.
This would bring so much ease to devops with tranditional databases like pg. I looked, there doesn't seem to be an equivilant. Maybe it's possible to use sqlmesh but I don't think it would be ideal.
yes indeed by creating addtional load (virtual envs) on the prod db you migh disturb the prod workload, bit dangerous ...
The concept of branching a database like you branch your code is great. I really enjoyed working with this tool for that - https://planetscale.com/
Thank you for this, I was working within a web base environments which could replicate a specific instance for their branch which was great. However I did not know how to implement this with data. Your article was very insightful. However I am wondering how you are supposed to handle data ingestion from different data sources with environments. How would you do it ?
Hi,
You mention “ When a developer begins a new branch, a zero-copy clone of the database is created, allowing them to experiment freely with actual production data.” I haven’t found a way to do that with DevOps. We use Snowflake and every time we create a new branch, we have to manually clone any database we need. Could you please share how you implement this feature?
I usually do it in GitLab/GitHub with a step triggered by the creation of a new branch. This step runs a Snowflake CLONE command via snowsql to create a new database based on the name of the new branch.
Feel from DM me if want to provide more info on your setup.