Excellent as usual! One of the key advantages from Dagster is the fact it is possible to capture metadata from external services (not actually managed via Dagster) and still represent the full picture. For example, using a daily Fivetran sync to ingest some data.
Also, it is possible to implement a fix and retry a "flow" from the point it failed - without needing to restart from scratch.
I would be curious to learn if Kestra supports such functionalities.
Lot of bigger companies have built a config based approach on top of Airflow, which I have seen much better in a sense it can have additional abstraction of custom functionality.
Excellent as usual! One of the key advantages from Dagster is the fact it is possible to capture metadata from external services (not actually managed via Dagster) and still represent the full picture. For example, using a daily Fivetran sync to ingest some data.
Also, it is possible to implement a fix and retry a "flow" from the point it failed - without needing to restart from scratch.
I would be curious to learn if Kestra supports such functionalities.
Great write up!
Interesting overview.
Lot of bigger companies have built a config based approach on top of Airflow, which I have seen much better in a sense it can have additional abstraction of custom functionality.