Agree 100% ! That’s why we at Keboola https://www.keboola.com/ actively work on the product addressing most of the challenges you have outlined above. We have created an API-first data platform. You can build and maintain the whole business logic in code, use our user-friendly interface or do both. It is possible to set up infrastructure as code and deploy with CI/CD pipelines.
Hi Petr, great articles! I was drawn here by a link shared by my colleague Victor on LI. Yep, totally agree with you. And as dataops manifesto says, data is different than sw as we have different personas and actually much complex ecosystem. The debug you described on SW DevOps side you can do on Keboola platform very similar way. We have full observability on top of the platform with FinOps and SecOps. We also understand that each persona need different access. So next to UI and APIs we've created whole CLI experience: https://www.keboola.com/product/cli Happy to show you, would love to get your feedback. pavel.dolezal@keboola.com or @pabu01
Article is all too real - definitely agree on the error detection and better lineage visualization aspect.
How do you make sure your stakeholders understand your value in all of this? And product analytics in general? This incident could be boiled down to "Petr found the root cause in a few hours" but ignores all the years of data wilderness exploration and communication skills to get you where you are. It may well be that a less experienced analyst might not have found the problem for a while...
Well written article, the issue that is described here resembles my work day. Lineage dependencies, error detection needs to be improved. End to end data flow visualization will be needed.
Agree 100% ! That’s why we at Keboola https://www.keboola.com/ actively work on the product addressing most of the challenges you have outlined above. We have created an API-first data platform. You can build and maintain the whole business logic in code, use our user-friendly interface or do both. It is possible to set up infrastructure as code and deploy with CI/CD pipelines.
Hi Petr, great articles! I was drawn here by a link shared by my colleague Victor on LI. Yep, totally agree with you. And as dataops manifesto says, data is different than sw as we have different personas and actually much complex ecosystem. The debug you described on SW DevOps side you can do on Keboola platform very similar way. We have full observability on top of the platform with FinOps and SecOps. We also understand that each persona need different access. So next to UI and APIs we've created whole CLI experience: https://www.keboola.com/product/cli Happy to show you, would love to get your feedback. pavel.dolezal@keboola.com or @pabu01
Article is all too real - definitely agree on the error detection and better lineage visualization aspect.
How do you make sure your stakeholders understand your value in all of this? And product analytics in general? This incident could be boiled down to "Petr found the root cause in a few hours" but ignores all the years of data wilderness exploration and communication skills to get you where you are. It may well be that a less experienced analyst might not have found the problem for a while...
Well written article, the issue that is described here resembles my work day. Lineage dependencies, error detection needs to be improved. End to end data flow visualization will be needed.
Glad you liked in Ramdas!