By Artyom Keydunov, Co-Founder & CEO at Statsbot
Once raw data is loaded into a warehouse, heavy transformations can be performed. It makes sense to have both real-time and background transformations in the BI platform. Users consume and operate on the business definitions level when querying data, and BI is either performing transformation on-the-fly or querying data already transformed in the background.
This approach gives flexibility and agility for development of a transformation layer.
Software engineers nowadays deploy several times a day and praise continuous delivery. The same principle should be adopted for how we approach a transformation. If metric definition changes or some new data is required, one can easily make this changes in hours, not weeks or months. It is especially valuable for fast-growing startups, where changes happen daily and data teams have to be flexible and agile to keep up with product development and business needs.
As data warehouses advance more and more, I’m sure we will see how query time transformations will entirely replace background transformations. Before that happens, we can run some transformations in the background with ELT. Since they are already SQL based and run in the data warehouses, the final switch would be easy and painless.
Statsbot is designed to work with raw data stored in modern data warehouses and run both query time and background transformations. Don’t hesitate to contact us and try it for free.
Bio: Artyom Keydunov is Co-Founder & CEO at Statsbot.
Original. Reposted with permission.