I’ve often been called into POCs to remediate poorly performing queries. I remember one POC in particular, on a different platform to Snowflake, where the volume of data and the complexity of the pipeline was such that I could only perform 2-3 tests per day. It would take around one hour to execute the pipeline, but almost three hours to reset all the data for its next execution. The test data set was tens of terabytes.

Customers ask for references when choosing a cloud data warehouse platform. Who else is using it, and at what scale? They often overlook a more useful and open alternative - the online technical reference. I find that StackOverflow is a great place to explore the comparative challenges being faced by users of data warehouse platforms, and the depth to which those users are supported by its vendor and community. How many posts per day?

I’ve lived the Microsoft data warehouse story through SQL Server, Parallel Data Warehouse (PDW), Application Platform System (APS), Azure SQL Data Warehouse, and Azure Synapse Analytics. I loved being part of Microsoft, and I’ve been fortunate to work with some great people and some wonderful customers from all over the world. But you have to believe in your product. One morning I woke up and realised that the passion wasn’t there any more.