performance

I’ve often been called into POCs to remediate poorly performing queries. I remember one POC in particular, on a different platform to Snowflake, where the volume of data and the complexity of the pipeline was such that I could only perform 2-3 tests per day. It would take around one hour to execute the pipeline, but almost three hours to reset all the data for its next execution. The test data set was tens of terabytes.