Track Talk Th5

The Curse of Tutankhamun

Iris Pinkster - O’Riordain

Bas Dam

11:30-12:15 CEST Thursday 15th June

On November 4th the British archaeologist Howard Carter discovered in a pyramid the nearly unharmed tomb of the Egyptian pharaoh Tutankhamun. A few months after the discovery Lord Carnavaron passed away, an Egyptologist who entered the tomb together with Carter. This sudden death is known as the “Curse of Tutankhamun”.

Within the process of Extract, Transform, Load (ETL) we also often experience some kind of “Curse of Tutankhamun”.

A common way of working within ETL is that after a change an end-to-end test is executed and maybe a user acceptance test is planned. During these tests, production data is used and the result is compared with the current situation. If the outcome is plausible, the change is approved. This doesn’t give us insight though whether the underlying business rules are implemented correctly and whether the outcome is really correct.

The test pyramid is a common tool for testing. Why do we only use the top layers of the test pyramid while testing for ETL and why don’t we “just” start at the bottom?

During this presentation we will explain that also for testing ETL the test pyramid is applicable. We will use an example to support this.

And there are more challenges for testing ETL. For example data quality, the throughput time of processes and creating a controlled data set. These subjects will also be highlighted and possible solutions will be given. These challenges don’t have to hamper a controlled ETL process. But, to overcome these challenges the entire team is needed, including stakeholders outside your team. Only by creating a strong system with solid social interaction one can achieve this as it needs some decisions and updated processes or tools to achieve a more controlled ETL test process.

By following the test pyramid we can prevent the “Curse of Tutankhamun” for testing ETL and strife for a predictable test result.