Joerek van Gaalen
Best practices of performance tests in CI
Best practises on setting up and improving automated performance tests in your continuous integration pipeline.
Automating performance tests is a though job. Across many projects I’ve tackled most challenges and want to share this. What works well, what doesn’t work. How does this compares (mainly differ) to traditional acceptance performance tests. In traditional tests the focus is on setting op realistic tests, in automated tests, it’s all about comparing performance automatically. And how to minimise or optimise your maintenance of the tests. Last but not least, the rest of the organisation should trust and rely on the automated results. If they give to many false positives/negatives, people will start ignoring those results. That’s the worst that could happen. So how to approach this.
And also, how does your job looks like when you’re operating in a devops organisation and have to setup these tests.
Take away: start small, focus on robustness rather than being realistic, set thresholds manually and keep setting smaller bandwidths. Run tests scheduled and after builds, try to increase the frequency.
Joerek van Gaalen is specialist in performance testing since 2005 and with many very large scaled (web)applications. He worked as CTO Performance and has his own performance testing organisation since 2018. In that role as specialist he seeks new techniques and methods related to performance testing in the all changing environment of IT.