15 December, 10:00, «02 Hall. Ararat»
Surprisingly, professionally organized load testing is the area of enterprise projects, banking, and government systems. Professionals and professional teams dedicated specifically to load testing processes work and this process is put on stream.
What is even more interesting is that the niche of load testing is the area of QA specialists, there is one group of people who write and perform such tests, and separately there are engineering teams that do something based on the results of testing. The situation is reminiscent of the distinction between Devs and Ops before 2008.
In commercial web development the situation is different: in most projects, with the exception of very large ones, load testing is carried out "in so far as", most often by the engineers themselves who developed the project. Time for this is allocated according to the residual principle, test scenarios are often worked out “by eye”.
While there are attempts to build load testing into CI/CD, this comes with its own challenges. Business people and management want to have builds and deploys as fast as possible, and adding a service load testing is a huge overhead. Even more, people are not ready, actually, to DDoS production environment every time when someone is doing a deployment. Load testing on such projects happens really rarely, from time to time, on special occasions, and software engineers don’t get the required experience, that they would get doing this regularly.
The results of the load testing in such cases might be completely incorrect, and the problem is not just the wrong numbers. Those wrong results might say that the limit is the sky and business decisions are based on these. It might be a huge marketing campaign that will overflow the servers, it might be a decision to restrain from investing in architecture scaling, or just the decision to release the project when it’s not ready to accept the traffic.
* The project was tested for 5 minutes instead of a long time;
* The test profile was defined incorrectly;
* Staging/Pre-prod environments were tested and production has a completely different infrastructure;
* site returned HTTP 200 when it wasn't actually working;
* some of the microservices were tested in isolation from others and in production they work in connection to each other;
* and a million more reasons.
In my presentation I want to go over the main problems that we see in our work and which lead to incorrect results of load testing or to incorrect interpretation of test results. I will tell you how to avoid them both from a technical point of view and from an organizational one (in a conversation with a business), and how to try to integrate the testing process into the regular process of development, breaking down silos.
The talk was accepted to the conference program