The talk was accepted to the conference program
Million RPS on Demand. Highload By Schedule. How works Ozon Performance Testing platform.
Ozon is growing up twice every year since 2019. Main technologies are: Go/ C#, gRPC, Kafka, A lot of code Generation, S2S Routing and security etc.
Performance testing platform was started as service, that helps regular Ozon engineers starts Performance Tests by one cli-util command or one click on UI.
Also, there are 3 main paradigms in Ozon Performance testing:
1) Confidence from Performance tests, can be achieved only on Production.
2) Bandwidth target for next season for concrete service, calculates by analytics, and uploads to Performance Testing platform.
3) Once a week, IT management, looks at consolidated performance reports, provided by our performance testing platform and collect confidence of Readiness Ozon for upcoming Season.
The platform grew with Ozon. It contains several microservices, that helps us and our users to start performance tests easier.
We standardize and simplified most performance Testing activities and provides a lot of integrations with Ozon Infrastructure.
Also we created CPU-effective load generators for http, gRpc and scenario traffic.
All activities, described below accompany by problems, caused by HighLoad nature of our environment.
- Problems with payload collection system and Kafka.
- Load Generators, that generate too much load for our Performance testing system.
- Statistics, that overload our statistics Storage.
- Bandwidth limits in our k8s cluster, that was reached one day.
- CPU limit, also was reached one day.
How we solved all these problems, we run 27 thousand tests a month, as well as how we generate more than a Million RPS on production every night, you will learn at my speech.
The largest professional conference for developers of high-load systems
The price is soaring — the closer the conference is, the more it costs.
The current price of a ticket is — 160000 AMD
Changed your mind?
Tell us why.
Thank you for your reply!
Professional conference for developers of high-load systems