Author: Galyna Pustova, co-founder of Catomize
When preparing for Black Friday, most retailers immediately focus on marketing promotions, discounts, and advertising campaigns. After talking with a few large online stores, our team noticed an interesting pattern: certain aspects of technical preparation are often overlooked. Servers, databases, caching — everything seems to be working, but this is where problems or desynchronization often arise, which can turn any success into a loss of sales and reputation.
The load on servers will grow non-linearly
During peak traffic periods, the load on server resources will grow non-linearly.
E-commerce platforms have centralized data storage places – databases. As the number of concurrent users increases, the number of requests grows not only proportionally, but also combinatorially – the number of cross-requests, table locks, etc. increases. If 1 user creates 5 requests, then 1,000 users simultaneously create not just 5,000 requests, but 5,000 that compete for cache, memory, and other resources. The result is resource blocking and performance degradation in proportion to the load.
This problem is usually addressed with aggressive caching (storing data for quick access). However, caching ceases to be effective with large amounts of traffic. As the number of visitors and unique pages increases, the proportion of cache misses increases: users view different products, filter, sort, and search. This leads to frequent database calls and a nonlinear increase in backend load.
In addition, the use of cache has another more significant drawback. More on that below.
Obviously, the nonlinear growth in iron costs will reduce profits in the same nonlinear way.
This will cause problems, first of all, for the IT department, which will not be able to plan the necessary scaling for the peak traffic period, leading either to overbooking of resources or downtime (work stoppages). Second, it will also be a problem for the finance and marketing departments, because due to the growing cost of servers, they will not be able to plan marketing campaigns, and the projected profits will most likely turn out to be overstated.
Solution: There is no quick fix for this problem. Linear growth in server load can only be achieved by competently building the architecture of an online catalog, where server responses to customer requests are generated in advance and do not create any load on the databases. Giants such as Amazon, eBay, and Jumia operate on this principle.
Products may sell out before the online store knows about it
This is the problem of aggressive caching mentioned in the first point. To reduce the number of database queries, thereby reducing the load on the server, pages are actively stored for reuse. This creates the risk of desynchronization between the catalog and the warehouse. A cached page may show that a product is available when in fact it has already been sold out. The main problem is that the business simply loses sales because customers could have bought a product that is in stock instead. Conversely, products that have arrived in the warehouse may not be displayed in the catalog. Again, this means lost revenue, not to mention reputational risks (I can already see those posts in threads), additional workload for customer support, and so on.
During periods of increased traffic, such as Black Friday, Halloween, or advertising campaigns, the likelihood of this problem occurring is particularly high due to the increased volume of transactions and the higher probability of the product being completely sold out.
Cache is a well-known double-edged sword. It is a lifelong attempt to solve one problem at the expense of another. Phil Carlton [1] spoke very eloquently about the complexity of working with cache back in 1994, but even today, the situation has not changed much:
There are only two hard things in Computer Science: cache invalidation and naming things.
Solution: Here, as with the previous problem, instead of using cache, it is better to use pre-generation of catalog pages. Pre-generation, or projection, is a fundamentally different solution that does not require invalidation. At the same time, a smart system for reporting all changes in the system is needed, which will initiate the regeneration of pages that have lost their relevance.
The risk of accumulating “unsold leftovers”
In the case of a complex retail structure with a large number of sales channels, there is a high probability that the procurement team does not have a realistic picture of the state of the warehouse. To be on the safe side, and to ensure the availability of goods even in the event of the above-described desynchronization, the purchasing department may preemptively increase the number of goods in warehouses on the eve of Black Friday. This will create a large amount of unsold inventory at the end of the peak season. More losses. One of our design partners admitted that due to the lack of up-to-date information on the number of goods in stock, they permanently hold more than €2 million in dead stock just to guarantee the availability of goods.
Solution: The only possible solution here may be a smart system for notifying changes in any part of your e-commerce landscape and a centralized subsystem for storing and instantly delivering (high availability) this information.
Preparing for Black Friday or other major marketing campaigns is not just about marketing and discounts. Real success depends on technical readiness: from server stability to up-to-date inventory data. Ignoring this can lead to lost sales, reduced margins, resource overload, and even damage to your reputation.
To ensure that peak traffic does not come as a surprise, it is worth building a system where technical processes work at the level of business goals: competently scaling servers, ensuring the relevance of the catalog, and controlling purchases. Therefore, preparing for Black Friday requires not only synchronization with the marketing team, but first and foremost with the technical team. Only a comprehensive approach allows you to maintain profits and make Black Friday truly profitable for your business.
