Scaling in Microservices

Background: No production application, especially a microservice, can live without adapting an incoming load. To sustain this situation, we need to scale up our Application.

Takeaways from this Article are:

• What is Scaling?
• Understanding the concept of Scaling in detail
• Type of Scaling and how they are different from each other?
• Preferred way of scaling and how to achieve it.

Scaling:

Firstly, what is scaling in general? This is related to overloading but the idea is very simple to solve the problem, make your system more powerful in a way, it can handle the overloading.

So, I am gonna try to give you a clear idea of how scaling can be considered in different ways.

An understanding of the concept in detail:

In microservices, in order to build a scalable application, we need to design for concurrency and Partitioning: concurrency allows each task to be broken into smaller pieces, while partitioning is essential for allowing the tasks to be processed in parallel.

So, while scalability is related to how we divide and conquer the processing of task, performance is the measure of how efficiently the application processes these tasks.

Resource Bottlenecks: Always first discover and quantify the performance and scalability by identifying the bottlenecks of your system.

Let’s say you have a service A, working as expected (normally) as designed and receiving some load, it can deal with. However, at some point load increases for a moment because of an increase in no. of requests or messages in a queue. The fact is at this moment, is your service is struggling to service as its not capable of handling the situation. We have 2 options to come out of this problem.

Option 1: Make your service big enough to handle.

Option 2: Make copies of the service and distribute the requests amount them.

The Options mentioned are nothing but types of Scaling.

Type of Scaling :

Vertical Scaling: In this kind of Scaling, you add more RAM, CPU, better disk so that the service can consume all requests as its now more powerful than before.

Horizontal Scaling: In this approach, you can add more machines/services to the cluster. In other words, you can make no. of instances of that service.

Difference between both type of scaling :

THE PITFALLS OF VERTICAL SCALING

Vertical scaling isn’t a sustainable or scalable way to architect microservices. It may appear to work out all right in situations where each microservice has dedicated hardware, but it will not work well with the new hardware abstraction and isolation technologies that are used in the tech world today. Always optimize for concurrency and partitioning if you want to build a scalable application.

Now, we understood what is vertical and horizontal scaling but it would be great if you should be able to scale it based on customer and application demands so that We will not pay when actually the load is low and your system is capable of handling it of its own.

So, here again, we can categories the scaling System as:

1. Static: In this category, we need to pay for all the servers as they are fixed in numbers, whether we need at some moment or may not need on the other moment. It is costly.
2. Dynamic: This is the best way, as the system can scale automatically whenever is needed.

An invaluable tool for ensuring the best possible end-user experience is an application delivery controller (ADC), which allows you to detect any possible performance issues. ADCs are inherently useful because they are layers of load balancers with extra features which facilitate the automation of scaling. From both the perspective of the service operator and the end-user, an automated solution is the best approach to take.