Popular scaling
approaches
BITS Pilani Soma Sundaram P
Guest Faculty
Pilani Campus
BITS Pilani
Pilani Campus
Merged - CCZG583/SEUSZG583,
Scalable Services
Lecture No. 3
BITS Pilani
Pilani Campus
Managing high volume transactions
Service Replicas
• An instance of a stateless service is a copy of the
service logic that runs on one of the nodes of the cluster.
• A replica of a stateful service is a copy of the service
logic running on one of the nodes of the cluster.
BITS Pilani, Pilani Campus
Lifecycle of stateful replicas
BITS Pilani, Pilani Campus
InBuild (IB)
• It is a replica that's created or prepared for joining the
replica set.
Types:
• Primary InBuild replicas
• IdleSecondary InBuild replicas
• ActiveSecondary InBuild replicas
BITS Pilani, Pilani Campus
Ready (RD)
• A Ready replica is a replica that's participating in
replication and quorum acknowledgement of operations.
• The ready state is applicable to primary and active
secondary replicas.
BITS Pilani, Pilani Campus
Closing (CL)
A replica enters the closing state in the following scenarios:
• Shutting down the code for the replica
• Removing the replica from the cluster
BITS Pilani, Pilani Campus
Dropped (DD)
• In the dropped state, the instance is no longer running
on the node.
• There is also no state left on the node.
BITS Pilani, Pilani Campus
Down (D)
• In the down state, the replica code is not running, but the
persisted state for that replica exists on that node.
• A down replica is opened by Service Fabric as required,
for example, when the upgrade finishes on the node.
• The replica role is not relevant in the down state.
BITS Pilani, Pilani Campus
Opening (OP)
• A down replica enters the opening state when Service
Fabric needs to bring the replica back up again.
• If the application host or the node for an opening replica
crashes, it transitions to the down state.
• The replica role is not relevant in the opening state.
BITS Pilani, Pilani Campus
StandBy(SB)
• A StandBy replica is a replica of a persisted service that
went down and was then opened.
• After the StandBy Replica Keep Duration expires, the
standby replica is discarded.
• If the application host or the node for a standby replica
crashes, it transitions to the down state.
BITS Pilani, Pilani Campus
Replica role
• Primary (P): There is one primary in the replica set that is responsible for
performing read and write operations.
• ActiveSecondary (S): These are replicas that receive state updates from
the primary, apply them, and then send back acknowledgements. There are
multiple active secondaries in the replica set. The number of these active
secondaries determines the number of faults the service can handle.
• IdleSecondary (I): These replicas are being built by the primary. They are
receiving state from the primary before they can be promoted to active
secondary.
• None (N): These replicas don't have a responsibility in the replica set.
• Unknown (U): This is the initial role of a replica before it receives
any ChangeRole API call from Service Fabric
BITS Pilani, Pilani Campus
Instances of stateless services
BITS Pilani, Pilani Campus
What is load balancing?
• Load balancing is defined as the methodical and efficient
distribution of network or application traffic across
multiple servers in a server farm.
BITS Pilani, Pilani Campus
What are load balancers?
A load balancer may be:
• It can be a physical device.
• It can be an incorporated into application delivery
controllers (ADCs)
BITS Pilani, Pilani Campus
Command Query Responsibility
Segregation (CQRS)
• It is a pattern that separates read and update operations
for a data store.
• Implementing CQRS in your application can maximize its
performance, scalability, and security.
BITS Pilani, Pilani Campus
Need for CQRS
• In traditional architectures, the same data model is used
to query and update a database.
• Data contention can occur when operations are
performed in parallel on the same set of data
• Managing security and permissions can become
complex, because each entity is subject to both read and
write operations
BITS Pilani, Pilani Campus
How CQRS works?
BITS Pilani, Pilani Campus
Benefits of CQRS
• Independent scaling
• Optimized data schemas
• Security
• Separation of concerns
• Simpler queries
BITS Pilani, Pilani Campus
Some challenges of
implementing CQRS
• Complexity
• Messaging
• Eventual consistency
BITS Pilani, Pilani Campus
Protocols for communication
Synchronous protocol
• The client sends a request and waits for a response from
the service.
Asynchronous protocol
• The client code or message sender usually doesn't wait
for a response.
BITS Pilani, Pilani Campus
Asynchronous
Communication
• Services communicating by exchanging messages over
messaging channels.
BITS Pilani, Pilani Campus
Example
BITS Pilani, Pilani Campus
Message Broker
• It is a way of implementing asynchronous
communication
• A message broker is an intermediary through which all
messages flow.
Examples of popular open source message brokers
include
the following:
• ActiveMQ
• RabbitMQ
• Apache Kafka
BITS Pilani, Pilani Campus
Benefits of Message Broker
• Loose coupling
• Message buffering
• Explicit interprocess communication
• Resiliency
BITS Pilani, Pilani Campus
Drawbacks of Message Broker
• Potential performance bottleneck
• Potential single point of failure
• Additional operational complexity
BITS Pilani, Pilani Campus
What is Caching?
• In computing, a cache is a high-speed data storage layer
which stores a subset of data, typically transient in
nature, so that future requests for that data are served
up faster than is possible by accessing the data’s
primary storage location
BITS Pilani, Pilani Campus
Distributed Caches
• A distributed cache may span multiple servers so that it
can grow in size and in transactional capacity.
• It is mainly used to store application data residing in
database and web session data.
BITS Pilani, Pilani Campus
Advantages of Distributed
Cache
When cached data is distributed, the data:
• Is coherent (consistent) across requests to multiple
servers.
• Survives server restarts and app deployments.
• Doesn't use local memory.
BITS Pilani, Pilani Campus
Use Cases for a Distributed
Cache
• Application acceleration
• Storing web session data
• Extreme scaling
• Reducing the impact of interruptions
BITS Pilani, Pilani Campus
Memcached
• Memcached is a distributed memory caching system
used to enhance the performance and scalability of web
applications by reducing the load on databases.
• It stores frequently accessed data in memory, allowing
for faster retrieval compared to traditional storage
methods like disk-based databases.
BITS Pilani, Pilani Campus
Global Caches
• A global cache is a repository for data that you want to
reuse.
• The cache facilitates sharing of data across processes
(both in the same integration node, and across
integration nodes) and eliminates the need for an
alternative solution, such as a database.
BITS Pilani, Pilani Campus
Global cache
BITS Pilani, Pilani Campus
Google Global Cache (GGC)
• It allows ISPs to serve certain Google content from
within their own networks.
• This eases congestion within your network, and reduces
the amount on traffic on your peering and transit links.
GGC features
• Transparent to users
• Reduced external traffic
• Robust
• Easy to set up
BITS Pilani, Pilani Campus
BITS Pilani
Pilani Campus
Scalability features in the Cloud
Horizontal and vertical scaling
• Vertical Scaling is defined as
increasing a single
machine’s capacity with the
rising resources in the same
logical server or unit. .
• Horizontal Scaling is an
approach to enhance the
performance of the server
node by adding new
instances of the server to the
existing servers to distribute
the workload equally
BITS Pilani, Pilani Campus
Comparison
BITS Pilani, Pilani Campus
Auto-scaling
• Scaling monitors your applications and automatically
adjusts capacity to maintain steady, predictable
performance at the lowest possible cost.
• AWS Auto Scaling makes scaling simple with
recommendations that allow you to optimize
performance, costs, or balance between them
BITS Pilani, Pilani Campus
Benefits of Cloud Scaling
• Fast and Easy
• Cost efficiency
• Optimized performance
• Capacity
BITS Pilani, Pilani Campus
Load balancing
BITS Pilani, Pilani Campus
Functions of Load Balancer
BITS Pilani, Pilani Campus
Benefits of Load Balancing
BITS Pilani, Pilani Campus
What is virtualization
BITS Pilani, Pilani Campus
Benefits of Virtualization
BITS Pilani, Pilani Campus
Types of Virtualization
BITS Pilani, Pilani Campus
Serverless architecture
BITS Pilani, Pilani Campus
Models under Serverless
BITS Pilani, Pilani Campus
When to use Serverless
Computing
BITS Pilani, Pilani Campus
Advantages of Serverless
Computing
BITS Pilani, Pilani Campus
Limitations of Serverless
Computing
BITS Pilani, Pilani Campus
Best Practices for achieving
Scalability
BITS Pilani, Pilani Campus
References
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-concepts-
replica-lifecycle
https://community.citrix.com/tech-zone/design/reference-architectures/adc-gslb/
https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs
https://medium.com/design-microservices-architecture-with-patterns/cqrs-
design-pattern-in-microservices-architectures-5d41e359768c
https://learn.microsoft.com/en-us/dotnet/architecture/microservices/architect-
microservice-container-applications/communication-in-microservice-
architecture
https://memcached.org/
https://www.ibm.com/docs/en/integration-bus/10.0?topic=caching-data-
overview
https://aws.amazon.com/
https://docs.nginx.com/nginx/admin-guide/load-balancer/http-load-balancer/
https://www.ibm.com/topics/virtualization
BITS Pilani, Pilani Campus
