Replication
❓ Why Replication?
In today's world, every system needs to keep multiple copies of their data. After partitioning the data, even if every shard in the system can comfortably handle all the traffic, we’d still want to keep multiple redundant copies of each shard. Here are some of the reasons why:
Fault tolerance: Machines go down all the time, and therefore it's always a good idea to keep multiple copies of your data. This way, if one shard goes down, you can have your system serve from another copy of the same shard.
Latency issues: If a system is being used by people all over the world, we might want to have multiple replicas distributed across the globe. This way, users can hit the replica that's geographically closest to them and get a much better experience.
Scaling: Even if all the data fits into one partition, it's possible that the query load is so high that we might need multiple replicas to serve all the queries coming in. In some cases lightweight caching might be able to help but even then, a lot of times one machine might not be able to serve all the load which is why having replicas might be needed.
🧑🤝🧑 Leader-Follower Replication
The process of creating multiple redundant copies of the database is called replication. Even after you have multiple replicas, the core challenge is propagating all the database changes to all the replicas. Usually, the way that is done is through the leader-follower model. In this model, one of the copies of the database is assigned the role of a “leader”, and others are “followers”.
To ensure all replicas have the same data, we make all the write queries happen on the leader first. The leader then propagates these to replicas. This propagation takes place in the form of a replication log or a change stream. Each replica takes this log from the leader and then applies it to its own copy of the database in the same order. This way, all the replicas end up with the same data as the leader.
If the leader were to go down, then we can choose any of the replicas and promote them as the new leader.
With this model, database writes only go to the leader, but reads can go to any replica. This can allow the system to handle a higher read throughput as compared to before.
More complex models of replication exist, but for the most part, their details will not be relevant to you at this point in time.
🔁 Synchronous vs. Asynchronous Replication
The concept and reasoning around replication may seem obvious at first, but there's a lot of thought that needs to go into its implementation. To begin with, replication can be broken into two types: synchronous, and asynchronous.
🔉 Synchronous Replication
As the name suggests, this style of replication is when the client issuing the query receives a success message only after all the replicas have processed that query. This means that if there is 1 leader and 2 follower-replicas of a shard, then every write query will be considered successful only if all the 3 machines have successfully updated their local databases. Replication to all followers happens synchronously (at the same time) and the client is blocked until the replication is fully completed.
🍏 Pros
One of the main advantages of this approach is that data consistency is maintained across the entire system. Therefore, every follower is always in full sync with the leader. If a leader goes down, any follower can be upgraded to the new leader without any loss of data.
Reads from all replicas yield the same value, so no matter which shard the read queries are routed to, they will all return the same value. There will never be a case where different replicas store different data at the same point of time.
🍎 Cons
This approach gives us high data consistency at the cost of write latency. Since the client receives a success message only after the write query has successfully been propagated to all the replicas, writes can become very slow.
Sometimes, a follower might go down or be unavailable for some time. In such cases, it's possible that all write queries are blocked because complete synchronous replication is unable to happen.
🔄 Asynchronous Replication
Again, as the name suggests, this style of replication is when the client writes to a leader and gets a successful response as soon as the leader has processed the write query correctly. The actual replication process of the leader propagating write queries to all the followers happens asynchronously after that. This means that the client need not wait or be blocked for all the replicas to have the updated write. The replicas will get the replication log from the leader sometime after the leader has processed the query, but the client will get a successful response as soon as the leader processes it.
🍏 Pros
Asynchronous replication gives us high availability and low latency at the cost of data consistency. For example, write queries execute faster since they only really require confirmation from the leader that the write query has happened. They do not need to wait for all the followers.
No matter how many followers there are in the system, the client will not face any performance degradation from its point of view. That makes adding replicas a much better and easier experience. Such systems can scale much faster.
🍎 Cons
Since the replication happens asynchronously, it's possible that all the replicas might not reflect the same state of the data at the same point in time. Imagine a scenario where a write query may have been issued to the leader. The client might get a successful response and then issue another query to read the data. If the read request gets routed to a replica before the replica has updated its local DB, then it might return stale information (since the write query hasn't propagated to the replica yet).
Asynchronous replication cannot guarantee “strict consistency”, since there is always a time lag between the leader receiving the update and all the replicas processing this update. If a query is processed by a replica within this time lag, it may return stale information. Asynchronous replication only guarantees that the replica will get the data at some point in the future. This is also known as “eventual consistency”. This means that eventually all the replicas will have the same information, but we just don't know how long it will take for all of them to be updated with the leader.
Another potential disadvantage with this system is that if a leader goes down, then it's possible that not all of its updates have propagated to the replicas. Therefore, when a replica is upgraded to a leader, we might land up with “lost updates” — updates that failed to replicate from the leader to the replicas.
🐢 Replication Lag
The term replication lag is usually used in conjunction with asynchronous replication. Replication lag is defined as the amount of time it takes for the leader to propagate its update to all the replicas. The lower the replication lag, the more “consistent” the system seems, since there will be fewer queries that might receive stale data. The high the lag, the more inconsistent the system will seem because more and more queries might receive stale data due to the large amount of time it takes for all the replicas to be in full sync.
Additional Reading (strongly recommended)
Chapter 5: Replication.
. It is a fairly large chapter so if you do not have time, you can skip the multi-leader and leaderless replication bits because they might be too advance. If you have absolutely no time, just go through the summary of the chapter at the end.