System Design Notes: Split Brain in Docker Swarm

Split Brain in distributed systems, such as Docker Swarm, occurs when a network partition causes nodes to lose communication with one another. This results in two or more subsets of nodes thinking they are the leader or primary controller of the cluster. This inconsistency can lead to: Data corruption Conflicting operations Duplicate tasks being executed How it Happens Network Partition: A temporary network failure splits the nodes into two or more isolated groups. Leader Election Conflict: Each isolated group might independently attempt to elect a leader. Independent Decisions: Each group operates as a separate cluster, leading to inconsistent states. In a Docker Swarm cluster: Nodes are classified into managers and workers. Managers coordinate service orchestration and maintain the cluster state. If a partition occurs: Each group of managers may elect its own leader. This results in multiple active leaders (split brain) and service conflicts. Consequences of Split Brain Data Inconsistency: Multiple leaders might make conflicting updates. Duplicate Workloads: Services may be scheduled redundantly. Unrecoverable State: Independent decisions by both partitions can be hard to reconcile. Reduced System Reliability: The system becomes unpredictable or unusable. Prevention Techniques in Docker Swarm Docker Swarm uses the following techniques to avoid split-brain scenarios: ...

July 14, 2025 · 3 min · Nimendra

System Design Notes: Redis Pub/Sub with Go

Redis is often described as a data structure server. In addition to being a key-value store, Redis offers features such as caching (like Memcached), queues, and Pub/Sub messaging. This article focuses on the Pub/Sub (publish-subscribe) pattern in Redis, how it works, its characteristics, and how to use it effectively with Go. What is Redis Pub/Sub? In Redis, a client can publish messages to a named channel, and other clients can subscribe to receive those messages from that channel. ...

July 10, 2025 · 3 min · Nimendra