Rokka Raft Consensus & Consistent Hash Ring
Distributed coordination, Raft MON consensus quorum, dynamic consistent hash ring, and node/volume topology.
Rokka (Module 2) is Aarkam’s distributed control plane and coordinator fabric. It is structured as a decentralized, partitioned, and replicated object-to-page storage fabric that strictly decouples the metadata consensus plane from the raw data pipeline, enabling clients to stream 4KB encrypted data chunks directly to Storage Nodes with microsecond coordinator lookups.
1. Raft MON Consensus Quorum
The Rokka.Coordinator layer operates as a distributed Raft consensus group composed of an odd number of peer nodes (typically 3 or 5 nodes):
Quorum Mechanics & Epoch Fencing
- Leader Election: Heartbeats are exchanged every 150ms. If the leader fails to respond within the election timeout (300ms), followers initiate an election.
- Strict Majority Commits: Mutations (adding nodes, registering NVMe volumes, updating failure domains) commit only after majority acknowledgment: \(\text{Quorum} = \lfloor N/2 \rfloor + 1\).
- Monotonic Epoch Fencing: Every topology update increments a cluster epoch counter. Writes stamped with an older epoch are immediately rejected to prevent stale state propagation.
2. Dynamic Consistent Hash Ring
Storage nodes are mapped onto a 32-bit keyspace ring (\(2^{32} - 1\)) using hash values computed from their unique hostname and disk volume ID:
- Virtual Nodes (VNodes): To ensure uniform data distribution and eliminate hot spots, each physical disk volume is assigned \(V = 256\) virtual nodes across the ring.
- Lock-Free Routing: Resolving the target storage nodes for an object key executes in \(\mathcal{O}(\log(N \cdot V))\) time via binary search over a flat, sorted array of active token values.
- Atomic Pointer Swaps: Ring topology updates build a new token array and atomically swap the volatile reference pointer without acquiring locks on read paths.
3. Node vs. Volume Topology: Decoupled I/O Isolation
Production deployments subject to extreme write throughput require physical hardware mapping decoupling:
Single agent on the host monitors host CPU/RAM and queries physical drive S.M.A.R.T. counters via WMI/sysfs.
Every physical NVMe volume hosts an independent Kdouja.Engine instance with its own WAL, MemTable, and 256MB PageStore.
Compaction disk thrashing or flash wear on Volume 1 never blocks, starves, or delays writes destined for Volume 2.
4. Formatted 64-Bit Page Key Space
Because multiple tenant volumes and partitions are co-located across storage hosts, Rokka guarantees absolute key isolation using a high-order bitwise partition formula:
- Bits 48–63 (16 bits): Storage Node / Volume ID (supports up to \(65,536\) unique storage nodes and volumes).
- Bits 0–47 (48 bits): Local Page ID (supports up to \(2.8 \times 10^{14}\) persistent 4KB pages per storage volume).
5. Storage Node Lifecycle States
Storage nodes transition through deterministic lifecycle states managed via the Raft MON coordinator:
| State | Write Eligibility | Read Eligibility | Operational Meaning |
|---|---|---|---|
| Active (Online) | Yes | Yes | Fully operational node receiving writes and serving reads. |
| Maintenance | No | Yes | Temporarily paused for planned maintenance (OS patching, RAM swap). |
| Draining | No | Yes | Actively migrating resident chunks to peer nodes prior to decommission. |
| Quarantined | No | Read-Only | Automatically isolated by Abodi upon predictive failure detection. |
| Evicted | No | No | Permanently removed from the dynamic hash ring. |