System Architecture & Data Flow

Aarkam is architected from six purpose-built, loosely coupled modules, each owning a strictly isolated layer of the storage pipeline:

Module	Architectural Role	Deployment Boundary
Aarkam.IO	Edge Gateway, SigV4 S3 REST Endpoint, RBAC, DMZ Rate Limiting	Client-facing edge (:57776)
Rokka	Distributed Control Plane (Raft MON) + Data Plane Daemon	Central Coordinator (:57777) & Storage Nodes (:57778)
Kdouja	Embedded LSM-Tree Key-Value Engine with Direct NVMe I/O	In-process within every Storage Node
Ahmodi	SIMD Galois Field GF(2⁸) Cauchy Reed-Solomon Math Engine	Embedded within data path and repair workers
Abodi	Out-of-band SMART Telemetry & Predictive Failure Sidecar	Background daemon on every physical host
Aarkam.AM (CLI)	Unified Cluster Administration, Automation & Operator Tool	Single-binary executable (`am.exe`)

The End-to-End Write Path

When an enterprise client initiates a file upload, data passes through five zero-allocation stages:

sequenceDiagram
    autonumber
    actor Client as Client App / S3 SDK
    participant Gateway as Aarkam.IO Gateway (:57776)
    participant Coord as Rokka Coordinator (:57777)
    participant Ahmodi as Ahmodi Math Engine (AVX-512)
    participant Node as Rokka StorageNode (:57778)
    participant Kdouja as Kdouja LSM Engine (NVMe)

    Client->>Gateway: 1. PUT Object (AWS SigV4 auth)
    Gateway->>Coord: 2. Query Consistent Hash Ring topology
    Coord-->>Gateway: 3. Return active node addresses and epoch
    Gateway->>Gateway: 4. Client-side Envelope Encryption (AES-256-GCM)
    Gateway->>Ahmodi: 5. Compute Cauchy RS (8+3) Parity Chunks
    par Stream Chunks in Parallel
        Gateway->>Node: 6. Stream 4KB data & parity chunks via gRPC mTLS
        Node->>Kdouja: 7. Append chunk to Write-Ahead Log (WAL)
        Kdouja->>Kdouja: 8. Insert into Arena-allocated SkipList MemTable
        Kdouja-->>Node: 9. Durability Acknowledged
        Node-->>Gateway: 10. Chunk Ack
    end
    Gateway-->>Client: 11. 200 OK (Upload Durably Committed)

Important

Client-Side Envelope Encryption: All encryption occurs at the gateway boundary before chunks traverse internal storage networks. Storage nodes and the Kdouja engine only ever store encrypted ciphertext blocks. Even physical possession of raw drives yields zero plaintext without customer HSM keys.

The Autonomous Self-Healing Loop

Unlike legacy storage systems where drive failure triggers massive emergency rebuild storms that degrade production I/O, Aarkam uses proactive eviction:

Telemetry Ingestion: Abodi.Agent collects continuous S.M.A.R.T. raw attributes, IOPS latency histograms, and flash wear counters directly from the host OS kernel.
Stress Correlation: Abodi correlates weak signals (e.g., rising sector reallocation paired with tail-latency spikes).
Failure Window Forecasting: A predictive model projects the drive's failure window (e.g. 72 hours remaining).
Topology Eviction: Rokka.Coordinator fences the degrading drive from new writes while it is still capable of serving reads.
Background Hedged Migration: Replica and parity blocks are rebuilt onto healthy nodes gracefully at throttled I/O levels without ever incurring an unpredicted catastrophic crash.