Aarkam.Gateway Architecture & Protocols

Aarkam.Gateway is the unified API gateway interface of the distributed storage cluster. Built on high-performance ASP.NET Core Kestrel and gRPC, it exposes high-concurrency streaming endpoints for S3-compatible REST clients and the native SDK (Aarkam.Client).

1. Gateway Routing Architecture

2. Core Responsibilities

1. Protocol Translation & S3 Compatibility: Converts incoming S3 REST requests (PUT, GET, DELETE, HEAD) into high-performance gRPC protocol buffer streams (StoreChunk, RetrieveChunk).
2. Stream Demultiplexing: Splits large incoming byte streams into 4KB data pages, coordinating parallel transmission to the physical storage nodes resolved from the hash ring.
3. Zero-Copy Memory Semantics: Uses unmanaged native memory buffers and ReadOnlySpan<byte> pipelines, guaranteeing zero intermediate memory allocations in the managed .NET heap.

3. Protocol Buffer Contracts (StorageService.proto)

The gateway hosts the core protocol buffer RPC contracts:

syntax = "proto3";
package aarkam.storage;

service StorageService {
    // Client-to-Gateway streaming upload RPC
    rpc StoreChunk(stream ChunkWriteRequest) returns (ChunkWriteResponse);
    
    // Gateway-to-Client streaming download RPC
    rpc RetrieveChunk(ChunkReadRequest) returns (stream ChunkReadResponse);
}

message ChunkWriteRequest {
    string bucket = 1;
    string key = 2;
    int64 chunk_index = 3;
    bytes payload = 4;
    string checksum_crc32c = 5;
}

message ChunkWriteResponse {
    bool success = 1;
    string etag = 2;
    int64 bytes_written = 3;
}

4. Port Allocations & Endpoints