Aarkam - SAT (Storage Substrate) Operations & DR AM CLI • ARK-CLI-0306

Operations, Maintenance & Disaster Recovery Reference

Authoritative command reference for am maintenance (Servicing Windows), am rebalance, am snapshot, am backup, am restore, am cert, and am job.

Last updated: Nov 03, 2026 Classification: Sovereign Tier-4 Enterprise

---
title: "Operations, Maintenance & Disaster Recovery Reference"
description: "Authoritative command reference for am maintenance (Servicing Windows), am rebalance, am snapshot, am backup, am restore, am cert, and am job."
category: "AM CLI"
order: 6
badge: "Operations & DR"
icon: "tool"
documentId: "ARK-CLI-0306"
status: "Production Standard"
classification: "Sovereign Tier-4 Enterprise"
version: "V-1.0.0-GA (Production Stable)"
readingTime: "5 min read"
audience: "Storage Operations Teams, SREs, Backup Admins & SecOps"
scope: "Aarkam Standalone (SAT) & Storage Centers (SCT)"
tier: "Aarkam - SAT (Storage Substrate)"
---

<div class="wiki-spec-card">
  <div class="wiki-spec-grid">
    <div class="wiki-spec-item">
      <span class="wiki-spec-label">Document Identifier</span>
      <span class="wiki-spec-value text-[#10B981]">ARK-CLI-0306</span>
    </div>
    <div class="wiki-spec-item">
      <span class="wiki-spec-label">Standard Status</span>
      <div><span class="status-pill status-normative">Production Standard</span></div>
    </div>
    <div class="wiki-spec-item">
      <span class="wiki-spec-label">Architecture Tier</span>
      <span class="wiki-spec-value text-amber-300">Aarkam - SAT (Storage Substrate)</span>
    </div>
    <div class="wiki-spec-item">
      <span class="wiki-spec-label">Release Version</span>
      <span class="wiki-spec-value text-cyan-300">V-1.0.0-GA (Production Stable)</span>
    </div>
  </div>
</div>

<div class="wiki-alert wiki-alert-note my-4 p-4 rounded-xl border border-emerald-500/20 bg-emerald-500/5 text-xs text-slate-300">
  <strong class="text-emerald-400 block mb-1">Architecture Tier Scope: Aarkam - SAT (Standalone Tier)</strong>
  These commands coordinate physical hardware maintenance windows, dynamic capacity rebalancing across racks, point-in-time volume snapshots, off-cluster backups, mTLS certificate rotations, and universal asynchronous background job execution.
</div>

<p>This reference details production workflows for servicing live hardware without taking the storage cluster offline, capturing point-in-time snapshots, rotating PKI identities, and tracking asynchronous cluster jobs.</p>

<hr />

<h2 id="am-maintenance">1. <code>am maintenance</code> — Safe Node Servicing Windows</h2>
<p>Orchestrates graceful maintenance windows, preventing split-brain or premature shard re-silvering during scheduled hardware repairs:</p>

<div class="wiki-code-block relative group my-3 rounded-xl border border-slate-800 bg-[#060a14] overflow-hidden">
  <div class="code-header flex items-center justify-between px-3 py-1.5 bg-slate-900/90 border-b border-slate-800 text-[10px] font-mono text-slate-400">
    <span class="uppercase font-bold text-emerald-400">bash</span>
    <button type="button" onclick="copyCodeSnippet(this)" class="hover:text-white px-2 py-0.5 rounded bg-slate-800 border border-slate-700 transition cursor-pointer print:hidden">Copy</button>
  </div>
  <pre class="p-3.5 overflow-x-auto text-xs font-mono text-slate-200 leading-relaxed m-0"><code class="language-bash"># Place node into maintenance for 2 hours with audit justification
am maintenance start storage-03 --duration 2h --reason "NVMe Drive Replacement"

# List all active maintenance windows across cluster
am maintenance list

# Complete maintenance and restore node to full active read/write status
am maintenance stop storage-03</code></pre>
</div>

<hr />

<h2 id="am-rebalance">2. <code>am rebalance</code> — Dynamic Capacity Balancing</h2>
<p>Re-distributes Cauchy chunks evenly across storage nodes following hardware expansions:</p>

<div class="wiki-code-block relative group my-3 rounded-xl border border-slate-800 bg-[#060a14] overflow-hidden">
  <div class="code-header flex items-center justify-between px-3 py-1.5 bg-slate-900/90 border-b border-slate-800 text-[10px] font-mono text-slate-400">
    <span class="uppercase font-bold text-emerald-400">bash</span>
    <button type="button" onclick="copyCodeSnippet(this)" class="hover:text-white px-2 py-0.5 rounded bg-slate-800 border border-slate-700 transition cursor-pointer print:hidden">Copy</button>
  </div>
  <pre class="p-3.5 overflow-x-auto text-xs font-mono text-slate-200 leading-relaxed m-0"><code class="language-bash"># Preview planned chunk rebalancing movements without mutating state
am rebalance start --dry-run

# Launch background rebalance throttled at 100 MB/s
am rebalance start --max-rate 100

# Monitor live rebalance migration progress
am rebalance status --watch 3s

# Temporarily pause rebalancing during heavy user ingest
am rebalance pause</code></pre>
</div>

<hr />

<h2 id="am-snapshot-backup-restore">3. Snapshots, Backups &amp; Restoration</h2>

<h3 id="snapshots">3.1 Point-in-Time Snapshots (<code>am snapshot</code>)</h3>
<div class="wiki-code-block relative group my-3 rounded-xl border border-slate-800 bg-[#060a14] overflow-hidden">
  <div class="code-header flex items-center justify-between px-3 py-1.5 bg-slate-900/90 border-b border-slate-800 text-[10px] font-mono text-slate-400">
    <span class="uppercase font-bold text-emerald-400">bash</span>
    <button type="button" onclick="copyCodeSnippet(this)" class="hover:text-white px-2 py-0.5 rounded bg-slate-800 border border-slate-700 transition cursor-pointer print:hidden">Copy</button>
  </div>
  <pre class="p-3.5 overflow-x-auto text-xs font-mono text-slate-200 leading-relaxed m-0"><code class="language-bash"># Create atomic point-in-time snapshot of volume 1
am snapshot create 1 --name "snap-pre-upgrade-2026-q3"

# List snapshots with creation timestamps and referenced sizes
am snapshot list

# Rollback volume to snapshot state in an emergency
am snapshot rollback snap_a7b8c9d0e1f2 --yes</code></pre>
</div>

<h3 id="backups-restoration">3.2 Off-Cluster Cold Backups &amp; Restoration (<code>am backup</code> &amp; <code>am restore</code>)</h3>
<div class="wiki-code-block relative group my-3 rounded-xl border border-slate-800 bg-[#060a14] overflow-hidden">
  <div class="code-header flex items-center justify-between px-3 py-1.5 bg-slate-900/90 border-b border-slate-800 text-[10px] font-mono text-slate-400">
    <span class="uppercase font-bold text-emerald-400">bash</span>
    <button type="button" onclick="copyCodeSnippet(this)" class="hover:text-white px-2 py-0.5 rounded bg-slate-800 border border-slate-700 transition cursor-pointer print:hidden">Copy</button>
  </div>
  <pre class="p-3.5 overflow-x-auto text-xs font-mono text-slate-200 leading-relaxed m-0"><code class="language-bash"># Export encrypted off-cluster backup bundle with Brotli compression
am backup create --target s3://remote-cold-archive/ --compression brotli

# Check status of backup worker
am backup status

# Launch asynchronous volume restoration from backup archive
am restore start bkp_9876543210 --target-volume 1
am restore status --watch 2s</code></pre>
</div>

<hr />

<h2 id="am-cert">4. <code>am cert</code> — PKI Identity &amp; Zero-Downtime Rotation</h2>
<p>Compliant with <strong>NCA ECC-2:2024 §2-2-4</strong> mandatory cryptographic rotation, rotates internal mTLS certificates without dropping active gRPC streams:</p>

<div class="wiki-code-block relative group my-3 rounded-xl border border-slate-800 bg-[#060a14] overflow-hidden">
  <div class="code-header flex items-center justify-between px-3 py-1.5 bg-slate-900/90 border-b border-slate-800 text-[10px] font-mono text-slate-400">
    <span class="uppercase font-bold text-emerald-400">bash</span>
    <button type="button" onclick="copyCodeSnippet(this)" class="hover:text-white px-2 py-0.5 rounded bg-slate-800 border border-slate-700 transition cursor-pointer print:hidden">Copy</button>
  </div>
  <pre class="p-3.5 overflow-x-auto text-xs font-mono text-slate-200 leading-relaxed m-0"><code class="language-bash"># List all active node certificates with expiration dates and SANs
am cert list

# Inspect certificate details for storage-01
am cert inspect storage-01

# Trigger automated zero-downtime mTLS certificate rotation for a node
am cert rotate storage-01 --yes</code></pre>
</div>

<hr />

<h2 id="am-job">5. <code>am job</code> — Universal Asynchronous Job Tracking</h2>
<p>Universal job control plane managing rebalances, disk evacuations, scrubbing, and restorations:</p>

<div class="wiki-code-block relative group my-3 rounded-xl border border-slate-800 bg-[#060a14] overflow-hidden">
  <div class="code-header flex items-center justify-between px-3 py-1.5 bg-slate-900/90 border-b border-slate-800 text-[10px] font-mono text-slate-400">
    <span class="uppercase font-bold text-emerald-400">bash</span>
    <button type="button" onclick="copyCodeSnippet(this)" class="hover:text-white px-2 py-0.5 rounded bg-slate-800 border border-slate-700 transition cursor-pointer print:hidden">Copy</button>
  </div>
  <pre class="p-3.5 overflow-x-auto text-xs font-mono text-slate-200 leading-relaxed m-0"><code class="language-bash"># List running background jobs
am job list --state Running

# Inspect deep execution telemetry for a job
am job inspect job-f956395e506c

# Stream the last 25 lines of execution logs for a job
am job logs job-f956395e506c --tail 25

# Abort a running background job cleanly
am job cancel job-f956395e506c</code></pre>
</div>

Document Identifier ARK-CLI-0306

Standard Status

Production Standard

Architecture Tier Aarkam - SAT (Storage Substrate)

Release Version V-1.0.0-GA (Production Stable)

Architecture Tier Scope: Aarkam - SAT (Standalone Tier) These commands coordinate physical hardware maintenance windows, dynamic capacity rebalancing across racks, point-in-time volume snapshots, off-cluster backups, mTLS certificate rotations, and universal asynchronous background job execution.

This reference details production workflows for servicing live hardware without taking the storage cluster offline, capturing point-in-time snapshots, rotating PKI identities, and tracking asynchronous cluster jobs.

1. `am maintenance` — Safe Node Servicing Windows

Orchestrates graceful maintenance windows, preventing split-brain or premature shard re-silvering during scheduled hardware repairs:

bash

# Place node into maintenance for 2 hours with audit justification
am maintenance start storage-03 --duration 2h --reason "NVMe Drive Replacement"

# List all active maintenance windows across cluster
am maintenance list

# Complete maintenance and restore node to full active read/write status
am maintenance stop storage-03

2. `am rebalance` — Dynamic Capacity Balancing

Re-distributes Cauchy chunks evenly across storage nodes following hardware expansions:

bash

# Preview planned chunk rebalancing movements without mutating state
am rebalance start --dry-run

# Launch background rebalance throttled at 100 MB/s
am rebalance start --max-rate 100

# Monitor live rebalance migration progress
am rebalance status --watch 3s

# Temporarily pause rebalancing during heavy user ingest
am rebalance pause

3. Snapshots, Backups & Restoration

3.1 Point-in-Time Snapshots (`am snapshot`)

bash

# Create atomic point-in-time snapshot of volume 1
am snapshot create 1 --name "snap-pre-upgrade-2026-q3"

# List snapshots with creation timestamps and referenced sizes
am snapshot list

# Rollback volume to snapshot state in an emergency
am snapshot rollback snap_a7b8c9d0e1f2 --yes

3.2 Off-Cluster Cold Backups & Restoration (`am backup` & `am restore`)

bash

# Export encrypted off-cluster backup bundle with Brotli compression
am backup create --target s3://remote-cold-archive/ --compression brotli

# Check status of backup worker
am backup status

# Launch asynchronous volume restoration from backup archive
am restore start bkp_9876543210 --target-volume 1
am restore status --watch 2s

4. `am cert` — PKI Identity & Zero-Downtime Rotation

Compliant with NCA ECC-2:2024 §2-2-4 mandatory cryptographic rotation, rotates internal mTLS certificates without dropping active gRPC streams:

bash

# List all active node certificates with expiration dates and SANs
am cert list

# Inspect certificate details for storage-01
am cert inspect storage-01

# Trigger automated zero-downtime mTLS certificate rotation for a node
am cert rotate storage-01 --yes

5. `am job` — Universal Asynchronous Job Tracking

Universal job control plane managing rebalances, disk evacuations, scrubbing, and restorations:

bash

# List running background jobs
am job list --state Running

# Inspect deep execution telemetry for a job
am job inspect job-f956395e506c

# Stream the last 25 lines of execution logs for a job
am job logs job-f956395e506c --tail 25

# Abort a running background job cleanly
am job cancel job-f956395e506c

← Previous Article

Erasure Coding, Storage Engines & Self-Healing Reference

Authentication, Diagnostics & Incident Triage Reference

Operations, Maintenance & Disaster Recovery Reference

1. am maintenance — Safe Node Servicing Windows

2. am rebalance — Dynamic Capacity Balancing

3. Snapshots, Backups & Restoration

3.1 Point-in-Time Snapshots (am snapshot)

3.2 Off-Cluster Cold Backups & Restoration (am backup & am restore)

4. am cert — PKI Identity & Zero-Downtime Rotation

5. am job — Universal Asynchronous Job Tracking

1. `am maintenance` — Safe Node Servicing Windows

2. `am rebalance` — Dynamic Capacity Balancing

3.1 Point-in-Time Snapshots (`am snapshot`)

3.2 Off-Cluster Cold Backups & Restoration (`am backup` & `am restore`)

4. `am cert` — PKI Identity & Zero-Downtime Rotation

5. `am job` — Universal Asynchronous Job Tracking