Cluster & Node Management Commands

This section provides operational syntax and real-world examples for administering cluster health, Raft quorum consensus, and physical storage node lifecycles.

1. am cluster — Fabric Control & Doctor

The am cluster command suite monitors holistic storage fabric health, capacity utilization, and system-wide invariant checks:

# Display live cluster summary, quorum status, and capacity
am cluster status

# Watch live cluster health during maintenance with 2s refresh
am cluster status --watch 2s

# Run exhaustive cluster diagnostic health check (Doctor)
am cluster doctor

# Set cluster-wide write freeze (emergency read-only mode)
am cluster set-mode --read-only --reason "Datacenter Power Maintenance"

2. am mon — Raft Consensus & Election Control

The am mon suite inspects the Raft consensus group governing the metadata control plane:

# Display active Raft leader, term, and committed log index
am mon status

# List all peer coordinator nodes, ping latencies, and sync offsets
am mon peers

# Step down current leader and trigger a safe re-election
am mon step-down --timeout 5s

# Force removal of a permanently destroyed coordinator node
am mon remove-peer coord-03 --force

3. am node — Storage Fleet Lifecycle & Maintenance

The am node suite manages registration, capacity draining, and decommissioning of physical storage hosts:

# List all registered storage nodes, IP addresses, and disk usage
am node list --format table

# Inspect deep hardware telemetry and mounted volume paths for a node
am node inspect storage-node-01

# Add a new storage node to the active hash ring
am node add storage-node-04 \
    --address "https://192.168.1.24:57775" \
    --failure-domain "Rack-B2" \
    --capacity 20TB

# Gracefully drain a storage node before hardware replacement
# Automatically migrates chunks to surviving nodes with throttle
am node drain storage-node-02 --max-rate 500MB/s --yes

# Permanently evict an evacuated storage node from the cluster
am node evict storage-node-02 --yes