Cluster · nyx-atlas · 7 nodes / 23 GPUs
live
1 degraded
1 offline
Fleet Pulseis nominal
6,893tokens / sec aggregate
MTP speculative decode · active
vLLM 0.6.3
CUDA 12.4
Autoscaler armed
63%cluster load
79%vram
6.7kWpower
Total servers+1 recovered
7/7
5 online
Active GPUsstable
19/23
avg util 74%
Running models+1 warmed
12deploys
MTP 3×
Queued jobsdraining
142depth
avg wait 38 s
Tokens / second+4.1%
6,893
spec. accepted 71%
Power usage+0.2 kW
6.71kW
grid cap 7.2 kW
Throughput
Tokens per second · 90 s window
live · 1 s poll
Power
Draw vs grid cap
peak 7.6 kW
Queue
Job completion time by model
Memory
VRAM occupancy per node
385 / 488 GB
Hot models
Deployment leaderboard
Nodes
Server strip
activity · fleetd
Signals
Top alerts
Section 02 · Nodes
Physical Servers
Rack thermogram
GPU utilization matrix
cell fill = util · hue = temp
Inventory
Server / model table
7 rows
Detail
Expanded node cards
click any card to expandSection 03 · Accelerators
GPU Monitor
Thermals
Core temperature · throttle line 88 °C
Allocation
VRAM per accelerator
Section 04 · Deployments
Model Deployments
sort
Throughput race
Tokens per second by deployment
MTP gains · Qwopus 3.6
Registry
Deployment table
13 models| Model | Quant | Context | Host | GPU set | Status | Jobs | VRAM | Tok/s |
|---|
Section 05 · Inference queue
Inference Queue
Queue depth
Active / waiting / done / failed
Latency
Time-to-first-token · ms
streaming0
queued0
completed0
failed0
scheduler · ray-gpu
Section 06 · Signals
Alert Center
Escalation
Runbook bindings
| Signal | Owner | Auto-action | Runbook | MTTA |
|---|---|---|---|---|
| GPU thermal ≥ 88 °C | infra-gpu | cap power → 420 W | rb-cooling-v3 | 00:41 |
| Volume free < 15% | storage | prune quant cache | rb-cache-prune | 02:10 |
| Job OOM / crash | serving | reslice ctx, retry ×2 | rb-oom-triage | 00:18 |
| tok/s below floor | perf | re-enable MTP draft | rb-slow-gen | 01:26 |
| Queue depth > 3× baseline | capacity | spin warm replica | rb-elastic | 00:52 |