Module

Remote Calls

— From Live Bucket Auto

Live

Running

Module	Function	User	Job	Origin	Worker	Queued	Running	Params

Queued (queue order; admission limits can start later entries first)

#	Module	Function	User	Job	Origin	Waiting

Requests, last 8 h 5 min buckets · click a module to hide it, double-click to show only it

Function

Requests

Request time

Stats by function

Function	Calls	P50	P70	P90	P98	Errors	Err %	Cancelled

Jobs by worker in range · exec = time on the worker, without the queue

Worker	IP	GPUs	Jobs	Share	Exec P50	Exec P90	Exec total	Errors	Err %	Cancelled	Last job

Top exceptions in range · click a row to list those calls

Function	Exception	Count	Last seen	Latest message

Requests in range (most recent first) show entries status exception click a failed row for details

Time	Module	Function	User	Job	Origin	Worker	Total	Queue	Exec	Ser/Des	In	Out	Status	Exception	Params

Machines

— From Live Bucket Auto 60 s

Fleet every box, card and worker that reported in the last 7 days, whatever the range above; missing = no report for 3 min (a card: its box reports it but it answers nothing); a restarted worker is not missing

Status	Box / worker	IP / devices	Cards	Processes	Restarts (7 d)	CPU	RAM	GPU util	Last report

Per engine GPU used = GPU time in cards' worth (1.0 = one card busy all the time); dashed = cards the engine's processes hold

GPU used vs cards held

Processes busy with a job (dashed: ready to take one)

Boxes container CPU and RAM against the container's own limits; stall = % of time some task waited for CPU / memory

Box	IP	CPU cores	CPU %	CPU stall	RAM GB: in use · all / limit	RAM %	Mem stall	Limit hits	OOM kills	Cards	Disk free	Redis	Last report

CPU % of limit

RAM % of limit (peak)

GPU utilisation % (mean of the box's cards)

Containers on the VMs, every Docker container, ours or not (redis, caddy): named by the services running in it; stall = % of time its tasks waited for CPU (contention under CPU weights)

Box	Container	CPU cores	Weight	CPU stall	Mem stall	RAM GB	Limit hits	OOM kills

Cards p95 over the range's minutes; saturated = time in minutes whose p95 reached 90%; shared = time two or more processes used the card at once

Box	Card	Util	Util p95	Saturated	VRAM peak	Power cap	Slowdown	Shared	Processes (util · VRAM)

Processes active = time with GPU work on the card; busy = time with a job running; faded = stopped reporting

Box	Engine	Settings	PID	Cards (util · p95 · active · VRAM)	CPU cores	RAM GB	Ready	Busy	Jobs	Started

Remote Calls

Live

Requests, last 8 h 5 min buckets · click a module to hide it, double-click to show only it

Function

Stats by function

Jobs by worker in range · exec = time on the worker, without the queue

Top exceptions in range · click a row to list those calls

Requests in range (most recent first) show entries status exception click a failed row for details

Gemini Usage

Requests & Cost over time

By Function

By Model

Latest requests show entries

Machines

Fleet every box, card and worker that reported in the last 7 days, whatever the range above; missing = no report for 3 min (a card: its box reports it but it answers nothing); a restarted worker is not missing

Per engine GPU used = GPU time in cards' worth (1.0 = one card busy all the time); dashed = cards the engine's processes hold

Boxes container CPU and RAM against the container's own limits; stall = % of time some task waited for CPU / memory

Containers on the VMs, every Docker container, ours or not (redis, caddy): named by the services running in it; stall = % of time its tasks waited for CPU (contention under CPU weights)

Cards p95 over the range's minutes; saturated = time in minutes whose p95 reached 90%; shared = time two or more processes used the card at once

Processes active = time with GPU work on the card; busy = time with a job running; faded = stopped reporting

Ludo Dashboard

Live

Requests, last 8 h 5 min buckets · click a module to hide it, double-click to show only it show all hide all

Function

Stats by function Clear event list

Jobs by worker in range · exec = time on the worker, without the queue

Top exceptions in range · click a row to list those calls

Requests in range (most recent first) show entries status (all) error cancelled success exception (all) click a failed row for details

Requests & Cost over time

By Function Clear events

By Model

Latest requests show entries

Fleet every box, card and worker that reported in the last 7 days, whatever the range above; missing = no report for 3 min (a card: its box reports it but it answers nothing); a restarted worker is not missing

Per engine GPU used = GPU time in cards' worth (1.0 = one card busy all the time); dashed = cards the engine's processes hold

Boxes container CPU and RAM against the container's own limits; stall = % of time some task waited for CPU / memory

Containers on the VMs, every Docker container, ours or not (redis, caddy): named by the services running in it; stall = % of time its tasks waited for CPU (contention under CPU weights)

Cards p95 over the range's minutes; saturated = time in minutes whose p95 reached 90%; shared = time two or more processes used the card at once

Processes active = time with GPU work on the card; busy = time with a job running; faded = stopped reporting

Requests, last 8 h 5 min buckets · click a module to hide it, double-click to show only it

Stats by function

Requests in range (most recent first) show entries status exception click a failed row for details

By Function