GPUMesh Architecture

P2P Connection Lifecycle gpumesh-network · gpumesh-core

Step 1 — Discovery

Peer Discovery

LAN: mDNS broadcast on _gpumesh._udp.local — automatic, zero config.
WAN: HTTP Rendezvous GET /v1/peers/{node_id} — requires configured server.

Consumer

MeshNode (client side)

Relay Server

optional fallback only

Provider

MeshNode (server side)

Step 2 — Transport

dial_with_fallback

Primary: QUIC direct dial — ConnectionMode::Direct
Fallback: QUIC via relay if GPUMESH_RELAY env is set — ConnectionMode::Relay

Step 3 — Identity Handshake

TLS 1.3 + Ed25519 Verification

Client sends Hello → Server replies HelloAck
Payload: node_id · public_key_hex · GPU model · VRAM · Ed25519 signature · timestamp TTL
Server verifies: signature validity · pubkey↔node_id binding · freshness window

Step 4 — Authorization

AllowList Check (provider side)

Three independent permission buckets — all checked before session opens

Job

run containers

Desktop

GUI tunnel

GPU Remote

CUDA remoting

Step 5 — Session (JSON-framed QUIC stream)

Bidirectional Message Exchange

All messages framed as [u32 BE length][UTF-8 JSON], max 64 MiB per frame

Compute

RunJob · FileOffer

Streaming

JobLog · JobStatus

Extended

CudaOp · Desktop

Step 6 — Execution

DockerRuntime + GPU Limits

VRAM check · utilization cap · allowed image list · container hardening · output packaging → outputs.gpk

Group (Cluster) Scheduler Flow gpumesh-storage · scheduler

Group Membership — groups.json

Owner

GroupRole::Owner
creates invite

Member A

GroupRole::Member
shares GPU

Member B

GroupRole::Member
shares GPU

Invite / Join Flow

Signed Invite Token

Owner generates base64-encoded GroupInvite (Ed25519 signed, TTL-limited).
Joiner sends GroupJoinNotify to owner's QUIC endpoint.
Owner verifies: signature · pubkey↔node_id · group_id match → adds to members list.

Scheduler Entry

schedule_peer(group, gpu_memory_mb)

Selects the best available member for a job submission

Stage 1 — Filter

candidate_peers

Load all paired peers from PeerStore.
If group specified: intersect with GroupStore member IDs.

Stage 2 — Probe (8s timeout per peer)

probe_peer → PeerInfoRequest

Opens a QUIC connection to each candidate.
Receives: PeerInfoMsg — GPU model, free VRAM, utilization, sharing status.
Skips peers that are Busy, Offline, or have insufficient free VRAM.

Stage 3 — Scoring

Score = free_vram + idle_bonus + (100 − util) × 10 + prefer_boost

+free VRAM mb base score · +10 000 if Idle · +(100−util)×10 low utilization bonus · +50 000 if matches prefer_peer

ScheduleResult — Best Peer Selected

peer_name · peer_node_id · gpu_model · vram_free_mb · status

Stage 4 — Execution

run_remote_job → selected peer

Package workdir → upload.gpk
Send FileOffer + RunJob request
Stream JobLog back in real-time
On success: pull_artifacts downloads outputs.gpk

Persistence (config dir)

Groups

groups.json

Peers

peers.json

Allow

allowlist.json

Crate Dependency Map full workspace

Binaries — user-facing executables

gpumesh-cli

Main UX — pair, share, run, group, status

gpumesh-agent

Background daemon — QUIC accept loop

gpumesh-control

Local IPC control bridge

gpumesh-relay

Optional WAN relay forwarder

depends on

Orchestration — domain logic

gpumesh-core

MeshNode bootstrap · pairing · share/disable
Inbound session dispatch · group scheduler
CUDA remote · desktop tunnels

What gpumesh-core orchestrates

node.rs
MeshNode · bootstrap · share enable/disable · peer connect

scheduler.rs
schedule_peer · score · probe · candidate filter

remote.rs
run_remote_job · serve_peer_session · file transfer

depends on

Libraries — specialized crates

gpumesh-network

QUIC endpoint (Quinn) · mDNS LAN discovery · relay client · rendezvous HTTP · public registry

gpumesh-protocol

Wire Message enum · ProtocolHello · JsonFrameCodec (length-prefix, 64 MiB max)

gpumesh-security

Ed25519 NodeIdentity · Hello/pairing/invite signing · AllowList (job · desktop · gpu_remote)

gpumesh-storage

PeerStore · GroupStore · GroupInvite · JobRecord — JSON files in config dir

gpumesh-runtime

DockerRuntime · JobRequest · log streaming · output packaging · GPU memory cap

gpumesh-gpu

GpuMonitor · NVML/nvidia-smi detection · VRAM & utilization sampling

gpumesh-common

Shared types (NodeConfig, PeerStatus, JobState) · GpuMeshError · path helpers · constants

optional link

Stub

gpumesh-cudart-stub

No-op CUDA runtime shim — used when real CUDA unavailable at compile time

All crates ultimately depend on gpumesh-common for error types and config structs.
gpumesh-security is a zero-dependency leaf crate (only ed25519-dalek + crypto primitives).

Layer Reference

Layer	Crate(s)	Responsibility	Key types / functions
Binary	cli · agent · control · relay	CLI commands (pair, run, share, group), daemon accept-loop, IPC bridge, optional WAN relay forwarder	`MeshNode`
Core	gpumesh-core	Bootstrap, pairing, share/disable, inbound session dispatch, group scheduler, CUDA remote, desktop tunnels	`MeshNode` · `schedule_peer` · `run_remote_job`
Network	gpumesh-network	QUIC endpoint (Quinn + TLS 1.3), mDNS LAN discovery, relay fallback dial, HTTP rendezvous signalling, public GPU registry	`NetworkEndpoint` · `PeerConnection` · `LanDiscovery` · `RendezvousClient`
Protocol	gpumesh-protocol	Wire message enum covering all session message types, length-prefixed JSON codec	`Message` · `ProtocolHello` · `JsonFrameCodec`
Security	gpumesh-security	Ed25519 identity generation/persistence, Hello/pairing/invite signing & verification, AllowList with three separate permission buckets	`NodeIdentity` · `AllowList` · `PairingPayload` · `GroupInvite`
Storage	gpumesh-storage	JSON-backed peer registry, group membership with roles & invite codes, job log records	`PeerStore` · `GroupStore` · `GroupRole` · `JobRecord`
Runtime	gpumesh-runtime	Docker container execution, GPU memory cap, utilization guard, log streaming, output archive packaging	`DockerRuntime` · `JobRequest` · `ShareLimits`
GPU	gpumesh-gpu	GPU enumeration via NVML or nvidia-smi, VRAM total/free/used, utilization, temperature, CUDA version	`GpuMonitor` · `GpuInfo`
Common	gpumesh-common	Shared types across all crates, error enum, path helpers, protocol constants (version, TTLs, ports)	`NodeConfig` · `GpuMeshError` · `PeerStatus` · `JobState`
Stub	gpumesh-cudart-stub	No-op CUDA runtime shim for builds without a real GPU or CUDA SDK	—