Automatic GPU runtime for indexing

On Linux x86_64, bobbin index detects an NVIDIA GPU through nvidia-smi. For the local ONNX backend, it installs a pinned ONNX Runtime 1.23.2 and CUDA 12 user space runtime in Bobbin’s user data directory when a CUDA runtime is missing. It never installs drivers, changes kernel modules, or runs a package manager. The first download is large; allow several GB of free disk space.

Downloads use HTTPS and the SHA-256 pins in src/gpu_runtime/artifacts.json. Archives are verified before extraction. An installation lock and staging directory prevent parallel commands from using a partial installation. Installed libraries are checked against their receipt on subsequent automatic cache loads. Damaged installations are retained as .damaged-* directories and replaced atomically. Licenses from each wheel are retained beside the libraries.

The process restarts once with the native library search path set, before loading ONNX Runtime. A successfully registered CUDA session logs:

ONNX session using CUDA GPU acceleration

Detection alone is not proof of acceleration. Registration errors fail explicitly; download errors report their cause and select CPU. bobbin status (including --json) reports device, runtime discovery, opt-out and hold state without installing anything. Remote status does not substitute the client’s GPU for the server’s GPU.

Controls

• BOBBIN_GPU=0 forces CPU and suppresses GPU downloads.
• BOBBIN_GPU_CACHE selects the runtime cache root.
• An explicit ORT_DYLIB_PATH pointing at a CUDA runtime remains authoritative. Its CUDA dependencies must already be available to the native loader.
• API embedding backends and remote index commands never provision local CUDA.
• Other operating systems retain their existing runtime path; automatic downloading currently supports Linux x86_64. Other commands can use the installed runtime by supplying its library path and native loader path explicitly.

Shared GPUs

Set BOBBIN_GPU_HOLD_FILE to a file whose presence means GPU work must yield, or set BOBBIN_GPU_HOLD_COMMAND to a JSON argument array. For example:

export BOBBIN_GPU_HOLD_COMMAND='["my-gaming-check", "--status"]'

The command must return zero only when clear. Nonzero, malformed, missing or timed-out probes are treated as held. It is executed directly, without a shell. On a managed Shantytown host (SHANTY_ROOT set), the default probe is st --root <root> fleet hold gaming --status. An explicit command overrides that default.

A hold present at index startup selects CPU without downloading. A hold that appears after a CUDA session starts pauses before the next bounded embedding batch and resumes when clear. File holds are checked before every batch. A successful command probe is cached for at most one second to avoid launching a policy CLI for every fast batch; command holds can therefore take that interval plus the current batch to be observed. The current batch may finish, and the paused session retains its GPU memory. This is cooperative scheduling, not a GPU-memory eviction mechanism.