GPU Acceleration

By the end of this page you will know when ScramDB dispatches work to a GPU, every reason it silently falls back to CPU instead, and how to confirm which path a query actually took.

GPU acceleration is a large-batch lever, not a default-on win for every query. Small workloads should expect CPU execution and that is the correct outcome, not a misconfiguration.

Requirements

GPU dispatch requires all of the following to be true:

1. The server binary was built with GPU support. A standard release build includes GPU support for NVIDIA, AMD, and Apple Metal by default.
2. [gpu] enabled is true (the default).
3. A supported GPU (NVIDIA, AMD, or Apple Silicon) with a working driver is present at startup.

[gpu] vendor accepts "auto" (default), "nvidia", "amd", or "apple". "auto" detects whichever supported device is present; set it explicitly to pin dispatch to one vendor on a machine with more than one GPU installed.

Enabling GPU

[gpu]
enabled = true

SCRAMDB_LOG=info scramdb -c scramdb-config.toml --pg-address 127.0.0.1:5432 --pg-no-auth

Expected: a startup log line containing GPU: and the detected device name, if a supported GPU (NVIDIA, AMD, or Apple Silicon) with a working driver is present. If GPU is disabled in config, you will instead see GPU: disabled by configuration. If no supported device is visible, you will see GPU: no devices found. Either fallback case is not an error, the server still starts and serves every query on CPU.

If it fails: confirm the binary was built with GPU support, and confirm your GPU driver sees the device from the same host or container the server runs in (for example, nvidia-smi on NVIDIA).

Why a query falls back to CPU

Fallback is always clean, never a crash. It happens at two different levels: server startup (GPU never initializes at all) and per-query (GPU is available but this specific query declines it).

Startup-level fallback

Condition	Log line (level)	Effect
[gpu] enabled = false	GPU: disabled by configuration (info)	GPU state never initializes; every query runs CPU-only.
Binary built without GPU support	no GPU log line at all	Same effect, no GPU code is compiled in.
No supported device found	GPU: no devices found (info)	Falls back to CPU.
Device detection fails	GPU: device detection failed: {reason} (warn)	Falls back to CPU.
Primary GPU context retain fails	GPU: primary context retain failed: {reason} (warn)	Falls back to CPU.
Kernel registry load fails	GPU: kernel registry failed: {reason} (warn)	Falls back to CPU.
Stream pool creation fails	GPU: stream pool failed: {reason} (warn)	Falls back to CPU.
Insufficient hardware tier (measured PCIe bandwidth or free VRAM too low)	GPU: insufficient hardware (PCIe {x} GB/s, {y} GB free VRAM) - GPU disabled (warn)	Falls back to CPU.
A sticky GPU context error during a query, any time after startup	GPU marked unhealthy, all queries run CPU-only until restart: {reason} (error)	GPU is marked unhealthy for the rest of the process lifetime. It is not retried per query. A server restart is required to try GPU again.

Per-query fallback

Even with a healthy GPU, an individual function or operation may decline GPU dispatch, always to a clean CPU execution:

• Any function that calls a UDF is never GPU-dispatched (there is no compiled representation for a host UDF callback on GPU).
• A join build or probe touching a currently round-active join slot (spill handling mid-round) never runs on GPU for that slot.
• A GROUP BY or join key whose physical type is outside the GPU whitelist (Int32, Date32, Int64, Timestamp, UInt64, Float64, dictionary-encoded Utf8) declines to CPU. This excludes Decimal128, Bool, Uuid, Int8/Int16, Bytea, and Interval, even though all of those are otherwise valid GROUP BY and join keys on CPU.

The size and cost floors

[gpu]
min_batch_size = 50000

A batch below min_batch_size (default 50000 rows) never dispatches to GPU, regardless of eligibility. Above that floor, dispatch is also cost-gated: ScramDB estimates the data transfer time to the device plus GPU compute time, and only dispatches when the total estimated GPU time is at least 1.5x faster than the estimated CPU (JIT) time. A batch can clear the row-count floor and still run on CPU if the transfer cost to move data onto the device dominates.

This is why small benchmarks should not force GPU: a single-scale-factor TPC-H run sits below the GPU size floor for most of its queries, and forcing GPU dispatch on data that small measurably lost performance in testing, by as much as 39% on one query. GPU is a lever for large, memory-bound batches, not a blanket accelerator.

Full [gpu] config reference

Key	Default	What it controls
enabled	true	Master on/off switch for GPU dispatch.
device_id	0	Which device to use; -1 auto-selects the highest compute-capability device.
vram_budget	0 (= 90% of free VRAM)	VRAM ScramDB is allowed to use.
vram_hash_table_budget	0 (= 60% of vram_budget)	VRAM reserved specifically for GPU hash tables.
pinned_memory_pool	"256MB"	Size of the pinned host memory pool used for host-to-device transfers.
streams_per_worker	2	GPU streams per worker (validated 1-16).
min_batch_size	50000	Row-count floor below which GPU is never dispatched.
kernel_timeout_ms	5000	Per-kernel timeout in milliseconds; 0 disables the timeout (validated to be 0 or >= 100).
vendor	"auto"	auto, nvidia, amd, or apple. Pins dispatch to one vendor; auto detects whichever supported device is present.
arch_override	""	Force a specific compute architecture instead of auto-detecting; empty means auto-detect.
metrics_enabled	true	Reserved for future Prometheus GPU metrics; has no observable effect today, see below.
debug_launches	false	Reserved for kernel-launch debug logging beyond the standard debug log lines below.

An opt-in benchmarking override

SCRAMDB_GPU_FORCE=1 scramdb -c scramdb-config.toml --pg-address 127.0.0.1:5432 --pg-no-auth

SCRAMDB_GPU_FORCE=1 (or "true") forces pipeline-shape eligibility, skipping nothing that is structurally eligible for GPU. The per-morsel and per-chain size and cost floors above still apply on top of it. This is an override for benchmarking, not something to run in production, given the size-floor behavior described above.

How to verify which path actually ran

GPU Prometheus metrics are not exposed on /metrics today. Do not scrape for scramdb_gpu_* counters, they will not appear even though metrics_enabled = true is the default. Verification is log-based:

1. Startup detection: at the default info log level, a successful GPU init logs a line naming the device, architecture, compute capability, VRAM, budget, kernel count, and measured PCIe bandwidth. Its absence (replaced by one of the fallback lines above) means the server is CPU-only for its whole lifetime.

2. Per-query dispatch: set SCRAMDB_LOG=debug and grep server output for [GPU]:

SCRAMDB_LOG=debug scramdb -c scramdb-config.toml --pg-address 127.0.0.1:5432 --pg-no-auth 2> scramdb.log
# in another terminal, run your query, then:
grep '\[GPU\]' scramdb.log

Expected lines: [GPU] {func} pipeline: DISPATCH {n} rows - {reason} when a pipeline actually dispatches to the device, [GPU] {func} pipeline: SKIP {n} rows - not worthwhile ({reason}) when eligible but not cost-effective, or (at trace level) [GPU] {func} pipeline: NOT ELIGIBLE - {reason} when structurally ineligible.

If it fails: no [GPU] lines appear at all, either the query's batches never cleared min_batch_size, the GPU was never initialized (check the startup log line first), or the query's operators and types are outside the GPU-eligible set described above.

Next

See Query Performance for the CPU-side JIT tier GPU sits alongside, or Observability for the full picture of what is and is not on /metrics today.