GPU Acceleration
By the end of this page you will know when ScramDB dispatches work to a GPU, every reason it silently falls back to CPU instead, and how to confirm which path a query actually took.
GPU acceleration is a large-batch lever, not a default-on win for every query. Small workloads should expect CPU execution and that is the correct outcome, not a misconfiguration.
Requirements
GPU dispatch requires all of the following to be true:
- The server binary was built with GPU support. A standard release build includes GPU support for NVIDIA, AMD, and Apple Metal by default.
[gpu] enabledistrue(the default).- A supported GPU (NVIDIA, AMD, or Apple Silicon) with a working driver is present at startup.
[gpu] vendor accepts "auto" (default), "nvidia", "amd", or "apple". "auto" detects whichever supported device is present; set it explicitly to pin dispatch to one vendor on a machine with more than one GPU installed.
Enabling GPU
[gpu]
enabled = true
SCRAMDB_LOG=info scramdb -c scramdb-config.toml --pg-address 127.0.0.1:5432 --pg-no-auth
Expected: a startup log line containing GPU: and the detected device name, if a supported GPU (NVIDIA, AMD, or Apple Silicon) with a working driver is present. If GPU is disabled in config, you will instead see GPU: disabled by configuration. If no supported device is visible, you will see GPU: no devices found. Either fallback case is not an error, the server still starts and serves every query on CPU.
If it fails: confirm the binary was built with GPU support, and confirm your GPU driver sees the device from the same host or container the server runs in (for example, nvidia-smi on NVIDIA).
Why a query falls back to CPU
Fallback is always clean, never a crash. It happens at two different levels: server startup (GPU never initializes at all) and per-query (GPU is available but this specific query declines it).
Startup-level fallback
| Condition | Log line (level) | Effect |
|---|---|---|
[gpu] enabled = false | GPU: disabled by configuration (info) | GPU state never initializes; every query runs CPU-only. |
| Binary built without GPU support | no GPU log line at all | Same effect, no GPU code is compiled in. |
| No supported device found | GPU: no devices found (info) | Falls back to CPU. |
| Device detection fails | GPU: device detection failed: {reason} (warn) | Falls back to CPU. |
| Primary GPU context retain fails | GPU: primary context retain failed: {reason} (warn) | Falls back to CPU. |
| Kernel registry load fails | GPU: kernel registry failed: {reason} (warn) | Falls back to CPU. |
| Stream pool creation fails | GPU: stream pool failed: {reason} (warn) | Falls back to CPU. |
| Insufficient hardware tier (less than 4 GB of free VRAM, or a link whose bandwidth could not be measured; a slow link keeps the device up, and each batch's transfer cost decides whether it runs there) | GPU: insufficient hardware (PCIe {x} GB/s, {y} GB free VRAM) - GPU disabled (warn) | Falls back to CPU. |
| A sticky GPU context error during a query, any time after startup | GPU marked unhealthy, all queries run CPU-only until restart: {reason} (error) | GPU is marked unhealthy for the rest of the process lifetime. It is not retried per query. A server restart is required to try GPU again. |
Per-query fallback
Even with a healthy GPU, an individual function or operation may decline GPU dispatch, always to a clean CPU execution:
- Any function that calls a UDF is never GPU-dispatched (there is no compiled representation for a host UDF callback on GPU).
- A join build or probe touching a currently round-active join slot (spill handling mid-round) never runs on GPU for that slot.
- A
GROUP BYor join key whose physical type is outside the GPU whitelist (Int32,Date32,Int64,Timestamp,UInt64,Float64, dictionary-encodedUtf8) declines to CPU. This excludesDecimal128,Bool,Uuid,Int8/Int16,Bytea, andInterval, even though all of those are otherwise validGROUP BYand join keys on CPU. - A filter predicate over a nullable column declines to CPU. On a table whose filtered columns are nullable, this is the most likely reason a scan you expected to dispatch reports as not eligible. Declaring the column
NOT NULLrestores GPU eligibility. - A comparison between two columns (
WHERE a > b) or against a computed expression (WHERE a > b * 2) is refused outright, as is aLIKEpredicate. A predicate comparing a column of an integer, date, timestamp, double precision or text type to a constant is the eligible shape. - A condition the GPU row filter cannot read exactly when the query runs, such as a
NUMERICconstant against an integer column or a parameter bound toNULL, sends that work back to the CPU and counts one fallback (scramdb_gpu_fallback_total). So does a batch whose aggregated or grouped columns hold aNULL. - A text comparison runs on the GPU path only where its collation orders by bytes (
C,POSIX,C.UTF-8, orCOLLATE "C"), or as an equality under a deterministic collation. An ordering comparison under a linguistic collation such asen_US, or an equality under a nondeterministic ICU collation, keeps the pipeline on the CPU, which compares in the collation's own order. - A grouped aggregation's hash table is sized from the data: from the most groups one batch has held, and again from the batch's own rows when a batch holds more, so any number of groups runs on the GPU. A batch whose table does not fit the GPU memory budget runs on the CPU instead and counts one fallback and one
scramdb_gpu_oom_total.
One refused predicate makes the whole pipeline ineligible, not just that one filter, so a single nullable-column comparison is enough to move an entire query back to the CPU.
The size and cost floors
[gpu]
min_batch_size = 50000
A batch below min_batch_size (default 50000 rows) never dispatches to GPU, regardless of eligibility. Above that floor, dispatch is also cost-gated: ScramDB estimates the data transfer time to the device plus GPU compute time, and only dispatches when the total estimated GPU time is at least 1.5x faster than the estimated CPU (JIT) time. A batch can clear the row-count floor and still run on CPU if the transfer cost to move data onto the device dominates.
This is why small benchmarks should not force GPU: a single-scale-factor TPC-H run sits below the GPU size floor for most of its queries, and forcing GPU dispatch on data that small measurably lost performance in testing, by as much as 39% on one query. GPU is a lever for large, memory-bound batches, not a blanket accelerator.
Full [gpu] config reference
| Key | Default | What it controls |
|---|---|---|
enabled | true | Master on/off switch for GPU dispatch. |
device_id | 0 | Which device to use; -1 auto-selects the highest compute-capability device. |
vram_budget | 0 (= 90% of free VRAM) | VRAM ScramDB is allowed to use. |
vram_hash_table_budget | 0 (= 60% of vram_budget) | VRAM reserved specifically for GPU hash tables. |
pinned_memory_pool | "256MB" | Size of the pinned host memory pool used for host-to-device transfers. |
streams_per_worker | 2 | GPU streams per worker (validated 1-16). |
min_batch_size | 50000 | Row-count floor below which GPU is never dispatched. |
kernel_timeout_ms | 5000 | Per-kernel timeout in milliseconds; 0 disables the timeout (validated to be 0 or >= 100). |
vendor | "auto" | auto, nvidia, amd, or apple. Pins dispatch to one vendor; auto detects whichever supported device is present. |
arch_override | "" | Force a specific compute architecture instead of auto-detecting; empty means auto-detect. |
metrics_enabled | true | Puts the scramdb_gpu_* families on /metrics once a GPU is in use; false leaves them off. |
debug_launches | false | Logs a GPU launch: line for every kernel launch (the kernel, its rows and grid) at debug level; needs SCRAMDB_LOG=debug. |
An opt-in benchmarking override
SCRAMDB_GPU_FORCE=1 scramdb -c scramdb-config.toml --pg-address 127.0.0.1:5432 --pg-no-auth
SCRAMDB_GPU_FORCE=1 (or "true") forces pipeline-shape eligibility, skipping nothing that is structurally eligible for GPU and skipping the transfer-versus-compute cost estimate. The min_batch_size floor above still applies on top of it. It is read once, when the server brings the GPU up at startup. This is an override for benchmarking, not something to run in production, given the size-floor behavior described above.
How to verify which path actually ran
-
Startup detection: at the
infolog level (the default iswarn, so setSCRAMDB_LOG=info), a successful GPU init logs a line naming the device, architecture, compute capability, VRAM, budget, kernel count, and measured PCIe bandwidth. Its absence (replaced by one of the fallback lines above) means the server is CPU-only for its whole lifetime, and/metricscarries noscramdb_gpu_*family. -
Per-query dispatch, from the metrics: scrape
/metricsbefore and after the query. It ran on the GPU whenscramdb_gpu_kernel_launches_totalmoved andscramdb_gpu_fallback_totaldid not. A moved fallback counter means work the GPU accepted ran on the CPU instead (see the fallback reasons above). On a busy server, run the query alone to read its own counts.
curl -s http://127.0.0.1:9090/metrics | grep -E '^scramdb_gpu_(kernel_launches|fallback)_total'
Expected: two lines, scramdb_gpu_kernel_launches_total <n> and scramdb_gpu_fallback_total <n>. The full family list is in Observability.
- Per-query dispatch, from the log: set
SCRAMDB_LOG=debugand grep server output for[GPU]:
SCRAMDB_LOG=debug scramdb -c scramdb-config.toml --pg-address 127.0.0.1:5432 --pg-no-auth 2> scramdb.log
# in another terminal, run your query, then:
grep '\[GPU\]' scramdb.log
Expected lines: [GPU] {func} pipeline: DISPATCH {n} rows - {reason} when a pipeline's batches ran on the device, [GPU] {func} pipeline: SKIP {n} rows - not worthwhile ({reason}) when a batch fell below the size or cost floor, or (at trace level) [GPU] {func} pipeline: NOT ELIGIBLE - {reason} when the pipeline has no GPU plan.
If it fails: no [GPU] lines and no movement of the launch counter, either the query's batches never cleared min_batch_size, the GPU was never initialized (check the startup log line first), or the query's operators and types are outside the GPU-eligible set described above.
Next
See Query Performance for the CPU-side JIT tier GPU sits alongside, or Observability for every family on /metrics.