Skip to main content

Architecture Overview

ScramDB is a UTAP (Unified Transactional Analytical Processing) SQL database written in Rust: transactions, analytics, and AI on one live copy of your data.

How a Query Executes​

SQL Query
  → PostgreSQL Wire Protocol
    → SQL Parser
      → Cost-Based Optimizer
        → Bytecode Compiler
          → JIT Compiler (background)
            → Parallel Execution (core-pinned workers)
              → Results (streamed via PgWire)

Key Design Decisions​

JIT Compilation​

ScramDB compiles SQL queries to native machine code instead of interpreting them. Queries are transparently compiled in the background, with no configuration needed. Compiled code is cached to disk and reused across restarts.

Morsel-Driven Parallelism​

Each query is split into pipelines that process data in parallel "morsels." Workers are pinned to CPU cores for zero-overhead scheduling and linear scaling.

Tundra Columnar Storage​

Custom columnar storage engine with automatic predicate pushdown, in-memory caching, block-level integrity checks, and crash recovery.

Transactions​

Full MVCC with PostgreSQL-compatible isolation levels: Read Committed, Repeatable Read, and Serializable.

Indexing​

Create indexes on the columns you query most. The optimizer builds and selects the right access path for point lookups, range scans, and equality checks automatically, with no manual tuning.

Distributed Serializable Cluster​

Run ScramDB on a single node or scale it out to a cluster. The cluster provides serializable transactions across nodes, with data replicated for durability and availability, and no separate coordinator or query router to operate. A single-node instance and a cluster speak the same SQL and the same wire protocol.

GPU Acceleration​

Eligible queries are accelerated on the GPU when one is available, on NVIDIA, AMD, and Apple Metal. The planner decides when the GPU wins and falls back to the CPU otherwise, so it is transparent to your SQL. See GPU acceleration for details.

Point-in-Time Recovery and Database Branching​

Continuous write-ahead logging lets you restore to any point in time. You can also branch a database instantly, including as of a past moment, to spin up a copy for testing, analytics, or auditing without duplicating storage.

Programmable UDF Runtime​

Write user-defined functions in JavaScript, TypeScript, Rust, Go, C, C++, Python, and Ruby, running in a sandboxed, resource-bounded runtime. Write JavaScript, Python and Ruby inline with AS $$...$$, or build any of the compiled languages into a package and bind it with LANGUAGE js AS 'namespace/pkg'. ScramDB also ships 15 built-in analytics packages covering full-text search, geospatial, embeddings, clustering, and approximate aggregates.

Access Control​

Role-based access control (RBAC) and row-level security let you grant privileges per role and restrict which rows each role can see, enforced inside the engine.

PostgreSQL Compatibility​

Full PostgreSQL wire protocol - connect with psql, JDBC, psycopg2, or any PG driver. Standard SQL syntax, not a custom query language.