Writing a package

Packages are built on afterburner, a polyglot, deterministic WebAssembly runtime with its own package format, registry, and Cargo-style toolchain. You author, test, build, and publish packages with the burn command-line tool.

A package is the right tool when you have more than a few lines of code, want to write in a compiled language, want to share the function with others, or want the code versioned and reviewable independently of the database. For a quick one-off function, skip packages entirely and write CREATE FUNCTION ... LANGUAGE js AS $$...$$ inline; see Program your Database.

Install the toolchain

Install burn once:

curl -fsSL https://afterburner.sh | sh

On Windows, run iwr -useb https://afterburner.sh | iex in PowerShell. To pin a version, set BURN_VERSION first (for example BURN_VERSION=v0.1.3), or grab a tarball from the releases page.

Scaffold a package

burn init creates a new package with the manifest, capability grant, and a starter source file already in place. We will follow the built-in embeddings_cosine_similarity package through the rest of this page, so scaffold it here (swap in your own namespace and name for your own packages):

burn init ./embeddings_cosine_similarity --namespace scramdb --name embeddings_cosine_similarity

Develop against it the way you would any project:

burn run      # run the package entry, like `cargo run`
burn test     # run the tests in tests/, sandboxed

Anatomy of a package

A package is a directory with three parts: a manifest that describes it, a capability grant that says what it may touch, and your source code.

embeddings_cosine_similarity/
  afb.toml         # the manifest: what this package is
  manifold.json    # the capability grant: what it may touch (nothing, by default)
  source/
    main.js        # your code: the function body

This three-part layout is the same across every supported language; only the entry file and the language field in afb.toml change (see Writing in another language below).

The manifest (afb.toml)

afb.toml names the package and points at its entry file. Here is the manifest for one of the built-in starter packages:

[format]
version = "1.0"

[package]
name = "embeddings_cosine_similarity"
namespace = "scramdb"
version = "0.1.0"
language = "js"
entry = "source/main.js"
description = "Cosine similarity of two embedding vectors, each a JSON array of numbers."

[runtime]
min = "0.1.0"

The fields you set are:

Field	Meaning
name	The package name, and the name you use when you register it in SQL
namespace	Your namespace, for example your team or organization
version	The package version (semantic versioning)
language	The source language (js for JavaScript, rust for Rust, and so on)
entry	The source file the package starts from
description	A short, human-readable description

The [format] and [runtime] sections are set for you when you scaffold a package and rarely need to change.

Sealed by default (manifold.json)

manifold.json is the package's capability grant. When you scaffold a package it grants nothing, so your code starts fully sealed:

{
  "fs": "None",
  "net": "None",
  "env": "None",
  "crypto": false,
  "child_process": false,
  "allow_exit": false,
  "http_timeout_ms": null,
  "listen": "None"
}

Each field is a door: fs is filesystem access, net is outbound network access, env is reading environment variables, crypto is cryptographic APIs, child_process is spawning other processes, allow_exit permits the process to exit on its own, and http_timeout_ms/listen are only meaningful once net/child_process are open.

For a database function you almost always keep every door shut. A function that only computes over its arguments needs nothing else, and sealing is exactly what makes it safe to run inside the database. Whatever a package may touch is declared here, in the open, and reviewable before anyone installs it.

Every installed package runs fully sealed today, regardless of what its own manifold.json declares. The manifest is bookkeeping and documentation for the reader; the engine additionally forces every module sealed no matter what it asks for. A package cannot currently open net, fs, crypto, or child_process at all, even if its own manifest claims one of those doors is open.

The function (source/main.js)

A ScramDB function is handed a whole batch of rows at once and returns one result per row. This batch-at-a-time shape is why user functions keep up with ScramDB's query engine instead of slowing it down. It is the calling convention for JavaScript, whether the source lives inline in a CREATE FUNCTION ... AS $$...$$ statement or in a source-only package's entry file.

The entry file exports one function:

module.exports = (batch) => {
  // read input columns, compute, return an output column
};

The batch your function receives, and the object it returns, follow a simple contract:

• batch.columns.<name> is the input column for each declared SQL argument: an array with one value per row.
• batch.row_count is how many rows are in the batch.
• Return { row_count, columns: { <output>: <array> } } with one output column, one value per row. The row count you return must match the row count you were given, or the call fails.

Here is the full body of the embeddings_cosine_similarity starter package. It takes two vectors (each a JSON array of numbers, stored as text) and returns their cosine similarity:

module.exports = (batch) => {
  const a = batch.columns.a;
  const b = batch.columns.b;
  const n = batch.row_count;
  const out = new Float64Array(n);

  for (let i = 0; i < n; i++) {
    const av = JSON.parse(a[i]);
    const bv = JSON.parse(b[i]);
    if (av.length !== bv.length) {
      throw new Error("vector length mismatch (" + av.length + " vs " + bv.length + ")");
    }
    let dot = 0, na = 0, nb = 0;
    for (let j = 0; j < av.length; j++) {
      dot += av[j] * bv[j];
      na += av[j] * av[j];
      nb += bv[j] * bv[j];
    }
    const denom = Math.sqrt(na) * Math.sqrt(nb);
    out[i] = denom !== 0 ? dot / denom : 0;
  }

  return { row_count: n, columns: { similarity: out } };
};

Reading it top to bottom:

• a and b are the two input columns, named to match the SQL arguments you will declare (a text, b text).
• The loop runs once per row, reading a[i] and b[i] and writing out[i].
• The function returns a single similarity column, which becomes the value of RETURNS double precision when you register it.

Notice there is no I/O, no clock, and no randomness: the same inputs always produce the same result. Throwing an error on bad input (a length mismatch here) fails loudly rather than returning a wrong answer.

Once a package like this is built, it is installed and bound the same way whether it is a starter package or one of your own: scram.install followed by CREATE FUNCTION ... AS 'namespace/pkg'. See Install and use a package.

Inline functions in Python and Ruby

Python and Ruby do not use packages: LANGUAGE python and LANGUAGE ruby only accept inline source (AS $$...$$), never AS 'namespace/pkg'. Both follow the same convention: a top-level function named exactly the SQL function's own name, taking one batch argument.

Python's batch is a plain dict, accessed with [...], not attribute access:

CREATE OR REPLACE FUNCTION double_it(x bigint) RETURNS bigint LANGUAGE python AS $$
def double_it(batch):
    xs = batch['columns']['x']
    n = batch['row_count']
    return {'row_count': n, 'columns': {'r': [v * 2 for v in xs]}}
$$;

Ruby's batch is a Hash with symbol keys at the top level (:columns, :row_count), but the column map inside :columns is keyed by plain strings, the argument name:

CREATE OR REPLACE FUNCTION double_it(x bigint) RETURNS bigint LANGUAGE ruby AS $$
def double_it(batch)
  xs = batch[:columns]['x']
  n = batch[:row_count]
  { row_count: n, columns: { 'r' => xs.map { |v| v * 2 } } }
end
$$;

Output type inference limit. For both Python and Ruby, the wire type of each output column is inferred from its first value only: a bool becomes boolean, an int becomes bigint, a float becomes double precision, a text string becomes text, and a raw byte string becomes bytea. There is no way today to select a narrower type such as int, real, or date from Python or Ruby. Declare RETURNS to match what the inference actually produces (bigint, double precision, boolean, or text), not the narrower SQL type you might otherwise reach for.

No per-call resource cap yet. JavaScript UDFs run under the fuel, memory, timeout, and output-size limits described in Install and use a package. Python and Ruby do not have an equivalent per-call cap today: a runaway Python or Ruby function body has no engine-side wall-clock or memory ceiling the way a JavaScript one does. Keep this in mind before exposing a Python or Ruby UDF to untrusted input.

Writing in another language

Only the language field in afb.toml and the entry file change across languages:

Language	afb.toml language	Entry file	Runs as
JavaScript	javascript (or js)	source/main.js	Interpreted by afterburner's own JS engine
TypeScript	typescript	source/main.ts	Interpreted by afterburner's own JS engine, same as JavaScript, once built into a package (see below)
Rust	rust	source/main.rs	Compiled ahead of time to a native WebAssembly binary
Go	go	source/main.go	Compiled ahead of time to a native WebAssembly binary
C	c	source/main.c	Compiled ahead of time to a native WebAssembly binary
C++	cpp	source/main.cpp	Compiled ahead of time to a native WebAssembly binary

Native languages are compiled by their own toolchains into the same sealed .afb artifact, and multi-file projects are fully supported: a package can have several source files with cross-module calls and its own private helpers. ScramDB's built-in starter set includes a compiled Rust package (rust_sha256) right alongside the JavaScript ones.

TypeScript today. burn scaffolds, builds, and publishes a TypeScript package exactly like a JavaScript one, and it binds and runs the same way once installed: CREATE FUNCTION ... LANGUAGE js AS 'namespace/pkg' (see Install and use a package). The one gap is inline source: LANGUAGE ts is a real, recognized keyword in CREATE FUNCTION, but LANGUAGE ts AS $$...$$ is always refused, because the engine has no TypeScript transpiler wired in for inline compilation. Write inline functions in JavaScript instead; a TypeScript package needs no such workaround.

Two different calling conventions depending on how the package was built

This is the part that trips people up, so it gets its own section: which convention your package's .afb uses depends on how it was built, not on what language you typed.

• A source-only package (JavaScript or TypeScript, kept as interpretable source inside the .afb) is executed the same way inline LANGUAGE js AS $$...$$ is: the entry file exports module.exports = (batch) => {...}, described above under The function.
• A precompiled package (Rust, Go, C, or C++, compiled ahead of time to a real WebAssembly binary) runs as a plain command: it reads one JSON value from stdin and writes one JSON value to stdout. The input is either a single row object, keyed by your function's own declared SQL argument names, or a JSON array of row objects for a batch call. The output is the matching shape: a single JSON scalar, or a JSON array of scalars. A malformed input must make your program exit non-zero with a message on stderr, never panic.

Concretely, a Rust package's main looks nothing like the JavaScript module.exports(batch) shape. It reads stdin, parses one row (or an array of rows) as JSON, computes a scalar (or array of scalars) per row, and writes that back to stdout as JSON. This is the contract the built-in rust_sha256 package follows, and it is the contract any Rust, Go, C, or C++ package must follow, because those languages are compiled ahead of time rather than interpreted inside afterburner's own JS/TS engine.

Both conventions are bound identically from SQL, once installed:

CREATE OR REPLACE FUNCTION rust_sha256(value text, salt text)
RETURNS text
LANGUAGE js
AS 'scramdb/rust_sha256';

The LANGUAGE js in that statement names the binding form, not the package's implementation language; it is the only accepted keyword for AS 'namespace/pkg' today (Python and Ruby package references are refused; see Install and use a package).

Once your package is written, the next page shows how to build it, install it, and call it from SQL.