SDB1 draft · controlled endpoints

Authenticate the bytes you already have.

SDB1 is a bounded binary record with raw attached or detached payloads, HMAC-SHA-256 or Ed25519, external AAD, opaque application claims, and a borrowed-buffer path. It performs no JSON, Base64, Marshal, or application-object serialization on that path. Applications—not the wire format—define identity and session meaning.

Choose it for the buffer path

SDB1 fits private protocols whose producers and consumers are under common control, whose payload already exists as bytes, and whose bounded parsing or allocation behavior matters. Choose JWS or COSE when an IETF standard, third-party tooling, multi-signatures, countersignatures, or a mature key ecosystem matters more.

Draft, not a standards replacementThe format has two same-project implementations and a portable C scanner, but no independent implementation or external audit. Pin the revision and plan migration before production use.

Sign and verify raw bytes in Ruby

require "signed_data/sdb1"

signer = SignedData::HMACSigner.new(
  ENV.fetch("SIGNED_DATA_SECRET").b, # at least 32 random bytes
  key_id: "edge-2026-08".b
)

record = SignedData.sign_binary(
  File.binread("packet.bin"),
  signer: signer,
  purpose: "packet/upload/1".b,
  content_type: "application/octet-stream".b,
  schema_id: "urn:example:packet:1".b,
  issuer: "ingress-a".b,
  audience: "object-store".b,
  not_before: Time.now.to_i,
  expires_at: Time.now.to_i + 60
)

verified = SignedData.verify_binary(
  record,
  signer: signer,
  purpose: "packet/upload/1".b,
  content_type: "application/octet-stream".b,
  schema_id: "urn:example:packet:1".b,
  issuer: "ingress-a".b,
  audience: "object-store".b
)

verified.payload # frozen byte-identical owned snapshot

The signed_data/sdb1 entry point loads only the raw-wire implementation and crypto adapters. It does not load the legacy JSON/Base64 envelope, serialization API, or ParcelStream.

HMAC lets every verifier mint records. When verifiers should hold only public material, use SignedData::Ed25519Signer to sign and signer.verifier to distribute the public-key verifier.

scan_binary returns unverified source offsets without materializing the payload during the scan; asking a Ruby span for bytes creates a byteslice String. verify_binary copies and re-scans the record to close mutable-buffer races, verifies native crypto, applies caller policy, and returns an owned frozen snapshot. sign_binary likewise snapshots payload/AAD bytes and assembles one contiguous record. The explicitly unsafe JavaScript visitor and vectored builder, not the Ruby API, are the current high-level borrowed fast paths.

Use the dependency-free browser module

Download the checked-in SDB1 ES module. It uses Uint8Array for byte-valued inputs and outputs and includes native Web Crypto HMAC and Ed25519 adapters.

import {
  createWebCryptoEd25519Verifier,
  verifyBinaryFrame
} from "./signed-data-wire.mjs";

const verifier = createWebCryptoEd25519Verifier(publicKey, { keyId });
const verified = await verifyBinaryFrame(receivedBytes, {
  verifier,
  expectedPurpose: purposeBytes,
  expectedContentType: contentTypeBytes,
  expectedSchemaId: schemaIdBytes,
  expectedIssuer: issuerBytes,
  expectedAudience: audienceBytes,
  externalAad: connectionContext,
  maxExpiresIn: 300
});

await consume(verified.payload); // defensive copy

For a genuinely leased immutable receive buffer, visitUnsafeBorrowedBinaryFrame exposes an unsafe trusted-callback optimization. Its scoped lease is contractual: JavaScript cannot revoke a Uint8Array retained by callback code. It rejects verifier adapters unless they explicitly attest borrowedVerificationCompatible === true; the built-in Web Crypto adapters attest, while custom adapters must opt in deliberately. That flag is still a contract, not a sandbox. Retaining any view after the callback, mutating it, or recycling its backing frame early violates the contract and creates a time-of-check/time-of-use bug. Use verifyBinaryFrame when callback code or the adapter is not fully trusted to honor that rule.

Critical application extensions require exact raw-byte policy:

expectedCriticalExtensions: new Map([
  [0x8001, expectedProfileBytes]
])

A fixed 64-byte front door

All integers use network byte order. The attached layout is header || raw payload || raw signature. Detached records omit the payload physically while retaining its signed logical length.

OffsetBytesField
04SDB1 magic
41revision 1
51flags
62header length
84physical record length
124logical payload length
162signature length
182algorithm ID
202extension count
222reserved zero
248routing hint
328key-ID classifier tag
408purpose classifier tag
488issued-at
568expires-at

Flags declare issued-at (01), expiry (02), key ID (04), purpose (08), routing hint (10), detached payload (20), and external AAD (40). Bit 80 is reserved and rejected.

Extensions are ordered, unique, eight-byte-aligned TLVs. The header is at most 4096 bytes with at most 32 entries. Registered critical entries cover key ID, purpose, content type, schema ID, AAD length, issuer, audience, subject, consume-once token ID, and not-before. Unknown critical entries fail closed.

Content type and schema ID are authenticated opaque bytes, not a decoder switch. A verifier must supply exact expected values. SDB1 never loads a codec, fetches a schema, or deserializes an application object.

signature input = complete header
                || logical payload
                || external AAD when declared

Length fields authenticate the boundaries. HMAC-SHA-256 is algorithm 0001 with a 32-byte MAC and requires at least 32 secret-key bytes; custom adapters must explicitly attest to that profile. Ed25519 is 0002 with 64 raw RFC 8032 signature bytes.

A valid signature is not yet authorization

  1. Apply local record, payload, header, signature, extension, and AAD limits.
  2. Validate every fixed field, length equation, extension, flag, and padding byte.
  3. Resolve an allowed verifier from trusted configuration; key and routing hints are still attacker-controlled.
  4. Authenticate the exact header, payload, and AAD spans.
  5. Require exact purpose, content type, schema ID, issuer, and audience expectations when present; apply subject and routing policy.
  6. Enforce not-before and expiry with explicit clock skew and a local future-expiry horizon via max_expires_in in Ruby or maxExpiresIn in JavaScript. Issued-at is provenance unless the application adds a freshness rule.
  7. Atomically consume a token ID last. A token ID requires both purpose and expiry.

The consume callback must be atomic across every process or region that accepts the token. An expiry field alone does not bound replay-state retention when its value can be far in the future, so configure the local future-expiry horizon before consumption. Clock leeway extends the acceptance boundary: retain the consumed marker through the callback's replay_retain_until/replayRetainUntil value (expiry plus skew), not merely through the unsigned expiry. Key replay state by a structured tuple of trust domain, complete purpose, token ID, and issuer when present; raw concatenation is ambiguous. The callback also receives key ID, algorithm ID, and routing hint for key-specific namespaces. Returning anything except true fails verification. Encryption, authorization rules, replay storage, key discovery, and payload schemas remain application responsibilities.

XDP classifies; userspace authorizes

An XDP/eBPF program may bounds-check the fixed header, exclude impossible sizes or fragmentation from a fast queue, rate-limit, classify by an untrusted tag, and redirect a candidate to AF_XDP. It must not authorize the payload. Userspace still performs the complete scan, MAC/signature verification, claim checks, and replay consumption.

The included classifier is maintained in typed examples/xdp_sdb1_redirect.nelua. Its adjacent .c file is deterministic generated output, committed so offline BPF builds need no Nelua compiler; the shared include/signed_data_wire.h scanner remains portable, hand-maintained C. bundle exec rake nelua:check detects stale output offline and bundle exec rake test:xdp compiles and inspects that checked-in C. Maintainers regenerate and byte-verify it with commit-pinned Nelua plus ambient self-hosted Clang, LLVM, libbpf headers, and a host C compiler via bundle exec rake nelua:generate and bundle exec rake nelua:verify.

An import-free examples/sdb1_udp_classifier.odin now implements the same raw-pointer userspace boundary independently. It is a native comparison and AF_XDP-consumer building block, not an XDP source: the official Odin compiler has no supported BPF/eBPF target. Both host peers return offsets into the caller-owned buffer, allocate nothing, and are checked against the C reference on the same mutation corpus.

The compile-tested classifier accepts one complete attached SDB1 record without external AAD in one unfragmented plaintext IPv4/UDP datagram at a known offset. VLAN, IPv6, multi-buffer, malformed, detached, and external-AAD traffic falls back to the normal stack with XDP_PASS. Redirect identifies a candidate for userspace verification; it is not authentication. The program is not yet loaded/verifier-tested or a complete AF_XDP UMEM verifier.

QUIC encrypts application bytes: scan QUIC DATAGRAM data after userspace decryption, and reassemble QUIC streams or TCP before SDB1 framing. AF_XDP zero-copy depends on the NIC, driver, queue, UMEM, headroom, and frame lifecycle; an SDB1 borrowed view alone cannot guarantee it.

Ruby HMAC feeds header, payload, and AAD incrementally, but sign_binary snapshots those bytes and assembles a record. Current Web Crypto HMAC/Ed25519 and Ruby OpenSSL Ed25519 adapters gather for one-shot primitives. A custom signParts/verifyParts adapter avoids that gather only when its native crypto really supports streaming or scatter/gather.

Measure the path, not the slogan

The checked-in CPU-pinned comparison completed 61,682 assertions with no failures and measured zero allocator activity in C, generated Nelua, and Odin across portable and native-tuned builds. Odin portable led complete valid IPv4/UDP classification on that worker; Nelua portable led the hot, alignment, and malformed scan rows. Aggressive tuning regressed several hot paths.

Explore the charts, allocation proof, code sizes, exact workload rows, and XDP artifact boundary →

Be precise against JWS and COSE

CapabilitySDB1JWS CompactCOSE_Sign1COSE_Mac0
Raw payload pathNativeBase64URL default; unencoded optionCBOR byte string or detachedCBOR byte string or detached
Detached payloadYesYesYesYes
Separate external AADYesNo general inputYesYes
HMAC-SHA-256Native IDStandard profileNot a signature algorithmStandard MAC algorithm
Ed25519Native IDEdDSA profileStandard signature algorithmNot a MAC algorithm
Critical extensionsYesYesYesYes
Self-descriptionExact critical type + schemaProtected headers + profileProtected headers + profileProtected headers + profile
Bounded fixed prefix64 bytesNoImplementation-dependent CBORImplementation-dependent CBOR
Standard ecosystemNoBroadBroadBroad
Multi-signature / countersignatureNoOther JWS serializationOther COSE structuresOther COSE structures
ConfidentialityOpenPGP outer compositionUse JWEUse COSE encryptionUse COSE encryption

SDB1 is better shaped for a bounded parse and borrowed hot path. Ed25519 corresponds to the signature-oriented COSE_Sign1 structure; HMAC-SHA-256 corresponds to the MAC-oriented COSE_Mac0 structure. Both COSE structures support external AAD. SDB1 does not match their standards status, ecosystem, countersignature, or encryption families.

What must happen before a durable release

  • Freeze a version and extension/algorithm registry policy.
  • Publish packages and stable test vectors.
  • Obtain external security review and a genuinely independent implementation outside this project.
  • Extend the structural/allocation benchmark to real crypto, AF_XDP/NIC paths, and well-configured JWS/COSE libraries.
  • Keep the older sd1 text profile explicit; migration always verifies and re-signs.

The live workbench on the home page is intentionally the legacy sd1 lab. It does not demonstrate SDB1's raw binary path.