Choose it for the buffer path
SDB1 fits private protocols whose producers and consumers are under common control, whose payload already exists as bytes, and whose bounded parsing or allocation behavior matters. Choose JWS or COSE when an IETF standard, third-party tooling, multi-signatures, countersignatures, or a mature key ecosystem matters more.
Sign and verify raw bytes in Ruby
require "signed_data/sdb1"
signer = SignedData::HMACSigner.new(
ENV.fetch("SIGNED_DATA_SECRET").b, # at least 32 random bytes
key_id: "edge-2026-08".b
)
record = SignedData.sign_binary(
File.binread("packet.bin"),
signer: signer,
purpose: "packet/upload/1".b,
content_type: "application/octet-stream".b,
schema_id: "urn:example:packet:1".b,
issuer: "ingress-a".b,
audience: "object-store".b,
not_before: Time.now.to_i,
expires_at: Time.now.to_i + 60
)
verified = SignedData.verify_binary(
record,
signer: signer,
purpose: "packet/upload/1".b,
content_type: "application/octet-stream".b,
schema_id: "urn:example:packet:1".b,
issuer: "ingress-a".b,
audience: "object-store".b
)
verified.payload # frozen byte-identical owned snapshot
The signed_data/sdb1 entry point loads only the raw-wire implementation and crypto adapters. It does not load the legacy JSON/Base64 envelope, serialization API, or ParcelStream.
HMAC lets every verifier mint records. When verifiers should hold only public material, use SignedData::Ed25519Signer to sign and signer.verifier to distribute the public-key verifier.
scan_binary returns unverified source offsets without materializing the payload during the scan; asking a Ruby span for bytes creates a byteslice String. verify_binary copies and re-scans the record to close mutable-buffer races, verifies native crypto, applies caller policy, and returns an owned frozen snapshot. sign_binary likewise snapshots payload/AAD bytes and assembles one contiguous record. The explicitly unsafe JavaScript visitor and vectored builder, not the Ruby API, are the current high-level borrowed fast paths.
Use the dependency-free browser module
Download the checked-in SDB1 ES module. It uses Uint8Array for byte-valued inputs and outputs and includes native Web Crypto HMAC and Ed25519 adapters.
import {
createWebCryptoEd25519Verifier,
verifyBinaryFrame
} from "./signed-data-wire.mjs";
const verifier = createWebCryptoEd25519Verifier(publicKey, { keyId });
const verified = await verifyBinaryFrame(receivedBytes, {
verifier,
expectedPurpose: purposeBytes,
expectedContentType: contentTypeBytes,
expectedSchemaId: schemaIdBytes,
expectedIssuer: issuerBytes,
expectedAudience: audienceBytes,
externalAad: connectionContext,
maxExpiresIn: 300
});
await consume(verified.payload); // defensive copy
For a genuinely leased immutable receive buffer, visitUnsafeBorrowedBinaryFrame exposes an unsafe trusted-callback optimization. Its scoped lease is contractual: JavaScript cannot revoke a Uint8Array retained by callback code. It rejects verifier adapters unless they explicitly attest borrowedVerificationCompatible === true; the built-in Web Crypto adapters attest, while custom adapters must opt in deliberately. That flag is still a contract, not a sandbox. Retaining any view after the callback, mutating it, or recycling its backing frame early violates the contract and creates a time-of-check/time-of-use bug. Use verifyBinaryFrame when callback code or the adapter is not fully trusted to honor that rule.
Critical application extensions require exact raw-byte policy:
expectedCriticalExtensions: new Map([
[0x8001, expectedProfileBytes]
])
A fixed 64-byte front door
All integers use network byte order. The attached layout is header || raw payload || raw signature. Detached records omit the payload physically while retaining its signed logical length.
| Offset | Bytes | Field |
|---|---|---|
| 0 | 4 | SDB1 magic |
| 4 | 1 | revision 1 |
| 5 | 1 | flags |
| 6 | 2 | header length |
| 8 | 4 | physical record length |
| 12 | 4 | logical payload length |
| 16 | 2 | signature length |
| 18 | 2 | algorithm ID |
| 20 | 2 | extension count |
| 22 | 2 | reserved zero |
| 24 | 8 | routing hint |
| 32 | 8 | key-ID classifier tag |
| 40 | 8 | purpose classifier tag |
| 48 | 8 | issued-at |
| 56 | 8 | expires-at |
Flags declare issued-at (01), expiry (02), key ID (04), purpose (08), routing hint (10), detached payload (20), and external AAD (40). Bit 80 is reserved and rejected.
Extensions are ordered, unique, eight-byte-aligned TLVs. The header is at most 4096 bytes with at most 32 entries. Registered critical entries cover key ID, purpose, content type, schema ID, AAD length, issuer, audience, subject, consume-once token ID, and not-before. Unknown critical entries fail closed.
Content type and schema ID are authenticated opaque bytes, not a decoder switch. A verifier must supply exact expected values. SDB1 never loads a codec, fetches a schema, or deserializes an application object.
signature input = complete header
|| logical payload
|| external AAD when declared
Length fields authenticate the boundaries. HMAC-SHA-256 is algorithm 0001 with a 32-byte MAC and requires at least 32 secret-key bytes; custom adapters must explicitly attest to that profile. Ed25519 is 0002 with 64 raw RFC 8032 signature bytes.
A valid signature is not yet authorization
- Apply local record, payload, header, signature, extension, and AAD limits.
- Validate every fixed field, length equation, extension, flag, and padding byte.
- Resolve an allowed verifier from trusted configuration; key and routing hints are still attacker-controlled.
- Authenticate the exact header, payload, and AAD spans.
- Require exact purpose, content type, schema ID, issuer, and audience expectations when present; apply subject and routing policy.
- Enforce not-before and expiry with explicit clock skew and a local future-expiry horizon via
max_expires_inin Ruby ormaxExpiresInin JavaScript. Issued-at is provenance unless the application adds a freshness rule. - Atomically consume a token ID last. A token ID requires both purpose and expiry.
The consume callback must be atomic across every process or region that accepts the token. An expiry field alone does not bound replay-state retention when its value can be far in the future, so configure the local future-expiry horizon before consumption. Clock leeway extends the acceptance boundary: retain the consumed marker through the callback's replay_retain_until/replayRetainUntil value (expiry plus skew), not merely through the unsigned expiry. Key replay state by a structured tuple of trust domain, complete purpose, token ID, and issuer when present; raw concatenation is ambiguous. The callback also receives key ID, algorithm ID, and routing hint for key-specific namespaces. Returning anything except true fails verification. Encryption, authorization rules, replay storage, key discovery, and payload schemas remain application responsibilities.
XDP classifies; userspace authorizes
An XDP/eBPF program may bounds-check the fixed header, exclude impossible sizes or fragmentation from a fast queue, rate-limit, classify by an untrusted tag, and redirect a candidate to AF_XDP. It must not authorize the payload. Userspace still performs the complete scan, MAC/signature verification, claim checks, and replay consumption.
The included classifier is maintained in typed examples/xdp_sdb1_redirect.nelua. Its adjacent .c file is deterministic generated output, committed so offline BPF builds need no Nelua compiler; the shared include/signed_data_wire.h scanner remains portable, hand-maintained C. bundle exec rake nelua:check detects stale output offline and bundle exec rake test:xdp compiles and inspects that checked-in C. Maintainers regenerate and byte-verify it with commit-pinned Nelua plus ambient self-hosted Clang, LLVM, libbpf headers, and a host C compiler via bundle exec rake nelua:generate and bundle exec rake nelua:verify.
An import-free examples/sdb1_udp_classifier.odin now implements the same raw-pointer userspace boundary independently. It is a native comparison and AF_XDP-consumer building block, not an XDP source: the official Odin compiler has no supported BPF/eBPF target. Both host peers return offsets into the caller-owned buffer, allocate nothing, and are checked against the C reference on the same mutation corpus.
The compile-tested classifier accepts one complete attached SDB1 record without external AAD in one unfragmented plaintext IPv4/UDP datagram at a known offset. VLAN, IPv6, multi-buffer, malformed, detached, and external-AAD traffic falls back to the normal stack with XDP_PASS. Redirect identifies a candidate for userspace verification; it is not authentication. The program is not yet loaded/verifier-tested or a complete AF_XDP UMEM verifier.
QUIC encrypts application bytes: scan QUIC DATAGRAM data after userspace decryption, and reassemble QUIC streams or TCP before SDB1 framing. AF_XDP zero-copy depends on the NIC, driver, queue, UMEM, headroom, and frame lifecycle; an SDB1 borrowed view alone cannot guarantee it.
Ruby HMAC feeds header, payload, and AAD incrementally, but sign_binary snapshots those bytes and assembles a record. Current Web Crypto HMAC/Ed25519 and Ruby OpenSSL Ed25519 adapters gather for one-shot primitives. A custom signParts/verifyParts adapter avoids that gather only when its native crypto really supports streaming or scatter/gather.
Measure the path, not the slogan
The checked-in CPU-pinned comparison completed 61,682 assertions with no failures and measured zero allocator activity in C, generated Nelua, and Odin across portable and native-tuned builds. Odin portable led complete valid IPv4/UDP classification on that worker; Nelua portable led the hot, alignment, and malformed scan rows. Aggressive tuning regressed several hot paths.
Explore the charts, allocation proof, code sizes, exact workload rows, and XDP artifact boundary →
Be precise against JWS and COSE
| Capability | SDB1 | JWS Compact | COSE_Sign1 | COSE_Mac0 |
|---|---|---|---|---|
| Raw payload path | Native | Base64URL default; unencoded option | CBOR byte string or detached | CBOR byte string or detached |
| Detached payload | Yes | Yes | Yes | Yes |
| Separate external AAD | Yes | No general input | Yes | Yes |
| HMAC-SHA-256 | Native ID | Standard profile | Not a signature algorithm | Standard MAC algorithm |
| Ed25519 | Native ID | EdDSA profile | Standard signature algorithm | Not a MAC algorithm |
| Critical extensions | Yes | Yes | Yes | Yes |
| Self-description | Exact critical type + schema | Protected headers + profile | Protected headers + profile | Protected headers + profile |
| Bounded fixed prefix | 64 bytes | No | Implementation-dependent CBOR | Implementation-dependent CBOR |
| Standard ecosystem | No | Broad | Broad | Broad |
| Multi-signature / countersignature | No | Other JWS serialization | Other COSE structures | Other COSE structures |
| Confidentiality | OpenPGP outer composition | Use JWE | Use COSE encryption | Use COSE encryption |
SDB1 is better shaped for a bounded parse and borrowed hot path. Ed25519 corresponds to the signature-oriented COSE_Sign1 structure; HMAC-SHA-256 corresponds to the MAC-oriented COSE_Mac0 structure. Both COSE structures support external AAD. SDB1 does not match their standards status, ecosystem, countersignature, or encryption families.
What must happen before a durable release
- Freeze a version and extension/algorithm registry policy.
- Publish packages and stable test vectors.
- Obtain external security review and a genuinely independent implementation outside this project.
- Extend the structural/allocation benchmark to real crypto, AF_XDP/NIC paths, and well-configured JWS/COSE libraries.
- Keep the older
sd1text profile explicit; migration always verifies and re-signs.
The live workbench on the home page is intentionally the legacy sd1 lab. It does not demonstrate SDB1's raw binary path.