Webhooks

@hellmai/lumenflow-surfaces’s http/webhooks/ module ships signed outbound webhooks with durable delivery history and idempotent replay, so a service account can verify a delivery and safely replay one it missed during an outage.

Signing and verification

Every outbound event is signed with HMAC-SHA256 over a canonical (key-sorted) JSON encoding of its payload:

import {
  createWebhookKeyStore,
  signWebhookEvent,
  formatWebhookSignatureHeader,
  verifyWebhookSignature,
} from '@hellmai/lumenflow-surfaces/http/webhooks/signing.js';

const keyStore = createWebhookKeyStore({ keyId: 'k1', secret: 'shared-secret', createdAt: now });

const header = signWebhookEvent(payload, keyStore.getSigningKey());
const headerValue = formatWebhookSignatureHeader(header); // "t=...,kid=...,v1=..."

verifyWebhookSignature(payload, headerValue, 'shared-secret'); // true

Verification needs only the raw shared secret — never the mutable key store — so a consumer can verify a delivery without holding any server-side mutable state. Signature comparison uses timingSafeEqual, and verification fails closed (returns false, never throws) on malformed input, since this is a trust boundary.

Rotation and revocation

keyStore.rotateKey({ keyId: 'k2', secret: 'new-secret', createdAt: now });
keyStore.revokeKey('k1');

Every sign/verify lookup reads the store at call time, never a value captured once at startup — rotating in a new key or revoking an old one takes effect on the very next call, with no redeploy or process restart required.

Signed delivery to a registered endpoint

One verifier and one header family cover every signed delivery to an endpoint a consumer registers (job callbacks, synced data, store status, bounces, app health). The module is @hellmai/lumenflow-surfaces/http/webhooks/signed-delivery.js (also re-exported from signing.js); the wire contract is version 1.

Header	Value
X-Lumenflow-Timestamp	Unix seconds
X-Lumenflow-Delivery-Id	Opaque id, unique per delivery ([A-Za-z0-9][A-Za-z0-9._:-]*)
X-Lumenflow-Signature	t=<unix seconds>,v1=<lowercase hex HMAC-SHA256 of "t.body">

The signed input is the decimal timestamp, a dot, then the raw body bytes, exactly as received. No header, field or event name carries a product name.

Verify (TypeScript)

import {
  createInMemoryReplayStore,
  verifyDelivery,
  RECEIVER_RESPONSES,
} from '@hellmai/lumenflow-surfaces/http/webhooks/signed-delivery.js';

const replayStore = createInMemoryReplayStore(); // one per process; use a shared store when scaled out

// rawBody must be the bytes as received, before any JSON middleware touches them.
const result = verifyDelivery(rawBody, request.headers, secret, {
  replayStore,
  replayWindowSeconds: 300, // default
});

if (result.outcome !== 'ok') {
  const { status, code } = RECEIVER_RESPONSES[result.outcome];
  return respond(status, { code });
}
handle(result.body); // strict JSON object, no duplicate keys

verifyDelivery never throws on invalid input and returns one typed outcome: ok, bad_signature, stale (timestamp further than the window from now, in either direction), replayed (delivery id already seen inside the window) or malformed (missing, duplicated or non-canonical headers, unknown signature fields, a body that is not a strict JSON object, or a body over the size limit). The checks run in a fixed order: headers, signature, window, body, replay; the replay store is only written after the signature verified, so a request with an invalid signature cannot write to it. The comparison is constant time, and an unknown anything is refused.

Verify (Python)

import hashlib, hmac, re, time

SIG = re.compile(r"^t=(0|[1-9]\d{0,14}),v1=([0-9a-f]{64})$")
seen: dict[str, float] = {}  # delivery id -> expiry; use a shared store when scaled out

def verify(raw_body: bytes, headers: dict[str, str], secret: bytes, window: int = 300):
    ts = headers.get("X-Lumenflow-Timestamp", "")
    delivery_id = headers.get("X-Lumenflow-Delivery-Id", "")
    m = SIG.match(headers.get("X-Lumenflow-Signature", ""))
    if not m or m.group(1) != ts or not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._:-]{0,127}", delivery_id):
        return "malformed"
    expected = hmac.new(secret, ts.encode() + b"." + raw_body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, m.group(2)):
        return "bad_signature"
    now = time.time()
    if abs(now - int(ts)) > window:
        return "stale"
    # parse raw_body as a strict JSON object here (reject duplicate keys); "malformed" on failure
    if seen.get(delivery_id, 0) > now:
        return "replayed"
    seen[delivery_id] = now + window
    return "ok"

Golden vectors

signed-delivery-vectors.js exports SIGNED_DELIVERY_VECTORS: a fixed secret, raw body, three headers and verifier clock per case, with the expected typed outcome (ok, bad_signature, stale, replayed, malformed). Run them in order against one verifier with one replay store. A drift test pins a digest of the vectors per SIGNED_DELIVERY_CONTRACT_VERSION, so the bytes cannot change without a version bump. Implement the verifier in any language and check it against these bytes.

Replies and retries

Receiver outcome	Status	Sender treats it as
ok	200 (any 2xx)	acknowledged
replayed	409 replayed	acknowledged
superseded epoch (stale_epoch)	409 stale_epoch	acknowledged
malformed	400	rejected
bad_signature, stale	401	rejected
unknown endpoint / oversize body	404 / 413	rejected
overloaded or unavailable	408, 429, 503, 5xx	retried

classifyReceiverStatus(status, retryAfter) returns the sender’s disposition and honours Retry-After (delta-seconds or HTTP date) on retried statuses. A status outside the table is rejected, never retried blindly.

Epoch ordering

For deliveries that carry a monotonic per-subject epoch, createEpochOrdering() returns applyIfNewer(subject, epoch, apply): a strictly newer epoch runs apply once, an equal or older epoch returns superseded (reply 409 stale_epoch, acknowledged, never applied). Calls for one subject are serialised, so concurrent deliveries cannot both apply or apply out of order. A failed apply does not record the epoch, so the retry can apply. Pass an EpochStore to persist the last applied epoch.

Moving from an older header family

Receivers and senders move without a flag day through the one-release transition option. signDelivery({ ..., transition }) emits the old header family next to the new one, and verifyDelivery(..., { transition }) accepts either family; when both are present both must verify and agree on timestamp and delivery id. The caller supplies the old header names. The option is removed on 2027-01-31 (DUAL_HEADER_TRANSITION_REMOVAL_DATE); after that date only the X-Lumenflow-* family is read and written.

Durable delivery history

Delivery history is the one durable outbox for signed webhooks: signing produces the signature, this store is where the resulting delivery is persisted and queried, and replay re-drains from this same store rather than standing up a second queue.

import { createDurableWebhookDeliveryHistoryStore } from '@hellmai/lumenflow-surfaces/http/webhooks/delivery-history.js';

const store = createDurableWebhookDeliveryHistoryStore({
  filePath: 'webhooks/delivery-history.ndjson',
});

const record = store.record({
  eventId: 'evt-1',
  endpointId: 'ep-1',
  kind: 'wu:completed',
  payload,
  signatureHeader: headerValue,
});
store.appendAttempt({ eventId: 'evt-1', outcome: 'success', responseStatus: 200 });

store.listByEndpoint('ep-1');
store.listFailedAttempts('ep-1');

Storage is append-only NDJSON — the same durable-outbox convention this repo already uses elsewhere — so the store survives a process restart: reopening it against the same file rebuilds identical in-memory state by replaying the log from the start. status (pending / in_flight / acked / abandoned) reuses the same ReplayArtifactStatus vocabulary @hellmai/lumenflow-control-plane-sdk already publishes, rather than a parallel status enum.

Idempotent replay

import {
  replayMissedWebhookDelivery,
  listReplayableWebhookDeliveries,
} from '@hellmai/lumenflow-surfaces/http/webhooks/replay.js';

const outstanding = listReplayableWebhookDeliveries(store, 'ep-1'); // not yet acked
await replayMissedWebhookDelivery({ eventId: 'evt-1', store, deliverer });

Replay never regenerates a payload, signature, or event id: it re-drains the same durable record and hands it, byte for byte, to the injected WebhookDeliverer. Because the event’s identity never changes across attempts, a consumer that dedupes by eventId sees a replay as a no-op retry of an event it may already have processed — exactly the at-least-once, idempotent-consumer contract this repo’s other outbox uses. The actual network call is injected via WebhookDeliverer (a port); this module owns replay policy, not HTTP transport.

Related

• Contract spine — the error taxonomy webhook error classes map onto.