Overflow control for JSON events

Keep one oversized event from becoming a database incident.

Spill explicitly allowed fields into encrypted S3-compatible storage. Leave a signed, readable stub your operators can restore in two clicks.

Test a payload

Zero runtime dependencies
AES-256-GCM
MinIO + S3

Technical cutaway showing a large coral JSON block diverted from a conveyor into a locked teal storage reservoir. — Plate 01 Oversized fields leave the hot path; the event keeps its shape.

Local test bench / no uploads

Run a spill decision in your browser.

Keys and encrypted objects live only in memory and disappear on refresh.

Spill ledgerWaiting

No decision yet

Run the sample event. The allowed oversized field will become a signed stub.

Incoming0 B

Database0 B

Avoided0%

Database stub

signed / readable

One narrow job

The event stays useful. The weight moves elsewhere.

01
Measure allowed fields
Exact JSON pointers are checked. Unlisted fields—even large ones—never leave the event.
02
Encrypt, then store
AES-GCM protects the value before an S3 or MinIO write. A failed write rolls the event back.
03
Leave an audit trail
The signed stub records size, digest, field, safe preview, expiry, and retrieval URL.

API / v0.1.0

Four decisions, one small surface.

Choose the fields, inline limit, object store, and retention window. Spillway handles encryption, signatures, atomic writes, and safe previews.

Typed ESM + CommonJS
No telemetry or runtime dependencies
Standards-based Request, fetch, and Web Crypto

npm install event-payload-spillway

const spillway = new Spillway({
  allowlist: ["/result/attachment"],
  maxInlineBytes: 32_000,
  store: new S3CompatibleStore(s3),
  encryptionKey,
  signingKey,
  expiresInMs: 604_800_000,
});

const handle = createSpillwayProxy({
  spillway,
  upstream: "http://automation:3000/events",
  verifyWebhook: ({ rawBody, headers }) =>
    verify(rawBody, headers),
});

Operational contract

Expiry is visible. Deletion is deliberate.

Report first

Count encrypted and expired bytes before changing anything. Reports inspect the configured storage prefix.

retentionReport()

Dry-run reclaim

Reclaim defaults to expired-only. Preview candidate count and size before deleting objects.

reclaim({ dryRun: true })

Respect holds

Objects marked with a legal hold are excluded. Bucket lifecycle policy remains your final backstop.

spill(data, { legalHold: true })

Protect the hot path

Give oversized events somewhere safe to go.

Read the source