MailHub Delivery Webhooks Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Implement signed HTTPS webhooks for terminal email events (sent / bounced / failed) with account default + domain override, multi-URL subscriptions, and an observable SQLite delivery queue (claim, retry, logs, replay).

Architecture: Pure helpers in webhook-model.js; CRUD + enqueue in db.js; HTTP worker in webhook-dispatcher.js started from server.js; REST under /api/webhooks and /api/webhook-deliveries; React pages replace Placeholder and domain Webhooks tab. Secrets use existing encryptSecret / decryptSecret.

Tech Stack: Node.js ESM, SQLite (node:sqlite), React + Ant Design admin UI, node:test

Spec: docs/superpowers/specs/2026-07-09-mailhub-webhooks-design.md

File map

File	Responsibility
Create `src/webhook-model.js`	Event map, resolve targets, payload, HMAC sign, backoff, URL precheck helpers (pure)
Create `src/webhook-dispatcher.js`	Worker loop, claim, SSRF-safe fetch, status updates
Create `test/webhook-model.test.js`	Pure model unit tests
Create `test/webhook-dispatcher.test.js`	Dispatcher with mocked fetch
Modify `src/db.js`	Tables, webhook CRUD, enqueue, claim, replay, list deliveries
Modify `src/server.js`	API routes; start worker; ensure enqueue after terminal writes if not fully inside db
Create `src/pages/Webhooks.tsx`	Global webhooks + deliveries UI
Modify `src/frontend/App.tsx`	Wire Webhooks page, load webhooks data
Modify `src/pages/Domains/DomainDetail.tsx`	Real Webhooks tab
Modify `src/frontend/types.ts`	Webhook / delivery types
Modify `src/frontend/services/api.ts`	Client methods
Modify `src/frontend/i18n/index.js`	zh/en strings
Modify `test/frontend-i18n.test.js`	New keys if asserted
Create `test/server-webhooks-api.test.js`	API auth + CRUD smoke (follow `server-admin-api.test.js` patterns)

Task 1: Pure webhook model

Files:

Create: src/webhook-model.js
Create: test/webhook-model.test.js

[ ] Step 1: Write failing tests

// test/webhook-model.test.js
import assert from 'node:assert/strict';
import { test } from 'node:test';
import {
TERMINAL_WEBHOOK_EVENTS,
eventTypeForStatus,
resolveWebhooksForEvent,
buildWebhookPayload,
signWebhookBody,
nextBackoffMs,
isTerminalWebhookStatus
} from '../src/webhook-model.js';

test('maps terminal statuses to email.* types', () => {
assert.equal(eventTypeForStatus('sent'), 'email.sent');
assert.equal(eventTypeForStatus('bounced'), 'email.bounced');
assert.equal(eventTypeForStatus('failed'), 'email.failed');
assert.equal(eventTypeForStatus('queued'), null);
});

test('domain webhooks override account for the same event', () => {
const account = [
{ id: 1, domainId: null, enabled: true, events: ['sent', 'failed'] },
{ id: 2, domainId: null, enabled: true, events: ['bounced'] }
];
const domain = [
{ id: 3, domainId: 9, enabled: true, events: ['sent'] }
];
const resolved = resolveWebhooksForEvent({
accountWebhooks: account,
domainWebhooks: domain,
eventType: 'sent'
});
assert.deepEqual(resolved.map((w) => w.id), [3]);
});

test('falls back to account when domain has no matching enabled subscription', () => {
const resolved = resolveWebhooksForEvent({
accountWebhooks: [{ id: 1, domainId: null, enabled: true, events: ['failed'] }],
domainWebhooks: [{ id: 3, domainId: 9, enabled: true, events: ['sent'] }],
eventType: 'failed'
});
assert.deepEqual(resolved.map((w) => w.id), [1]);
});

test('signs body with Stripe-style t and v1', () => {
const body = '{"id":"whd_1"}';
const header = signWebhookBody(body, 'secret', 1_700_000_000);
assert.equal(header.startsWith('t=1700000000,v1='), true);
assert.match(header, /^t=\d+,v1=[0-9a-f]{64}$/);
});

test('backoff grows then caps', () => {
assert.ok(nextBackoffMs(1) < nextBackoffMs(2));
assert.equal(nextBackoffMs(10), nextBackoffMs(20));
});

[ ] Step 2: Run tests — expect FAIL

Run: node --test test/webhook-model.test.js

Step 3: Implement src/webhook-model.js

Export at least:

export const TERMINAL_WEBHOOK_EVENTS = ['sent', 'bounced', 'failed'];
export const MAX_WEBHOOK_ATTEMPTS = 8;
export const WEBHOOK_LEASE_MS = 2 * 60 * 1000;

export function isTerminalWebhookStatus(status) {
  return TERMINAL_WEBHOOK_EVENTS.includes(status);
}

export function eventTypeForStatus(status) {
  if (status === 'sent') return 'email.sent';
  if (status === 'bounced') return 'email.bounced';
  if (status === 'failed') return 'email.failed';
  return null;
}

/** @param {{ accountWebhooks: any[]; domainWebhooks: any[]; eventType: string }} input */
export function resolveWebhooksForEvent({ accountWebhooks, domainWebhooks, eventType }) {
  const matches = (list) => (list || []).filter(
    (w) => w.enabled !== false && w.enabled !== 'false' && Array.isArray(w.events) && w.events.includes(eventType)
  );
  const domainHits = matches(domainWebhooks);
  if (domainHits.length) return domainHits;
  return matches(accountWebhooks);
}

export function buildWebhookPayload({ deliveryId, eventType, createdAt, sendEvent, test = false }) {
  const status = sendEvent.status;
  return {
    id: `whd_${deliveryId}`,
    type: eventTypeForStatus(status) || eventType,
    created_at: createdAt,
    data: {
      ...(test ? { test: true } : {}),
      message_id: test ? 'mh-test' : `mh-${sendEvent.id}`,
      send_event_id: sendEvent.id,
      queue_id: sendEvent.queueId || '',
      status,
      domain: sendEvent.domain || '',
      from: sendEvent.sender || '',
      to: sendEvent.recipients || [],
      subject: sendEvent.subject || '',
      detail: sendEvent.detail || '',
      delivered_at: sendEvent.deliveredAt || null
    }
  };
}

export function signWebhookBody(rawBody, secret, unixSeconds = Math.floor(Date.now() / 1000)) {
  const crypto = awaitImportOrRequireCrypto(); // use import crypto from 'node:crypto' at top
  const signed = `${unixSeconds}.${rawBody}`;
  const v1 = crypto.createHmac('sha256', secret).update(signed).digest('hex');
  return `t=${unixSeconds},v1=${v1}`;
}

/** attemptCount after increment for failed path; attempt 1 → 60s, … cap 12h */
export function nextBackoffMs(attemptCount) {
  const table = [60_000, 300_000, 1_800_000, 7_200_000, 21_600_000, 43_200_000];
  const index = Math.max(0, Math.min(table.length - 1, attemptCount - 1));
  return table[index];
}

Use top-level import crypto from 'node:crypto'. Fix any pseudocode above to real ESM.

Also export parseWebhookEventsJson, normalizeWebhookEvents(input) validating non-empty subset of three events.

Step 4: Run tests — PASS

Run: node --test test/webhook-model.test.js

[ ] Step 5: Commit

git add src/webhook-model.js test/webhook-model.test.js
git commit -m "feat(webhooks): add pure model for events, resolve, sign, backoff"

Task 2: Schema + DB CRUD + enqueue

Files:

Modify: src/db.js
Create: test/webhook-db.test.js (or extend test/db.test.js if preferred — prefer dedicated file)

[ ] Step 1: Failing tests for isolation and enqueue idempotency

// test/webhook-db.test.js — follow openTempDb / createUser patterns from test/db.test.js
test('creates webhooks per user and lists by domain scope', () => { /* ... */ });
test('enqueueWebhookDeliveries is idempotent per webhook+event+send_event', () => { /* ... */ });
test('domain override skips account webhooks for that event', () => { /* ... */ });
test('secret not returned on list; create returns plaintext once', () => { /* ... */ });

[ ] Step 2: Run — FAIL

[ ] Step 3: Schema in initDb / migrations block

CREATE TABLE IF NOT EXISTS webhooks (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
domain_id INTEGER,
name TEXT NOT NULL,
url TEXT NOT NULL,
secret_ciphertext TEXT NOT NULL,
secret_prefix TEXT NOT NULL,
events_json TEXT NOT NULL,
enabled TEXT NOT NULL DEFAULT 'true',
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
FOREIGN KEY(user_id) REFERENCES users(id) ON DELETE CASCADE
);

CREATE TABLE IF NOT EXISTS webhook_deliveries (
id INTEGER PRIMARY KEY AUTOINCREMENT,
webhook_id INTEGER NOT NULL,
user_id INTEGER NOT NULL,
send_event_id INTEGER NOT NULL,
event_type TEXT NOT NULL,
payload_json TEXT NOT NULL,
status TEXT NOT NULL,
attempt_count INTEGER NOT NULL DEFAULT 0,
next_attempt_at TEXT NOT NULL,
last_attempt_at TEXT,
response_status INTEGER,
response_body_preview TEXT NOT NULL DEFAULT '',
error TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL,
FOREIGN KEY(webhook_id) REFERENCES webhooks(id) ON DELETE CASCADE,
UNIQUE(webhook_id, send_event_id, event_type)
);

CREATE INDEX IF NOT EXISTS idx_webhooks_user_id ON webhooks(user_id);
CREATE INDEX IF NOT EXISTS idx_webhooks_user_domain ON webhooks(user_id, domain_id);
CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_status_next ON webhook_deliveries(status, next_attempt_at);
CREATE INDEX IF NOT EXISTS idx_webhook_deliveries_user_created ON webhook_deliveries(user_id, created_at);

No FK on send_event_id (test sentinel 0).

[ ] Step 4: Implement API functions
listWebhooks(userId, { domainId }?)
createWebhook(userId, { name, url, events, domainId, enabled }) → returns row + secret plaintext once; store encryptSecret(secret), prefix first 8 chars
updateWebhook(userId, id, patch)
rotateWebhookSecret(userId, id) → new secret once
deleteWebhook(userId, id)
enqueueWebhookDeliveries(sendEvent) — see Enqueue contract below.
claimWebhookDeliveries(limit) — transaction: select pending due, set processing + lease next_attempt_at; return claimed rows already joined with webhook url and decrypted secret (decrypt only inside db.js; do not export encryptSecret/decryptSecret). Prefer shape: { delivery, webhook: { id, url, secret } }.
getWebhookDeliveryTarget(deliveryId) optional alternative if claim returns thin rows — still decrypt only in db.js.
reapExpiredWebhookProcessing()
completeWebhookDeliverySuccess(id, { responseStatus, bodyPreview }) — set attempt_count = attempt_count + 1, status success
completeWebhookDeliveryFailure(id, { responseStatus, bodyPreview, error }) — set attempt_count = attempt_count + 1; if attempt_count < 8 then status pending + next_attempt_at = now + nextBackoffMs(attempt_count); else dead
listWebhookDeliveries(userId, filters)
replayWebhookDelivery(userId, id) — reject if processing
enqueueWebhookTestDelivery(userId, webhookId) — send_event_id 0, reuse unique key via reset-to-pending

Enqueue contract

Canonical input (all fields preferred; helper may fill gaps):

{
  id,              // send_events.id (required)
  userId,          // required; no-op if null/undefined
  domainId,        // may be null
  domain,          // optional display name; if missing, JOIN domains by domainId
  status,          // sent|bounced|failed
  sender,
  recipients,      // array
  subject,
  detail,
  queueId,
  deliveredAt
}

Idempotent insert (critical): For each resolved webhook, do not blindly INSERT OR IGNORE then UPDATE by lastInsertRowid().

Preferred pattern inside a transaction per webhook (or one transaction for all):

SELECT id FROM webhook_deliveries WHERE webhook_id=? AND send_event_id=? AND event_type=?
If row exists → skip (leave payload/status untouched)
Else INSERT with placeholder payload_json='{}', read lastInsertRowid() only when result.changes === 1
UPDATE that id with final payload_json from buildWebhookPayload in the same transaction

Alternatively: try INSERT final payload in one statement using a pre-allocated id strategy only if SQLite version allows; simplest is insert+update on new rows only (changes === 1).

Wire points

logSendEvent: after successful insert, if terminal status, call enqueueWebhookDeliveries({ ...event, id: lastInsertRowid, recipients: normalized array, domain: resolved name }) inside try/catch (log errors; never throw to caller).
updateSendEventDelivery: when nextStatus is terminal and nextStatus !== row.status, load user_id/domain/recipients from row, resolve domain name, enqueue with new status/detail/deliveredAt; try/catch.

Keep enqueue try/catch logged so webhook failure never breaks mail path.

[ ] Step 5: Tests PASS + commit

git add src/db.js test/webhook-db.test.js
git commit -m "feat(webhooks): add schema, CRUD, and terminal enqueue hooks"

Task 3: Dispatcher worker + SSRF

Files:

Create: src/webhook-dispatcher.js
Create: test/webhook-dispatcher.test.js
Modify: src/server.js (start/stop worker on listen)

[ ] Step 1: Tests with injectable fetch and clock

test('posts signed body and marks success on 2xx', async () => { /* mock fetch ok */ });
test('schedules retry on 500', async () => { /* */ });
test('marks dead after max attempts', async () => { /* */ });
test('rejects private IP targets without calling fetch', async () => { /* */ });

[ ] Step 2: Implement dispatcher

// Core loop
export function startWebhookWorker({ intervalMs = 10_000, batchSize = 3, fetchImpl = fetch } = {}) { ... }
export function stopWebhookWorker() { ... }
export async function processWebhookBatch({ fetchImpl, batchSize } = {}) {
reapExpiredWebhookProcessing();
const rows = claimWebhookDeliveries(batchSize);
for (const row of rows) {
await deliverOne(row, fetchImpl);
}
}

deliverOne (row already includes url + decrypted secret from db claim/join):

If url/secret missing → complete as dead/error
Validate URL (https; optional WEBHOOK_ALLOW_HTTP_LOCAL only for loopback HTTP)
DNS lookup / block private IPv4 and IPv6 ULA/link-local/loopback (fail closed); never call fetch on blocked targets
fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json', 'User-Agent': 'MailHub-Webhook/1.0', 'X-MailHub-Signature': sign..., 'X-MailHub-Event': type, 'X-MailHub-Delivery': id }, body: payload_json, signal: AbortSignal.timeout(10_000), redirect: 'manual' })
Success / failure complete helpers (each finished HTTP attempt increments attempt_count once)

[ ] Step 3: Start worker at module bootstrap (same pattern as startDnsAutoChecker / delivery tracker — not only inside listen). Optionally skip when WEBHOOK_WORKER_ENABLED=0 for tests.

[ ] Step 4: Tests PASS + commit

git add src/webhook-dispatcher.js test/webhook-dispatcher.test.js src/server.js
git commit -m "feat(webhooks): add delivery worker with claim and SSRF guards"

Task 4: REST API

Files:

Modify: src/server.js
Create: test/server-webhooks-api.test.js
[ ] Step 1: Tests — session auth required; user A cannot see user B webhooks; create returns secret; list omits secret; test + replay endpoints

Follow patterns in test/server-admin-api.test.js (login cookie, temp data dir).

[ ] Step 2: Routes (session user required, same as other user APIs)

GET    /api/webhooks
POST   /api/webhooks
PATCH  /api/webhooks/:id
POST   /api/webhooks/:id/rotate-secret
DELETE /api/webhooks/:id
POST   /api/webhooks/:id/test
GET    /api/webhook-deliveries
POST   /api/webhook-deliveries/:id/replay

Validate body; map errors to 400.

[ ] Step 3: Tests PASS + commit

git add src/server.js test/server-webhooks-api.test.js
git commit -m "feat(webhooks): expose management and delivery APIs"

Task 5: Frontend API + types + i18n

Files:

Modify: src/frontend/types.ts
Modify: src/frontend/services/api.ts
Modify: src/frontend/i18n/index.js
Modify: test/frontend-i18n.test.js (if new keys asserted)

[ ] Step 1: Types

export type WebhookEvent = 'sent' | 'bounced' | 'failed';
export interface Webhook {
id: number;
userId: number;
domainId: number | null;
name: string;
url: string;
secretPrefix: string;
events: WebhookEvent[];
enabled: boolean;
createdAt: string;
updatedAt: string;
secret?: string; // only on create/rotate responses
}
export interface WebhookDelivery {
id: number;
webhookId: number;
sendEventId: number;
eventType: string;
status: 'pending' | 'processing' | 'success' | 'dead';
attemptCount: number;
// ...
}

[ ] Step 2: api.ts methods mirroring REST
[ ] Step 3: i18n keys for page chrome (zh-CN + en-US): titles, events labels, empty states, secret warning, replay, test

[ ] Step 4: Commit

git add src/frontend/types.ts src/frontend/services/api.ts src/frontend/i18n/index.js test/frontend-i18n.test.js
git commit -m "feat(webhooks): add frontend types, API client, and i18n"

Task 6: Global Webhooks page + App wiring

Files:

Create: src/pages/Webhooks.tsx
Modify: src/frontend/App.tsx
Optionally: small components under src/components/webhook/
[ ] Step 1: Build Webhooks.tsx with optional domainId?: number prop

Use PageHeader, SectionCard, StatusPill, Table, Drawer/Form, Modal for secret once, Popconfirm delete, CodeBlock for sample receiver docs.

When domainId is set: hide global page chrome / filter list to that domain; pre-fill create form scope.

Sections:

Endpoints table + create button (columns: name, scope, URL truncated, events, enabled, last delivery snippet)
Deliveries table (filter by webhook) + replay

Handlers call api.* and refresh list. Toggle enabled via PATCH.

[ ] Step 2: App.tsx
Import Webhooks page
Replace Placeholder for webhooks view
Load webhooks (and optionally deliveries) in loadAll or lazy on view enter
[ ] Step 3: npx tsc --noEmit PASS

[ ] Step 4: Commit

git add src/pages/Webhooks.tsx src/frontend/App.tsx src/components/webhook
git commit -m "feat(webhooks): add global webhooks management UI"

Task 7: Domain detail Webhooks tab

Files:

Modify: src/pages/Domains/DomainDetail.tsx
Possibly reuse list component with domainId prop from Task 6
[ ] Step 1: Replace Placeholder tab with filtered Webhooks panel (domainId={domain.id}) + help Alert about override semantics

[ ] Step 2: tsc PASS + commit

git add src/pages/Domains/DomainDetail.tsx src/pages/Webhooks.tsx
git commit -m "feat(webhooks): wire domain-scoped webhook overrides in detail tab"

Task 8: Full gate

Files: polish only if needed

Step 1: npm test — all pass
Step 2: npm run build — pass; commit built public/ assets if hashes change
Step 3: Manual checklist
- Create account webhook, copy secret
- Send test
- Force fail (bad URL) → pending retry fields
- Replay
- Domain override skips account for same event

[ ] Step 4: Final commit if assets/docs changed

git add -A
git commit -m "chore(webhooks): build assets after webhook feature"

Notes

Do not block mail send on webhook errors.
Never log full secrets.
Align field naming with existing API camelCase JSON helpers in db.js / server.
Follow encryptSecret usage from DNS credentials.
After implementation, use @superpowers:verification-before-completion before claiming done.
Deploy only after user requests per Agents.md.

Execution handoff

Plan complete. Choose:

Subagent-Driven (recommended) — fresh subagent per task + review
Inline Execution — this session with checkpoints

2026-07-09-mailhub-webhooks.md 20 KB Пермалинк Историја Датотека

MailHub Delivery Webhooks Implementation Plan

File map

Task 1: Pure webhook model

Task 2: Schema + DB CRUD + enqueue

Enqueue contract

Wire points

Task 3: Dispatcher worker + SSRF

Task 4: REST API

Task 5: Frontend API + types + i18n

Task 6: Global Webhooks page + App wiring

Task 7: Domain detail Webhooks tab

Task 8: Full gate

Notes

Execution handoff

2026-07-09-mailhub-webhooks.md 20 KB

Пермалинк Историја Датотека