Skip to main content

Recovering a Broken Webhook Endpoint

When your endpoint has had downtime — a deploy that broke verification for an hour, a database outage, a cert that expired silently — follow this six-step runbook to find what you missed, replay what's recoverable, and reconcile the rest by reading current state.

Step 1 — Is the subscription still active?

GET /openapi/v1/webhooks/{webhookId}

Read data.attributes.delivery.status:

ValueMeaningAction
ACTIVELive, deliveringSkip to Step 3
DISABLEDYou paused itRe-enable (Step 2)
AUTO_DISABLEDOho disabled it — ~50 consecutive failuresFix the endpoint, then re-enable (Step 2)

AUTO_DISABLED is the only signal that Oho stopped delivering on you. There's no failure-count field exposed — status is the flag.

Step 2 — Confirm reachable, then re-enable

Only if DISABLED / AUTO_DISABLED. Test first so you don't immediately trip the threshold again:

POST /openapi/v1/webhooks/{webhookId}/ping
→ { "delivered": true, "statusCode": 200, ... }

If delivered: true, re-enable:

POST /openapi/v1/webhooks/{webhookId}/enable
→ data.attributes.delivery.status == "ACTIVE"

Live delivery resumes from here. Past misses are recovered in Steps 3–5.

Step 3 — Find when you last received a good event

There's no stored "last success" field — compute it from history. List deliveries (returned newest-first) filtered to successes:

GET /openapi/v1/webhooks/{webhookId}/deliveries?outcome=DELIVERED&limit=1

Take data[0].timestampMillis → that's your last-good watermark T_last.

If the list is empty, you've never received one — use your own last-known-good time or skip straight to the full reconcile in Step 6.

Step 4 — List what failed since the watermark

GET /openapi/v1/webhooks/{webhookId}/deliveries
?outcome=EXHAUSTED
&startTimeMillis={T_last}
&endTimeMillis={now}
&limit=1000

Each row (DeliveryDto) carries: deliveryId, eventType, attempt, outcome, statusCode, timestampMillis, emittedAt, errorMessage, payloadTruncated.

  • Note any rows with payloadTruncated: true — those can't be replayed; Step 6 handles them.
  • Repeat with outcome=FAILED_PERMANENT if you also want events your endpoint 4xx-rejected.

Step 5 — Replay (redrive)

Re-send the stored originals for that window:

POST /openapi/v1/webhooks/{webhookId}/redrive
{
"startTimeMillis": {T_last},
"endTimeMillis": {now},
"outcomes": ["EXHAUSTED", "FAILED_RETRYABLE"]
}

The response tells you the split:

{
"matched": 120,
"dispatched": 118,
"skippedTruncated": 1,
"skippedNoPayload": 1,
"deliveryIds": ["..."]
}

Replays arrive at your endpoint as normal signed deliveries. Verify the signature + dedupe on X-Oho-Delivery — the redriven event re-uses its original delivery id.

Step 6 — Reconcile the un-replayable remainder

For skippedTruncated + skippedNoPayload (and as a belt-and-braces sweep), read current state instead. Convert T_last (ms) to ISO-8601:

GET /openapi/v1/credentials?updatedAfter={T_last as ISO}&pageSize=100&sort=lastUpdated:desc

Page through; for each credential, overwrite your local copy from attributes.verification.eligibility / .statusDetail / .expiryDate. This closes any gap replay couldn't.

Where to go next