# Webhooks and integration events: authenticity, tenant binding and replay

Verify the event signature, provider-to-tenant binding and durable worker effect together across retries and out-of-order delivery.

## Investigation sequence

1. Capture genuine event
2. Verify signed bytes
3. Bind provider and tenant
4. Check replay and order
5. Inspect durable effect
6. Revoke and clean up

## Used in these playbooks

[Web & API](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/web-api.html) · [OSS & self-hosted](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/oss.html) · [Cloud & containers](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/cloud.html) · [AI, agents & MCP](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/ai.html)

### Setup and invariant

Treat an integration event as a separate privileged entry point. Inventory the subscription, receiving route, signing/authentication scheme, provider account, installation, tenant mapping, accepted event types, queue and final consumer. Use a local replica, a provider sandbox or local provider stub, two controlled tenants, and a reversible nonfinancial action. The invariant is: **only an authentic, correctly bound event may cause its permitted effect, once under the workflow's rules**.

Keep inbound event authenticity separate from [outbound destination checks](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/modules/ssrf-boundary.html) and [object authorization](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/modules/authz-matrix.html). A valid signature does not itself identify which local tenant may be changed.

### Numbered test procedure

1. **Capture a genuine baseline.** Generate one ordinary event through the controlled integration. Save exact body bytes, verification headers, event/object IDs, provider account and mode, local tenant, queue ID and final state. Write down the expected change, such as granting one sandbox entitlement. Capture a correctly signed irrelevant event as a no-effect control. An HTTP `2xx` can acknowledge receipt before the worker runs.
2. **Test authenticity before side effects.** In the replica, compare the baseline with a missing signature, an invalid signature, and a one-byte body change retaining the original signature. Restore the local object, queue and deduplication fixture between authenticity trials: an already-consumed event can conceal a missing verifier. Follow both receiver and worker logs; rejection after a privileged item was accepted is too late. Respect the actual verification scheme. GitHub's documented HMAC covers payload bytes, so JSON reserialization changes the signed bytes; separately supplied event/delivery headers are not automatically covered. Prefer the official verifier and confirm failures stop processing. [GitHub verification](https://docs.github.com/en/webhooks/using-webhooks/validating-webhook-deliveries).
3. **Bind provider identity to the local object.** Use valid sandbox events from two controlled provider accounts/installations. Verify each maps through trusted configuration to its correct tenant and provider object. For a local mutation experiment, change one untrusted routing field or unsigned header; a signed-field mutation requires an appropriately signed stub event and is a source-test hypothesis until its real attacker path is established. Check test/live mode and disconnected integrations. Do not claim a production forgery merely because the tester controls the lab signing key.
4. **Separate freshness, duplicates and order.** Replay one valid event once and inspect durable effects. If the provider signs a timestamp, check the documented acceptance window using a local clock/fixture. A fresh signature is not a deduplication guarantee. Deliver two distinct legitimate events in reverse order in the sandbox and verify that an old update cannot undo a later revocation. Stripe documents duplicate delivery and does not guarantee event order; event creation seconds are not a safe total ordering or deduplication key. Use event identity and the product's object-state rules. [Stripe webhook delivery behavior](https://docs.stripe.com/webhooks).
5. **Check dispatch and lifecycle.** Compare the allowed event type/action with a genuine unsupported type and malformed-but-authentic local fixture. Ensure a loosely selected handler does not treat every event as success. Rotate a sandbox signing secret, record the intentional overlap, then check the old secret after that overlap ends. Remove the integration and test the documented handling of already queued events. GitHub recommends checking event type/action and keeps the delivery ID unchanged on redelivery. Treat any deduplication based on an unsigned header according to the actual authenticated binding. [GitHub best practices](https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks).
6. **Inspect the transaction boundary.** In the replica only, introduce one controlled worker failure before or after the effect, then retry. Check whether deduplication and the protected update can disagree, causing lost processing or duplicate effects. Use the existing [state/replay procedure](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/modules/state-business-logic.html) for transactional controls. Restore fixtures, cancel queued test deliveries, and remove local capture files when evidence has been redacted.

### Command card: signed-body tamper control

Prepare `event-valid.json` and `event-valid.headers` from a newly generated local/sandbox delivery. The headers file contains only required verification and event metadata, not copied Host, Content-Length or production cookies. Prepare `event-tampered.json` by changing one known marker byte without re-signing. Start each authenticity case from the same clean local snapshot, including queue and deduplication state. Use a nonfinancial event and remain inside any signature freshness window.

```bash
mkdir -p evidence/webhooks
curl -q --noproxy '*' --max-time 10 \
  --header @event-valid.headers -H 'Content-Type: application/json' \
  --data-binary @event-valid.json \
  -D evidence/webhooks/valid.h -o evidence/webhooks/valid.txt \
  http://127.0.0.1:8000/webhooks/lab
# Inspect the valid effect, then restore the local fixture before this case.
curl -q --noproxy '*' --max-time 10 \
  --header @event-valid.headers -H 'Content-Type: application/json' \
  --data-binary @event-tampered.json \
  -D evidence/webhooks/tampered.h -o evidence/webhooks/tampered.txt \
  http://127.0.0.1:8000/webhooks/lab
```

For the separate replay experiment, restore once, send the valid request, wait for completion, and repeat it once without resetting. Inspect the durable effect count; two acknowledgements can correctly refer to one effect. The endpoint is illustrative. `--data-binary` preserves file bytes; these requests do not follow redirects. [curl manual](https://curl.se/docs/manpage.html).

### Report threshold and checklist

Report a forged protected transition, authentic event applied to the wrong controlled tenant, or replay/order failure with a durable security consequence. Acceptance of an intentionally unsigned public notification, permitted secret-rotation overlap, a harmless no-op acknowledgement, or a mock-only signing capability is not enough.

- [ ] Exact authenticated bytes and unsigned metadata distinguished.
- [ ] Provider identity, tenant, object and mode binding verified.
- [ ] Invalid/tampered controls caused no privileged worker effect.
- [ ] Freshness, duplicate delivery and order tested separately.
- [ ] Final state, retry behavior and cleanup recorded.


## Vulnerability reverse references

- [Workflow order or approval bypass](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/vulnerabilities/workflow-order-validation-bypass.html) — An actor can complete a protected workflow without its required prior approval or immutable conditions.
- [Single-use action or idempotency replay failure](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/vulnerabilities/single-use-idempotency-replay.html) — A replay creates another protected effect where the operation must be unique or idempotent.
- [Unverified or tampered webhook causes a protected effect](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/vulnerabilities/webhook-event-authenticity-failure.html) — An event is processed with privileged consequences without establishing the required sender and content authenticity.
- [Authentic integration event applied to the wrong tenant or object](https://bug-bounty-field-manual.kmahshooq.chatgpt.site/vulnerabilities/webhook-provider-tenant-binding.html) — A valid provider event is misbound to a local tenant, installation, mode or resource the event is not authorized to modify.