Make.com reliability patterns: retries, idempotency & DLQ
Problem Context
Make.com scenarios are great for rapid automation, but moving integrations into production surfaces recurring reliability problems: platform retries triggering duplicate downstream writes, long-running or stuck executions, partial commits across multiple services, and noisy or missing alerts. Teams often see billing spikes, inconsistent downstream state, and time-consuming manual recovery. The core operational challenges to address are (1) how Make’s webhook and scenario retry semantics interact with external systems, (2) how to make processing deterministic and safe to replay, and (3) how to detect, quarantine and surface permanent failures without dropping customer data.
Implementation Workflow
The pattern below converts ephemeral Make executions into controlled, observable, replayable processing. It is intentionally implementation-focused so you can apply it to real Make.com scenarios that ingest webhooks or API events and call external systems (databases, CRMs, payment providers).
- Ingestion contract & event_id
Require a client-supplied event_id (UUID or natural composite key) included in the webhook payload. If the sender cannot provide that, generate a canonical processing_id at the first durable write and return it to the caller when feasible. Treat this id as the single source of truth for deduplication and tracing.
- Durable staging layer (staging table / queue)
Rather than executing side-effecting operations in the webhook-triggered scenario, write the raw payload + event_id + metadata to a durable staging store you control. Options: a relational DB table, an object store (S3) plus index, or a managed message queue. In Make, the webhook scenario should quickly ACK the incoming request (202) and persist the payload using HTTP/SQL modules or an external integration. This turns ephemeral Make executions into an owned, replayable queue and decouples ingestion availability from processing logic.
- Worker scenario with an idempotency gate
Implement a separate worker scenario that polls or is triggered for new staging rows. At the start, perform an idempotency gate by attempting to insert a processing-state record keyed by event_id. Example SQL:
INSERT INTO processing_state (event_id, status, attempts, last_seen) VALUES (:event_id, 'processing', 0, NOW()) ON CONFLICT (event_id) DO NOTHING;
If the insert succeeds you own the work. If the record exists with status=’success’, skip. If status=’processing’ and last_seen is recent, skip (concurrency guard); if status=’failed’ and attempts < retry_threshold, allow reprocessing.
- Controlled retries and backoff
Map Make error conditions to retryable vs permanent categories using error handlers. For transient network or 5xx API errors, increment attempts and schedule next_retry with exponential backoff (e.g., base 2, jitter, max interval 1 hour). Persist attempts and next_retry in processing_state so retries survive Make scenario restarts. For non-retryable client errors (4xx indicating invalid payload), mark as failed and move to DLQ immediately.
- Dead-letter queue (DLQ)
After N attempts (commonly 3–5 depending on business SLAs) move the item to a DLQ with full context: payload, event_id, attempts, last_error, timestamp, and Make run_id/log-link. The DLQ is intended for human review and manual replay; do not delete entries automatically. Provide a replay mechanism that will re-insert the DLQ item into the staging table after remediation.
- Safe external writes and idempotency tokens
Where downstream APIs support idempotency, forward the same event_id or a derived idempotency_key in the outbound request headers. For systems without native idempotency, implement an application-level dedupe by upserting based on a processed_event_id field (atomic upsert with WHERE processed_event_id IS NULL). Example pseudo-SQL for target DB:
UPDATE orders SET status = :status, processed_event_id = :event_id WHERE order_ref = :ref AND (processed_event_id IS NULL OR processed_event_id <> :event_id); IF row_count = 0 THEN -- maybe insert or skip END IF;
- Observability & alerting basics
Emit structured monitoring events for: items moved to DLQ, staging queue depth, worker processing latency, and repeated transient failures for the same event_id. Alerts should include event_id, attempts, last_error, timestamp and a direct link to the Make scenario run log. Use thresholded alerts (e.g., DLQ growth > X/hour, queue depth > Y) and route severe incidents to paging channels while sending lower-priority notices to Slack/email.
Architecture Notes
- Two-tier topology: webhook ingestion (fast ack + durable write) and worker processors (idempotent, retry-aware). This pattern reduces front-line latency and improves operational control.
- State model: processing_state table schema should include event_id (PK), status (processing|success|failed), attempts, next_retry_at, last_error, processed_at, and a run_reference. Keep payloads in a separate blobs table to avoid large-row contention.
- Make specifics: Make webhooks are queued and can run in parallel; to avoid duplicate side effects, perform external writes from the worker scenario that enforces idempotency. Use Make error handlers (break, commit, ignore, rollback) to ensure the scenario sets appropriate state and does not leave operations half-applied.
- Scaling: Add horizontal workers but preserve uniqueness via the database constraint. Use rate limits on outbound APIs and backpressure from the worker to avoid cascading failures.
Risks and Guardrails
- Concurrent processing race: Without DB uniqueness and an acquire-lock pattern you risk duplicate processing. Implement an atomic compare-and-set or unique constraint-based insert to avoid races.
- DLQ bloat: If many invalid payloads flood the system, DLQ can grow quickly. Apply lightweight validation earlier (schema checks) and auto-classify clearly invalid events to a separate archive with short retention.
- Partial success / compensation: Multi-step external workflows may partially succeed. Record each external action outcome and provide compensating flows or idempotent corrections so replays are safe.
- Secrets & PII leakage: Redact sensitive fields from logs, DLQ and monitoring; store tokens and secrets in secure vaults and never persist them with payloads.
- Operational complexity: This pattern introduces more moving parts (staging store, state table, DLQ). Automate deployments and include runbook steps for manual replay to reduce cognitive load on on-call teams.
What To Do Next
- Change the webhook-triggered scenario to persist payloads to a staging table and return either 202 or a processing_id where allowed.
- Build the worker scenario that enforces the idempotency gate via a unique processing_state insert and performs idempotent external calls using event_id as idempotency_key.
- Configure Make error handlers to categorize retryable vs permanent failures and update processing_state appropriately.
- Implement DLQ movement after your retry threshold and add a manual replay pathway with safeguards to prevent mass replays.
- Instrument metrics: staging queue depth, DLQ growth, per-event attempts and processing latency. Run fault-injection tests (transient API failures, duplicate incoming webhooks) to validate no duplicate side effects and correct DLQ behavior.
Sources
- Make: Webhooks – Help Center. https://help.make.com/webhooks
- Make Gateway Apps docs: Webhooks app. https://apps.make.com/gateway
- Make community: How to restart a scenario with a webhook trigger. https://community.make.com/t/tutorial-how-to-restart-a-make-com-scenario-with-a-webhook-trigger/55027
- Reliability Layer: Make.com Retry Logic: Replay Failed Webhooks Without Duplicates. https://reliabilitylayer.com/blog/make-com-retry-logic-duplicate-safe
- 4Spot Consulting: Webhooks best practices for resilient Make.com workflows. https://4spotconsulting.com/make-com-webhooks-best-practices-for-resilient-secure-workflows/
- Make blog: Error handling for workflow automation. https://www.make.com/en/blog/workflow-automation-error-what-to-do
