Make.com Error‑Handling Playbook: Idempotent Retries & Error Routing

Published by GreenCircuit on

Concept diagram of a Make.com scenario router splitting into success, retry, and dead-letter queue routes; overlays show idempotency key propagation and finance icons (invoice, bank)

Problem Context

Back‑office automations such as accounts payable, vendor onboarding, reconciliations and payroll make irreversible changes: payments, supplier records and ledger entries. Make.com scenarios integrate with external APIs (banks, ERPs, tax services) that can return transient failures, rate limits, or inconsistent partial responses. Without a disciplined approach to retries and routing, automatic replays risk duplicate payments, double invoices, or inconsistent state across systems. The design objective is predictable, safe replays: ensure that retries are duplicate‑safe (idempotent), errors are routed deterministically, and failed executions are surfaced for recovery without manual guesswork.

Implementation Workflow

Below are concrete patterns and a step‑by‑step workflow to implement in Make.com scenarios. The example flow used throughout: incoming invoice -> validate -> ensure supplier exists -> create invoice in ERP -> schedule payment.

  1. Classify each module by side effect risk.

    Label modules as READ (safe to retry) or WRITE (side effect). Build a small registry in your design doc. Only READ modules can be retried aggressively; WRITEs must use idempotency checks.

  2. Generate a deterministic idempotency key at the start.

    Pattern: idempotency_key = service_shortname + “:” + business_id + “:” + sha1(payload_fields). Example: make_pay:invoice:INV-2026-0456:3f5a2c. Keep the format consistent across teams. Persist this key in Make Data Store or include it as a header/field in downstream requests. When a downstream API supports an Idempotency‑Key header (common for payment APIs), pass it verbatim.

  3. Router topology and route filters.

    Add a Router module after the first response that can produce multiple routes: success flow, retriable error flow, non‑retriable flow, fallback/DLQ. Use explicit filters on router routes that inspect module outputs or HTTP status codes (e.g., status >=500 or contains “timeout” -> retriable; status 429 -> rate_limit; status 4xx with business message -> non_retriable). Order routes from specific -> general. Always include a fallback route that writes to DLQ rather than letting scenarios stop silently.

  4. Retry schedules tied to error class.

    Define a small set of retry policies and encode them into a reusable module or scenario fragment:

    • Transient (5xx, connection reset): exponential backoff — delays: 1m, 5m, 20m, 60m (max 4 retries).
    • Rate limit 429: if Retry‑After present, honor it. If absent, backoff: 2m, 10m, 30m; include jitter.
    • Service degraded but with queued processing (some banks): long backoff with low attempts (30m, 2h).
    • Business errors (validation, duplicate invoice): no automatic retry; route to manual review pipeline.

    Implement the above with conditional sleep/wait modules and attempt_count stored in scenario variables or the Data Store.

  5. Pre‑write existence checks for idempotent safety.

    Before any write, perform an existence check using the idempotency_key. Example: call ERP invoice GET?filter=idempotency_key=KEY. If present, treat as success and attach the external identifier to the current execution; skip the create call. If absent, perform create with the same idempotency_key header/field.

  6. Dead‑Letter Queue (DLQ) pattern.

    For executions that exhaust retries or hit non‑retriable errors, write a DLQ record to a persistent store: fields to include — idempotency_key, source_reference, payload_hash, last_error_message, last_status_code, attempt_count, timestamps, link_to_original. Use a structured JSON blob for the payload so recovery scenarios can read it reliably. Tag money‑sensitive records (payments) with high priority for ops alerts.

  7. Programmatic recovery & replay.

    Build a recovery scenario that reads DLQ entries, re‑runs the existence check against downstream systems, and performs one of two actions:

    • Auto‑replay when the existence check confirms no side effect and the payload_hash matches the original.
    • Escalate to manual review when hashes differ, the external system returns ambiguous state, or the DLQ entry is high‑severity.

    Expose replay modes via a small ops UI (Google Sheet / Airtable) or call the Make Incomplete Executions API to programmatically retry runs that are safe.

Concrete scenario snippet (logical)

Start -> compute idempotency_key -> validate invoice -> router -> (success route: existence check -> create/write -> notify) | (retriable route: increment attempt_count, wait per policy, requeue) | (non_retriable route: write DLQ & alert)

Architecture Notes

  • Centralise the idempotency generator and router into a reusable scenario library (Make scenario templates or cloned sub‑scenarios) so every finance flow uses the same key format and error classification.
  • Persist minimal context for each run to the DLQ to enable deterministic recovery: idempotency_key, payload_hash, attempt_count, last_responses. Avoid storing full PII unless encrypted and access controlled.
  • Where downstream systems support server‑side idempotency, prefer that over client checks; it is the strongest defense against duplicates during partial failures.
  • Use Make.com’s Incomplete Executions API and execution metadata to correlate failed runs with DLQ records and enable programmatic bulk retries in controlled windows.

Risks and Guardrails

  • Incorrect idempotency design. If keys are too coarse, distinct business actions collapse; if too granular (include timestamps), replays look new. Guardrail: adopt a tenant‑approved format and run automated tests to detect collisions.
  • Payload drift between original run and replay. Replaying a mutated invoice can create inconsistent ledgers. Guardrail: persist payload_hash and require manual review if hash mismatch.
  • DLQ overgrowth and alert fatigue. Without prioritisation, teams ignore DLQ entries. Guardrail: classify by monetary impact; only send high‑severity real‑time alerts and provide a daily digest for low‑severity items.
  • Excessive automated retries causing rate limit bans. Guardrail: implement backoff policies, honor Retry‑After headers, and monitor error trends with alert thresholds when retry volume spikes.

What To Do Next

  1. Create a small pilot: pick one finance scenario (e.g., supplier invoice ingest) and apply the idempotency + router + DLQ patterns end‑to‑end in a staging workspace.
  2. Run fault‑injection tests: simulate 5xx responses, 429s, and business validation failures to confirm routing, backoff timings, and DLQ writes.
  3. Build the recovery scenario that consults DLQ, checks existence via idempotency_key, and provides auto‑replay/manual modes. Test with stale and mutated payloads.
  4. Document key format, retry schedules, and triage steps in your ops runbook and train the support team to use the recovery tools.
  5. 5. Monitor production: track retries per scenario, DLQ inflow rate, and duplicate incidents. Tune backoff and classification rules quarterly.

Sources

Related Reading