n8n complex workflow patterns: branching, fan-out, retries, and error handling

1. Conditional branching done right

For a binary condition, an IF node expresses the split directly. When a workflow has several explicit routes, a Switch node keeps the routing rules in one place instead of spreading them across a chain of IF nodes.

Pattern: route customer events by plan tier.

Trigger (webhook: customer.event)
  ↓
Switch node — rules:
  rule 1: {{ $json.plan === 'enterprise' }}  → Enterprise handler
  rule 2: {{ $json.plan === 'pro' }}         → Pro handler
  rule 3: {{ $json.plan === 'free' }}        → Free handler
  fallback: → Slack alert (unknown plan)

Each branch ends in a NoOp node before merging back — it makes the visual graph readable and gives you a stable handle to insert metrics later. The fallback rule is non-negotiable: an unhandled branch is a silent bug factory.

2. Fan-out and fan-in (the API-batch pattern)

Fan-out describes how data is routed; it does not by itself guarantee concurrent execution. For workflows created in n8n 1.0 and later, n8n completes one canvas branch before starting the next by default. Separate the batching/merge pattern from the concurrency decision:

1. Within one execution: Loop Over Items (formerly SplitInBatches) emits batches through its loop output and returns processed data through its done output. Use Merge only when separate streams must be combined, choosing the mode from the data semantics.
2. Across executions: Queue mode lets multiple workers process independent workflow or sub-workflow executions concurrently. Dispatching independent work is what creates parallelism; drawing multiple canvas branches does not.

Example: enrich 500 leads from an internal API, then write to Postgres.

Postgres (SELECT 500 leads)
  ↓
SplitInBatches (batchSize=10)
  ↓
HTTP Request (enrich) — retry on fail: 3 tries, wait 2000ms
  ↓
Postgres (UPSERT enriched row)
  ↓
[loop back to SplitInBatches until done]
  ↓
Slack (summary: "enriched 500/500 in {{ $execution.duration }}ms")

The example uses batchSize=10, but the production value must follow the documented rate limits and concurrency behavior of the specific downstream API. Tune to the weakest downstream. For the cost implications of running this kind of volume, see n8n vs Zapier self-hosting cost.

3. Retries with exponential backoff

HTTP Request nodes that expose Retry On Fail can apply bounded tries and a fixed wait between attempts. When the downstream system requires exponential backoff, jitter, or a circuit breaker, make that policy explicit in a sub-workflow:

// Inside the retry sub-workflow, before the HTTP node:
const attempt = $json.attempt ?? 0;
const baseMs = 1000;
const maxMs = 30000;
const jitter = Math.random() * 500;
const waitMs = Math.min(2 ** attempt * baseMs + jitter, maxMs);

await new Promise((r) => setTimeout(r, waitMs));
return [{ json: { ...$json, attempt: attempt + 1, waitMs } }];

Then the parent calls the sub-workflow with retryOnFail: true, maxTries: 6 and you get 1s, 2s, 4s, 8s, 16s, 30s backoff with jitter. The Code node above is 6 lines; the equivalent in Zapier is "buy a higher tier or write it externally".

4. Idempotency: stop processing the same webhook twice

Webhook delivery, source retries, and overlapping schedules can produce duplicate attempts. Workflows with non-idempotent side effects such as charging a card, sending an email, or writing to a CRM should use duplicate protection appropriate to the source and downstream API. The important boundary is an atomic claim before the side effect:

Webhook trigger
  ↓
Code node — compute idempotency key:
  const crypto = require('crypto');
  const key = crypto.createHash('sha256')
    .update(JSON.stringify({ id: $json.id, event: $json.event }))
    .digest('hex');
  return [{ json: { ...$json, idempotencyKey: key } }];
  ↓
Idempotency store — atomically claim the key:
  Postgres INSERT ... ON CONFLICT DO NOTHING
  or Redis SET {{ $json.idempotencyKey }} processing NX EX <ttl>
  ↓
IF (claim acquired?)
  false → completed: NoOp / return stored result
          processing: defer or retry later
  true  → [side effect; pass the key downstream when supported]
          ↓
          Idempotency store — mark completed

A separate GET followed later by SET has a race window: two concurrent executions can both observe a missing key and both perform the side effect. Use a unique insert, SET ... NX, or another atomic claim primitive. Expiry and failed-claim recovery are workload-specific; retain the key at least through the period in which the source may retry.

5. Sub-workflows: refactor around clear responsibilities

Sub-workflows in n8n are typed function calls — they take input items, produce output items, and version independently. The three triggers to extract one:

• Reuse. The same logic is called from multiple parent workflows and should have one maintained implementation.
• Readability. The parent workflow is difficult to navigate or review. Extract by responsibility, such as auth, enrich, persist, and notify.
• Partial retry. You want to retry the "persist" stage without re-running "auth" and "enrich". Each stage as a sub-workflow gives you per-stage retry and per-stage error workflows.

Naming convention that scales: {domain}.{verb} — e.g. billing.charge, billing.refund, billing.dunning-step. Folder structure in Git follows the same.

6. Error workflows: the one feature that pays for itself

In Workflow Settings → Error Workflow, point production workflows at a dedicated error-handling workflow that starts with Error Trigger. When an automatic execution fails, n8n runs the linked error workflow with failure context; manual executions do not trigger this path in the same way:

{
  "execution": {
    "id": "abc123",
    "url": "https://n8n.your-domain/execution/abc123",
    "retryOf": null,
    "error": { "message": "...", "stack": "..." },
    "lastNodeExecuted": "HTTP Request — charge customer",
    "mode": "trigger"
  },
  "workflow": { "id": "wf_42", "name": "Billing: charge customer" }
}

A typical error workflow does four things:

1. Alert — Slack/PagerDuty with severity by workflow tag.
2. Log — append to your observability stack (Datadog, Loki, OpenSearch).
3. Classify — Code node decides: retryable (network blip), business (validation failed), or critical (auth revoked).
4. Act — initiate a controlled retry through a separate supported or manual path for retryable failures, open a ticket for business failures, or page on-call for critical failures. Error Trigger itself does not resume the failed parent execution.

One error workflow per project, not per workflow. Treat it as your workflow-level catch.

7. Scaling complex workflows in production

The patterns above are correctness primitives. Once correctness is solved, scale comes from infrastructure:

• Queue mode. EXECUTIONS_MODE=queue distributes independent executions across Redis-backed worker replicas. Capacity depends on workload shape, worker resources, concurrency settings, Redis, and database performance.
• Database choice. SQLite is the default database. Distributed queue-mode deployments should use PostgreSQL; migration timing depends on workload and retained execution data.
• Worker autoscaling. On Kubernetes, use observed queue depth and worker resource metrics to set scaling policy. Cost depends on the hosting model and minimum provisioned capacity.
• Execution data pruning. Execution pruning is enabled by default, with EXECUTIONS_DATA_MAX_AGE=336 (14 days) as the default maximum age. Tune retention and maximum count to storage, observability, and compliance needs.
• External secrets. Pull credentials from Vault / AWS Secrets Manager on Enterprise. On community, inject via $env + Docker secrets — works fine for most teams.

For the broader cost picture at each scale band — including when self-hosting stops paying off — see n8n vs Zapier self-hosting cost.

8. Real complex-workflow use cases

1. Multi-stage billing pipeline. Stripe webhook → idempotency check → Switch on event type → sub-workflow per type (charge / refund / dispute) → error workflow logs and opens a ticket on failure. ~40 nodes total, 3 sub-workflows, 1 error workflow.
2. AI document pipeline. S3 trigger → SplitInBatches → LangChain summarize + vector embed as separate stages → Postgres upsert → Slack digest. A local Ollama model avoids a hosted model's per-token API charge but still consumes local compute and operational resources.
3. SLA breach watcher. Cron every 5 min → Postgres query for open tickets past SLA → Switch on severity → escalate via PagerDuty / Slack / email. Idempotency key per ticket per hour to avoid spam.
4. CI/CD release coordinator. GitHub Release webhook → branch on semver → sub-workflows for changelog generation, Docker build trigger, multi-channel announce, customer-tier-aware email. One workflow, replaces three CI scripts and a Slack bot.
5. Customer onboarding state machine. Sign-up webhook → wait nodes between stages (welcome → 24h reminder → 7d activation check → 14d at-risk) → sub-workflow per stage. For longer waits, n8n offloads execution data to the database and reloads it on resume; this avoids continuous active execution but still uses database and infrastructure resources.

9. When the workflow is too complex for any workflow tool

Honest signal: if a workflow combines many sub-workflows, custom error classification logic, per-stage retry policies, and a state machine that needs persistence, it might be an application, not a workflow. The escape hatches:

• Stay in n8n when integration orchestration remains the dominant requirement and tested capacity meets the workload.
• Move to Windmill if you want the workflow tool to behave more like a script runner with a UI on top.
• Move to a real backend (Temporal, durable-execution libraries) if you need durable state across multi-hour workflows, targeted recovery, and strict ordering. External side effects still require idempotency.

For a wider landscape, see best Zapier alternatives.

10. Next reads

• Why developers choose n8n over Zapier — the strategic case
• n8n vs Zapier — head-to-head spec sheet
• n8n vs Zapier self-hosting cost — line-item math at 10k, 100k, 1M tasks/mo
• Best Zapier alternatives — the wider field
• Best n8n hosting for beginners — 5 hosting options ranked
• Windmill vs n8n — when the script-runner model wins
• Full n8n review — pros, cons, lock-in caveats