Skip to main content

Workflow Engine Integration Model

Circus doesn't execute workflows itself — it dispatches them to an external workflow engine (n8n today) and coordinates their lifecycle. The integration is deliberately narrow: the platform talks to engines only through a small, uniform engine-package contract, so supporting a different engine means writing one package, with no changes to the execution lifecycle.

The registry and per-workflow scoping​

Each engine instance is registered in the workflow_engine table with its engine_type, webhook base URLs, and API base URL. Every workflow is scoped to exactly one engine via workflow.workflow_engine_id and can't be enabled or dispatched without one. A key separation: the base URL lives on the engine, while the workflow carries only an opaque webhook_identifier. The engine package combines the two at dispatch time — so an engine's base URL can be updated once and every workflow using it follows, without touching workflow rows.

The package contract​

engine_type (n8n or other) selects a package implementing four methods, and the core never builds engine URLs, handles engine auth, or parses engine responses directly:

  • webhookStart / testWebhook — dispatch a production or test execution.
  • stop — remotely stop a running engine execution.
  • getExecutionStatus — query the engine's status.

The last two are the backbone of platform↔engine sync: operator termination and the reconciler both use them to keep the platform's status truthful against the engine's real state.

Three distinct credentials​

The n8n integration uses three separate credentials, and keeping them straight clarifies the whole boundary:

  1. Webhook dispatch is authenticated by a short-lived HS256 JWT signed with a per-engine webhook shared secret (platform → engine, to start a run).
  2. Machine callbacks (/logs, /complete, /terminate) are authenticated by an API Key — a JWT signed with workflow_jwt_secret (engine → platform).
  3. Remote stop / status calls use the engine's own n8n API key (X-N8N-API-KEY), so the platform can reach into the engine's REST API (platform → engine).

Why engine-agnostic matters​

By funnelling every engine interaction through this four-method contract, the platform keeps its own lifecycle logic free of engine specifics — dispatch, stop, and status are uniform regardless of engine. n8n is simply the first implementation; a genuinely different engine is a new package behind the same contract. It's also why an operator can point Circus at any suitable engine instance — the platform only needs it registered.