What causes WORKFLOW_RUN_ALREADY_EXISTS?

The job's Maximum Concurrent Runs setting is set to 1 and the previous run has not yet completed. An upstream trigger (API call, webhook, or dbt Cloud integration) fired before the prior run finished. A long-running job is slower than its schedule interval, causing runs to queue and collide

How do I fix WORKFLOW_RUN_ALREADY_EXISTS?

Step 1: Check the active run in the Workflows UI and decide whether to cancel it or wait.. Step 2: If runs frequently overlap, increase Maximum Concurrent Runs in the job settings — or increase job parallelism.. Step 3: If the job consistently runs longer than its schedule interval, widen the interval or break the job into smaller parallel tasks to reduce runtime.. Step 4: For API-triggered jobs, add idempotency checks in the caller before submitting a new run.

Medium severityexecutionDatabricks →

Databricks Error:
WORKFLOW_RUN_ALREADY_EXISTS

What does this error mean?

A new Workflows job run was rejected because another run of the same job is already active and the job's concurrency policy prohibits parallel executions.

Common causes

1The job's Maximum Concurrent Runs setting is set to 1 and the previous run has not yet completed
2An upstream trigger (API call, webhook, or dbt Cloud integration) fired before the prior run finished
3A long-running job is slower than its schedule interval, causing runs to queue and collide

How to fix it

1Step 1: Check the active run in the Workflows UI and decide whether to cancel it or wait.
2Step 2: If runs frequently overlap, increase Maximum Concurrent Runs in the job settings — or increase job parallelism.
3Step 3: If the job consistently runs longer than its schedule interval, widen the interval or break the job into smaller parallel tasks to reduce runtime.
4Step 4: For API-triggered jobs, add idempotency checks in the caller before submitting a new run.

Frequently asked questions

Can I set concurrency to unlimited?

Databricks allows up to 1000 concurrent runs per job. Setting a high limit is fine for burst workloads, but ensure your cluster pool can support simultaneous runs without resource contention.

Does cancelling the active run prevent data corruption?

Usually yes, provided your notebooks are idempotent. Use transactional writes (Delta MERGE/REPLACE WHERE) to ensure a cancelled run does not leave partial data.

Source · docs.databricks.com/api/workspace/jobs/getrun