What causes DEADLOCK_DETECTED?

Two concurrent MERGE or UPDATE operations on the same Delta table acquire conflicting row-level locks in opposite orders. A pipeline reads from table A and writes to table B while a concurrent pipeline reads from B and writes to A, creating a circular lock dependency. Concurrent DML operations on tables with aggressive locking modes enabled via Delta table properties

How do I fix DEADLOCK_DETECTED?

Step 1: Identify the two operations involved using the Databricks query history and job run timestamps.. Step 2: Refactor the write order so that all pipelines acquire locks on shared tables in a consistent sequence.. Step 3: Use partition pruning in MERGE predicates to narrow the lock scope, reducing the chance of lock overlap.. Step 4: Retry the aborted transaction automatically — Databricks aborts one side of the deadlock, and re-running it after a short delay is usually sufficient.

High severityexecution

Power BI Refresh Error:
DEADLOCK_DETECTED

What does this error mean?

Two or more concurrent Databricks transactions entered a deadlock state where each holds a lock the other needs, causing both to be blocked indefinitely until the system resolves the conflict by aborting one.

Common causes

1Two concurrent MERGE or UPDATE operations on the same Delta table acquire conflicting row-level locks in opposite orders
2A pipeline reads from table A and writes to table B while a concurrent pipeline reads from B and writes to A, creating a circular lock dependency
3Concurrent DML operations on tables with aggressive locking modes enabled via Delta table properties

How to fix it

1Step 1: Identify the two operations involved using the Databricks query history and job run timestamps.
2Step 2: Refactor the write order so that all pipelines acquire locks on shared tables in a consistent sequence.
3Step 3: Use partition pruning in MERGE predicates to narrow the lock scope, reducing the chance of lock overlap.
4Step 4: Retry the aborted transaction automatically — Databricks aborts one side of the deadlock, and re-running it after a short delay is usually sufficient.

Frequently asked questions

Does Delta Lake automatically retry deadlocked transactions?

Delta Lake does not automatically retry at the SQL engine level. The application layer must handle retries. Databricks workflows can be configured with automatic retry on failure to handle this.

Are deadlocks more common with Photon?

No. Photon changes query execution performance but not Delta Lake's locking semantics. Deadlock frequency depends on workload concurrency patterns, not the execution engine.