What causes DELTA_CONCURRENT_DELETE_DELETE?

Two concurrent compaction or OPTIMIZE operations attempted to rewrite the same underlying Parquet files at the same time. A concurrent DELETE statement and a VACUUM operation both tried to remove overlapping file sets from the same Delta table. Two concurrent MERGE operations with DELETE clauses both targeted the same rows and their file rewrites conflicted

How do I fix DELTA_CONCURRENT_DELETE_DELETE?

Step 1: Retry the failed operation — Delta Lake's optimistic concurrency will resolve the conflict on retry once the first delete completes.. Step 2: Avoid running simultaneous OPTIMIZE or compaction jobs on the same table; serialize compaction or schedule it during low-write windows.. Step 3: Set delta.isolationLevel = Serializable on the table if write-write conflicts are frequent, though this will reduce write concurrency.. Step 4: Review the table's concurrent writer pattern and partition the table to give each writer exclusive ownership of distinct partition ranges.

High severityexecution

Power BI Refresh Error:
DELTA_CONCURRENT_DELETE_DELETE

What does this error mean?

A Delta Lake DELETE or MERGE operation failed because a concurrent operation already deleted one or more of the same files that the current operation was attempting to delete, creating an unresolvable conflict.

Common causes

1Two concurrent compaction or OPTIMIZE operations attempted to rewrite the same underlying Parquet files at the same time
2A concurrent DELETE statement and a VACUUM operation both tried to remove overlapping file sets from the same Delta table
3Two concurrent MERGE operations with DELETE clauses both targeted the same rows and their file rewrites conflicted

How to fix it

1Step 1: Retry the failed operation — Delta Lake's optimistic concurrency will resolve the conflict on retry once the first delete completes.
2Step 2: Avoid running simultaneous OPTIMIZE or compaction jobs on the same table; serialize compaction or schedule it during low-write windows.
3Step 3: Set delta.isolationLevel = Serializable on the table if write-write conflicts are frequent, though this will reduce write concurrency.
4Step 4: Review the table's concurrent writer pattern and partition the table to give each writer exclusive ownership of distinct partition ranges.

Frequently asked questions

How is this different from DELTA_CONCURRENT_DELETE_READ?

DELTA_CONCURRENT_DELETE_READ occurs when a concurrent operation deleted a file that your operation was reading. DELTA_CONCURRENT_DELETE_DELETE occurs when two operations both try to delete the same file.

Does enabling Auto Optimize prevent this?

Auto Optimize (auto compaction and optimized writes) can reduce the frequency by managing file sizes proactively, but it does not eliminate the possibility of concurrent delete conflicts on heavily written tables.