What causes DF-Executor-BlockCountExceedsLimitError?

The data flow sink is writing to Azure Blob Storage with too many output partitions — each partition becomes a block, and the 50,000-block limit is exceeded. The 'Current partitioning' setting on the sink is using Spark's default partition count, which can produce hundreds or thousands of tiny partitions for large datasets. The sink is configured to write to a single blob but the data volume produces more than 50,000 blocks when split by the Spark partition scheme. A high-cardinality partition key in 'Hash' or 'Round robin' partitioning generates more partitions than the blob can accommodate

How do I fix DF-Executor-BlockCountExceedsLimitError?

Reduce the number of output partitions in the sink transformation — too many small partitions causes the Azure Blob or ADLS block count to exceed the 50,000-block limit.. In the sink Optimize tab, increase the partition size to produce fewer, larger files instead of thousands of tiny ones.. Switch the sink partition type from 'Current partitioning' to 'Single partition' or a fixed partition count to control block usage.. If writing to Azure Blob Storage, consider switching to ADLS Gen2 which has higher block count tolerances.. Enable data flow debug mode to preview the partition count before writing to the sink.

Medium severitydata flow

Power BI Refresh Error:
DF-Executor-BlockCountExceedsLimitError

What does this error mean?

The data flow sink exceeded Azure Blob Storage's 50,000-block limit per blob. When Spark writes too many small partitions to a single blob, each partition becomes a block — and Azure Blob Storage enforces a hard maximum of 50,000 blocks per blob.

Common causes

1The data flow sink is writing to Azure Blob Storage with too many output partitions — each partition becomes a block, and the 50,000-block limit is exceeded
2The 'Current partitioning' setting on the sink is using Spark's default partition count, which can produce hundreds or thousands of tiny partitions for large datasets
3The sink is configured to write to a single blob but the data volume produces more than 50,000 blocks when split by the Spark partition scheme
4A high-cardinality partition key in 'Hash' or 'Round robin' partitioning generates more partitions than the blob can accommodate

How to fix it

1Reduce the number of output partitions in the sink transformation — too many small partitions causes the Azure Blob or ADLS block count to exceed the 50,000-block limit.
2In the sink Optimize tab, increase the partition size to produce fewer, larger files instead of thousands of tiny ones.
3Switch the sink partition type from 'Current partitioning' to 'Single partition' or a fixed partition count to control block usage.
4If writing to Azure Blob Storage, consider switching to ADLS Gen2 which has higher block count tolerances.
5Enable data flow debug mode to preview the partition count before writing to the sink.

Frequently asked questions

What is Azure Blob Storage's block limit and why does it matter for data flows?

Azure Blob Storage allows a maximum of 50,000 blocks per blob. Each Spark output partition becomes a block — more than 50,000 output partitions for a single blob causes this error.

Does switching from Blob Storage to ADLS Gen2 fix this?

ADLS Gen2 uses multi-part upload, which is not subject to the 50,000-block limit. Switching the sink to ADLS Gen2 is the cleanest fix if reducing partition count is not desirable.

How do I reduce the partition count in a data flow sink?

In the sink Optimize tab, change the partition option to 'Set partitioning'. Choose a fixed number (e.g., 10 or 50) or 'Single partition' to control how many blocks Spark writes.

Will downstream Power BI datasets be affected?

Yes — the pipeline fails and the target storage path receives no new data.

Official documentation: https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-guide