What causes DF-Executor-PartitionDirectoryError?

The source dataset uses Hive-style partitioning (year=2024/month=01/) but a required partition directory does not exist yet for the current date range. A wildcard path in the source dataset does not correctly match the actual folder hierarchy — e.g., `*/data/*.parquet` when the actual path is `year=2024/month=01/data.parquet`. The managed identity or service principal has RBAC access at the container level but not on specific partition subdirectories created by a different process. An upstream pipeline wrote partition directories with an inconsistent naming convention, breaking the pattern the data flow source expects

How do I fix DF-Executor-PartitionDirectoryError?

Check the ADF activity run output for the specific partition directory path that caused the error.. Verify the source partition directory structure matches the pattern configured in the source dataset — for example, if using Hive-style partitioning (col=value), the folder names must match exactly.. Ensure the service principal or managed identity has read permissions on the partition directories, not the root container.. If using wildcard paths in the source dataset, confirm the wildcard pattern correctly matches the partition directory structure.. Enable debug mode and preview the source with a specific partition filter to validate that the partition directories are accessible and correctly structured.

Medium severityconfiguration

Power BI Refresh Error:
DF-Executor-PartitionDirectoryError

What does this error mean?

The data flow source encountered an error reading partition directories from the storage layer. The partition folder structure does not match the configured pattern, a required partition directory is missing, or the identity lacks read permissions on a subdirectory.

Common causes

1The source dataset uses Hive-style partitioning (year=2024/month=01/) but a required partition directory does not exist yet for the current date range
2A wildcard path in the source dataset does not correctly match the actual folder hierarchy — e.g., `*/data/*.parquet` when the actual path is `year=2024/month=01/data.parquet`
3The managed identity or service principal has RBAC access at the container level but not on specific partition subdirectories created by a different process
4An upstream pipeline wrote partition directories with an inconsistent naming convention, breaking the pattern the data flow source expects

How to fix it

1Check the ADF activity run output for the specific partition directory path that caused the error.
2Verify the source partition directory structure matches the pattern configured in the source dataset — for example, if using Hive-style partitioning (col=value), the folder names must match exactly.
3Ensure the service principal or managed identity has read permissions on the partition directories, not the root container.
4If using wildcard paths in the source dataset, confirm the wildcard pattern correctly matches the partition directory structure.
5Enable debug mode and preview the source with a specific partition filter to validate that the partition directories are accessible and correctly structured.

Frequently asked questions

The path exists in the storage account — why is ADF still failing?

Check permissions — the identity may have container access but not subdirectory access if ACLs differ. ADF paths are case-sensitive on ADLS Gen2; verify the path is an exact match.

Can I make the data flow skip missing partition directories instead of failing?

Yes — enable 'Allow no files found' in the source settings. The data flow returns zero rows when no matching directories exist instead of failing — useful for date-partitioned sources where recent partitions may not exist.

We use dynamic partition paths with parameters — could that cause this error?

Yes — if a parameter-driven path evaluates to a non-existent directory, the error is raised. Confirm the base path with a fixed-path debug run, then test the dynamic expression using the Refresh button.

Will downstream Power BI datasets be affected?

Yes — the pipeline fails and no data is written to the target. Dependent datasets and reports serve stale data until the partition path issue is resolved.

Official documentation: https://learn.microsoft.com/en-us/azure/data-factory/data-flow-troubleshoot-guide