Medium severitydata source
Power BI Refresh Error:
UserErrorCosmosDbRequestRateOrLimitExceeded
What does this error mean?
An Azure Data Factory copy activity was throttled by Azure Cosmos DB because the provisioned throughput (RU/s) for the collection was exceeded. Cosmos DB returns HTTP 429 which ADF surfaces as this error.
Common causes
- 1The ADF copy activity is writing documents at a rate that exceeds the collection's provisioned RU/s
- 2The copy activity's parallelism setting is too high, sending too many concurrent writes
- 3Another application is simultaneously consuming the collection's RU budget while ADF is copying
- 4The Cosmos DB collection throughput is set too low for the data volume being copied
How to fix it
- 1Reduce the copy activity's parallelism: lower 'Data integration units' (DIU) and 'Degree of copy parallelism'.
- 2Temporarily scale up the Cosmos DB collection's provisioned throughput (RU/s) during the copy window, then scale back down after.
- 3Enable autoscale on the Cosmos DB container to automatically handle throughput spikes.
- 4Check Cosmos DB metrics in the Azure portal for current RU consumption.
- 5Stagger the pipeline schedule to avoid running the Cosmos DB copy simultaneously with other applications.
Frequently asked questions
Official documentation: https://learn.microsoft.com/en-us/azure/data-factory/connector-troubleshoot-azure-cosmos-db