Microsoft Fabric Error:
Pipeline Error 3203
What does this error mean?
A Fabric Data Factory activity targeting an interactive Databricks cluster failed because the cluster was in a Terminated state and could not accept new jobs. Interactive clusters auto-terminate after idle periods; this is a race condition that does not occur with job clusters.
Common causes
- 1The interactive (all-purpose) Databricks cluster was terminated due to the auto-termination setting before the pipeline submitted the job
- 2A race condition where the cluster was starting up but not yet ready when the pipeline activity ran
- 3The cluster was manually terminated or failed to start due to resource constraints in the Databricks workspace
How to fix it
- 1Switch from an interactive (all-purpose) cluster to a job cluster in the Databricks activity configuration — job clusters are created on demand and terminated after the job completes, eliminating the race condition.
Beyond the docs
Common practitioner solutions not covered in the official documentation.
- 1If an interactive cluster is required, increase the auto-termination timeout to reduce the chance of it being terminated between pipeline runs
- 2Add a retry policy to the Databricks activity in the pipeline (1–2 retries with a 60-second wait) to handle transient termination states
- 3Check the Databricks workspace for resource quota issues if clusters frequently fail to start