Databricks Goes Dark for Hours and Nobody Gets an Error

On June 9, 2026, multiple users reported on the Databricks Community forums that the Free Edition login page simply wouldn't load. No error code. No timeout message. The page refused to open for hours, then quietly started working again. The original poster summarized it: "The issue was resolved by itself on the same day after few hours. Not sure why it was happening."

The Databricks status page showed all systems operational throughout. A Community Manager confirmed no known outage. Yet at least three independent users across different regions experienced the same blackout in the same timeframe.

This pattern is not unique to the Free Edition. Paid Databricks workspaces experience regional degradation that falls below the threshold for a formal incident. A control plane hiccup in a single availability zone. A serverless compute pool that takes 12 minutes to provision instead of 90 seconds. An OAuth token refresh that silently fails. These events don't trigger status page updates because they're transient and localized — but they're not transient to the job that was scheduled at 06:00 and needed a cluster at 06:01.

The Free Edition makes this visible because it has no SLA, no guaranteed reliability, and aggressive quota enforcement. Databricks documentation states plainly that Free Edition accounts operate in a "serverless-only, quota-limited environment" where exceeding fair usage shuts down compute for the remainder of the day. But the symptom users reported wasn't a quota message — it was a page that wouldn't load at all. That distinction matters. Quota enforcement returns a specific error. A service interruption returns nothing.

Databricks jobs are orchestrated through the control plane — the service layer that manages cluster lifecycle, job scheduling, and API requests. When the control plane is degraded, the consequences depend on timing.

A job that's already running on an active cluster may continue unaffected. Its compute resources are provisioned; the data plane handles execution independently. But a job scheduled to start during the interruption never launches. There's no retry by default unless you've configured retry policies in the job definition, and even then, the retry only fires if the initial attempt registered a failure. If the API call to start the job never reached the control plane, there's nothing to retry.

This creates a specific failure mode: a job run that doesn't exist. No failed run in the job history. No error in the logs. The run simply didn't happen, and the only evidence is the absence of an expected entry in the run history — or stale data in the target table.

For teams using the Databricks REST API via external orchestrators like Azure Data Factory or Apache Airflow, the failure surfaces differently. ADF's Web Activity calling api/2.1/jobs/run-now returns an HTTP timeout or connection refused error. That at least produces a pipeline failure. But ADF's default retry behavior may mask it — three retries with exponential backoff can silently absorb a 15-minute blip, making the eventual success look routine while hiding that the data landed 45 minutes late.

The 2024 Stack Overflow Developer Survey shows Databricks SQL at 1.9% adoption among all respondents. That number undersells its production footprint. Databricks tends to sit at the center of pipelines, not at the edge — one workspace feeds dozens of downstream consumers. A single missed job cascades.

When a Databricks job doesn't run, the tables it was supposed to update retain yesterday's data. Every system reading those tables proceeds normally because the data is valid — it's just stale.

A Power BI dataset scheduled to refresh at 07:00 connects to a Databricks SQL Warehouse, runs its queries, and completes successfully. The refresh log shows no errors. The semantic model updates. Dashboards render. But the numbers are from yesterday because the ETL job that should have refreshed the source table at 06:00 never executed. The Power BI refresh succeeded against stale data, and the service has no mechanism to know the difference.

The same applies to dbt models with a source freshness check — but only if you've defined freshness tests and actually run them. A dbt source freshness command checks the loaded_at field against expected intervals. If you've set warn_after: {count: 6, period: hour} and the source hasn't been updated, dbt will flag it. Most teams configure this. Fewer teams run it on every invocation, and fewer still route the warnings somewhere visible at 06:30 on a Monday.

ADF pipelines that depend on Databricks notebook activities face their own version. If the upstream Databricks pipeline failed (or never ran), ADF's Lookup or Copy activities may succeed against stale source data or fail with unexpected schema if a delta table's checkpoint is in an inconsistent state. The ADF error message in that case is typically DeltaSourceException or a generic UserError — neither of which points back to a missed upstream Databricks run as the root cause.

The gap between "the system is working" and "the data is current" is where stale dashboards live. Every tool in the chain reports green. The data is a day old.

Standard alerting catches failures. A Databricks job that starts and throws an exception produces a run with result_state: FAILED. You can alert on that with native Databricks notifications, PagerDuty webhooks, or custom polling against the Jobs API.

But alerting on a run that never happened requires a different approach. You need to know what should have run and compare it against what did run. This is schedule-aware monitoring.

The Databricks Jobs API endpoint GET /api/2.1/jobs/runs/list returns runs for a given job. If you poll this after the expected execution window and find no run with a start_time in the expected range, you've detected the gap. But building this polling infrastructure means maintaining a registry of expected schedules, accounting for time zones, handling DST transitions, and deciding what "late" means for each job.

Some teams build this with a lightweight Python script on a cron:

import requests, time
JOB_ID = 12345
EXPECTED_WINDOW_START = int(time.time()) - 7200  # 2 hours ago
runs = requests.get(
    f"{WORKSPACE_URL}/api/2.1/jobs/runs/list",
    params={"job_id": JOB_ID, "start_time_from": EXPECTED_WINDOW_START * 1000},
    headers={"Authorization": f"Bearer {TOKEN}"}
).json()
if not runs.get("runs"):
    send_alert(f"Job {JOB_ID} has no runs in the last 2 hours")

This works until you have 200 jobs across 4 workspaces. Then it becomes its own reliability problem — a monitoring script that itself needs monitoring.

MetricSign approaches this differently by tracking expected refresh cadences across Databricks jobs, Power BI datasets, and ADF pipelines. When a Databricks job doesn't produce a run within its expected window, MetricSign raises a refresh_delayed signal and correlates it with downstream consumers that are now running on stale data. The detection isn't "did the job fail" — it's "did the job happen at all, and who's affected."

The Databricks Free Edition is a useful canary. Its limitations — no SLA, quota-based throttling, potential account deletion after inactivity — make service interruptions frequent enough that users notice patterns. The June 9 incident wasn't an isolated event; the Community forums contain similar reports scattered across months, each following the same arc: sudden inaccessibility, no formal incident, self-resolution within hours.

Paid Databricks workspaces have SLAs and dedicated compute, but the control plane is shared infrastructure. Databricks publishes uptime targets, not uptime guarantees, for the control plane. The data plane — where your Spark jobs actually execute — runs on your cloud account's compute. But job scheduling, cluster provisioning, and API access all route through Databricks-managed services.

The practical risk isn't extended downtime. Databricks is generally reliable. The risk is the three-hour window where everything looks fine because no system is designed to notice a gap. Your Airflow DAG retried and succeeded 40 minutes late. Your Power BI refresh pulled yesterday's data and reported no error. Your Slack channel stayed quiet because nothing technically failed.

Three hours later, a finance director asks why the revenue dashboard shows Friday's numbers on Monday morning. The answer is a Databricks control plane blip at 05:47 that resolved by 08:30 — but by then, the damage to credibility is real. Every monitoring tool said green. The data said Friday.

Freshness monitoring requires three components: a schedule registry, a state comparison loop, and a dependency map.

The schedule registry defines what should happen and when. For Databricks jobs, this is the cron expression in the job definition — extractable via GET /api/2.1/jobs/get under settings.schedule. For ADF pipelines, it's the trigger definition. For Power BI, it's the dataset refresh schedule in the workspace settings. Collecting these into a single view is the first step.

The state comparison loop checks actual execution against expected execution. For Databricks, poll runs/list and verify a successful run exists in the expected window. For ADF, query the Monitor API for pipeline runs. For Power BI, use the GET /groups/{group_id}/datasets/{dataset_id}/refreshes REST endpoint to check the latest refresh timestamp.

The dependency map connects these checks. If Databricks job A feeds table X, and Power BI dataset B reads from table X, then a missed run of job A should generate an alert that names both the missed job and the affected dataset. Without this mapping, you get two separate alerts hours apart — one for the missed Databricks job (if you detect it) and one for stale Power BI data (if someone notices). With the mapping, you get one alert at 06:15 that says: "Job A didn't run. Dataset B's next refresh at 07:00 will pull stale data."

This is achievable with custom tooling. A combination of Azure Functions polling each API, a metadata store in a SQL database, and a notification layer can cover it. Budget 2-3 weeks of engineering time for a robust implementation across one Databricks workspace and its downstream consumers. Multiply that for each additional workspace, cloud region, or BI tool in your stack. The build-versus-buy calculation depends on how many 05:47 gaps you can absorb before someone asks why the dashboard is wrong.