Azure Monitor Alerts for Data Pipelines: What They Catch and What They Miss for Power BI and ADF

Azure Monitor alerts for ADF are built on top of the diagnostic logs that ADF emits when you enable diagnostic settings on the factory. Once diagnostic settings route logs to a Log Analytics workspace, you have three signal types available: pipeline run logs, activity run logs, and trigger run logs. Each pipeline execution writes a record with run status, start time, end time, duration, and failure message when applicable.

Alert rules in Azure Monitor evaluate a metric or a KQL log query on a scheduled interval — commonly every 5 minutes — and fire when the condition is met. For a basic ADF pipeline failure alert, the metric approach is simpler: FailedPipelineRuns fires whenever the count of failed runs crosses zero in the evaluation window. You attach an action group to route the alert to an email address, a webhook, a Logic App, or an Azure Function.

The KQL-based approach gives more control. A log query alert that scans ADFPipelineRun for records where Status == 'Failed' and joins to ADFActivityRun to include the specific failing activity gives you richer alert payloads — the error message, the activity name, and the run ID that links to the ADF portal for drill-down.

For teams that have already invested in Log Analytics and have Azure Monitor set up for infrastructure monitoring, extending it to ADF pipeline failures is a relatively small incremental step. The diagnostic settings take a few minutes to configure; the alert rule and action group are reusable. This is the right starting point for ADF alerting.

These KQL patterns cover the most common ADF monitoring scenarios in Azure Monitor Log Analytics. All of them assume diagnostic settings are configured to route PipelineLogs and ActivityRuns to the workspace.

Detect failed pipeline runs in the last hour:

ADFPipelineRun
| where TimeGenerated > ago(1h)
| where Status == "Failed"
| project TimeGenerated, PipelineName, RunId, FailureType, Message
| order by TimeGenerated desc

Detect pipelines that ran longer than their expected duration (lateness proxy):

ADFPipelineRun
| where TimeGenerated > ago(7d)
| where Status == "Succeeded"
| summarize avg_duration = avg(DurationInMs), stdev_duration = stdev(DurationInMs) by PipelineName
| join kind=inner (
    ADFPipelineRun
    | where TimeGenerated > ago(1h)
    | where Status == "Succeeded"
) on PipelineName
| where DurationInMs > avg_duration + 2 * stdev_duration
| project PipelineName, DurationInMs, avg_duration

Detect a specific activity failure within a succeeded pipeline:

ADFActivityRun
| where TimeGenerated > ago(1h)
| where Status == "Failed"
| project TimeGenerated, PipelineName, ActivityName, ActivityType, Error
| order by TimeGenerated desc

These queries are the foundation for log query alert rules. The slow-pipeline query (the second one) requires the seven-day lookback window, which means the alert rule evaluation period needs to be set to at least 7 days — increasing the cost of the Log Analytics query each time it runs. For pipelines with stable schedules, a fixed threshold is usually more cost-effective than a dynamic baseline query.

Azure Monitor has no native connector to Power BI Import mode dataset refresh results. Power BI Premium workspaces can route diagnostic logs to a Log Analytics workspace, but the events available are capacity-level: AI events, dataset refresh duration in aggregate, query duration, and Premium capacity health signals. Individual dataset-level refresh success or failure is not in the Log Analytics schema for Power BI.

This means Azure Monitor cannot alert you when a specific Power BI dataset refresh fails. It cannot detect that a dataset refresh succeeded but loaded data from a source table that ADF failed to update. It cannot tell you that a dataset that normally refreshes in 8 minutes took 31 minutes today — a leading indicator that the next refresh will miss its SLA window.

Power BI's own notification system fills part of this gap: dataset owners receive email alerts on refresh failure. But these notifications are per-owner, not per-team; they are email-only with no webhook support; and they carry no upstream context. When a dataset fails because an upstream ADF pipeline loaded stale data, the Power BI notification tells the dataset owner the dataset failed — not why.

For organizations that have standardized on Azure Monitor as their central observability platform, the Power BI monitoring gap is an architectural problem. It means the monitoring estate is split: Azure Monitor covers the pipeline layer; Power BI Service notifications cover the BI layer; and nothing connects the two. An ADF pipeline that runs late and causes a Power BI dataset to refresh on incomplete data does not produce a correlated alert in either system.

The most damaging ADF failure mode for data freshness is not a pipeline that fails — it is a pipeline that never runs at all. A trigger that stops firing, a schedule that was inadvertently disabled, a trigger run that failed to start the pipeline execution: all of these result in no ADFPipelineRun record being written, which means no FailedPipelineRuns metric fires, and no log query alert can detect the absence.

Detecting missing runs in Azure Monitor requires a different approach: a scheduled KQL query that looks for the absence of a successful run record within the expected window. For a pipeline that should run daily at 05:00, the query checks whether there is a Succeeded record for that pipeline with a start time after 04:30 today. If not, the alert fires.

ADFPipelineRun
| where TimeGenerated > ago(2h)
| where PipelineName == "your-pipeline-name"
| where Status == "Succeeded"
| summarize count()
| where count_ == 0

This works, but it requires one alert rule per pipeline — or a parameterized approach using a watchlist of pipeline names and expected schedule windows. As your pipeline count grows, this maintenance overhead becomes substantial. A data platform with 40 pipelines needs 40 separate missing-run queries, each updated when schedule changes happen, each validated after ADF changes that rename or restructure pipelines.

The structural limitation is that Azure Monitor is reactive: it processes events as they arrive. Detecting the absence of an expected event requires the monitoring layer to maintain a model of what should have happened — a capability that Log Analytics alert rules approximate but do not natively support.

When an Azure Monitor alert fires for an ADF pipeline failure, the alert payload contains the pipeline name, the run ID, the failing activity, and the error message. What it does not contain — and cannot contain, because Azure Monitor has no model of cross-tool dependencies — is the downstream impact.

In a typical data stack, an ADF pipeline failure has an impact chain: the pipeline loads a staging table that a dbt model reads, which outputs a production table that a Power BI dataset imports, which feeds three executive dashboards. The ADF failure is the root cause. The stale executive dashboard is the business impact. The time between the alert and someone understanding the impact is investigation time spent tracing a chain that is nowhere documented in Azure Monitor.

This lineage gap is the reason data teams spend so much time on incident response even when alerting is nominally in place. The alert fires at 05:04. The on-call engineer acknowledges it at 05:20. By 06:00 they have traced the impact to the Power BI layer. The executive dashboards have already been opened by stakeholders at 06:30 with wrong data.

Azure Monitor with OpenTelemetry can capture distributed traces across application components — useful for microservices and API latency tracing. But OpenTelemetry traces require instrumentation at each component boundary. ADF does not emit OpenTelemetry traces; Power BI does not consume them. The data pipeline layer was not designed around distributed tracing, and retrofitting it is not a practical path for most teams.

MetricSign approaches this differently: it reads the native output of each tool (ADF run history, Power BI dataset metadata, dbt run results) and builds the lineage graph from the data relationships it discovers — not from trace headers. When an ADF pipeline fails, MetricSign already knows which Power BI datasets are downstream, and the alert payload includes that impact context from the first notification.

Azure Monitor is the right tool for infrastructure-level alerting: Azure resource health events, AKS node failures, App Service availability, and the high-volume telemetry that Log Analytics handles well at scale. For data teams, it is a reasonable foundation for ADF pipeline failure alerts — particularly teams that have already invested in Log Analytics and have the KQL capability in-house.

MetricSign operates above Azure Monitor, not in place of it. It adds the monitoring layer that Log Analytics cannot provide: dataset-level Power BI visibility, missing-run detection without per-pipeline query maintenance, baseline-aware lateness detection, and cross-tool lineage from ADF pipelines to downstream BI reports.

In practice, the two layers complement each other. Azure Monitor handles the infrastructure and ADF diagnostic log signals that your platform team already manages. MetricSign handles the data observability signals — freshness, completeness, schedule adherence, and BI-layer impact — that require awareness of the data relationships between tools, not just the health of individual Azure resources.

Setup is straightforward: MetricSign connects to your ADF factory via service principal, to your Power BI tenant via the Power BI REST API, and to other tools in your stack. It begins monitoring within minutes of connection and builds the lineage graph from the run history and metadata it discovers — no KQL, no diagnostic settings, no Log Analytics workspace required on the MetricSign side.