Direct answers to common questions about monitoring Power BI, data lineage, pipeline integrations, and incident response — written for data engineering teams.
Connect to the Power BI REST API to poll dataset refresh history, set up alerts on failure status, and route notifications to email or Teams. Purpose-built tools automate this across all workspaces with no polling code required.
Data observability for Power BI means monitoring five signals beyond refresh status: schema changes, volume anomalies, schedule drift, stale data, and end-to-end pipeline latency. Together these tell you whether the data in a report is correct, not just whether the refresh completed.
After each refresh, query the dataset's row count using the Power BI REST API or a direct database query, compare it to a rolling baseline for that dataset and day of week, and alert when the count drops more than a configured threshold (typically 15–20%).
Schedule drift is when Power BI refresh durations gradually increase over time — a refresh that used to finish in 10 minutes now takes 45. The refresh still succeeds, but it increasingly misses SLA windows for reports that users expect to be current at a specific time.
Monitor a watermark column — the maximum value of a date field like transaction_date, modified_at, or event_date in the loaded dataset. If this maximum hasn't advanced after a refresh, the refresh succeeded but loaded outdated data rather than new records.
Monitor the gateway Windows service health, cluster capacity (concurrent jobs), version against latest release, and per-data-source connectivity. A gateway outage silently fails every dataset routed through it — proactive monitoring prevents this from reaching users.
End-to-end latency measures the total time from when source data was last updated to when it's available in a Power BI report. It tracks each hop — source update, pipeline completion, dataset refresh — to confirm the full SLA was met, not just that individual steps completed.
Use the Power BI Admin API (requires admin role) or per-workspace API calls with a service principal that has workspace access. Aggregate refresh history across all workspaces into a single view and alert on failures regardless of which workspace they occur in.
Data lineage for Power BI maps how data flows from source systems through pipelines and transformations to Power BI datasets and reports. It shows which pipeline produced which table, which dataset reads from it, and which reports display that data.
Start with the report's dataset, check its data source configuration (server, database, table), find the pipeline that loads that table, and trace that pipeline back to its source. The Power BI REST API provides datasource metadata; pipeline run logs complete the chain.
Downstream impact analysis starts from a failing component and traces forward to identify everything it affects. Given a failed ADF pipeline, it answers: which tables are now stale, which datasets read from those tables, and which reports are currently serving incorrect data.
Match ADF pipeline output table names to Power BI datasource configurations retrieved via the REST API. When a dataset's configured server, database, and table match an ADF pipeline's output destination, you have a lineage link. Enrich with run timestamps to confirm each pipeline feeds each dataset.
The dbt manifest is a JSON artifact generated on every dbt build that documents all models, sources, columns, and their dependencies. For Power BI lineage, it provides the mapping between dbt model outputs (database tables) and the Power BI datasets that read from those tables.
The credentials stored for the data source connection are rejected. Common causes: a service account password was changed without updating the gateway, an OAuth token expired, or the database user was disabled or had its role revoked. Re-enter credentials in dataset settings to resolve.
After a gateway install or update, credentials may need to be re-entered, gateway cluster mappings need to be reassigned, and the service account permissions must carry over. Common errors are certificate-not-trusted, gateway-not-found, and credential validation failures.
Power BI disables a dataset's scheduled refresh after 4 consecutive failures by default. The dataset owner receives an email and must manually re-enable the schedule. This prevents repeated failed attempts but can leave datasets stale indefinitely if the owner doesn't notice.
DM_GWPipeline errors indicate the data gateway infrastructure failed to process the refresh request. The specific sub-error (after the prefix) tells you whether the gateway is unreachable, credentials are rejected, a query failed, or there's a network or certificate issue.
Query timeouts in Power BI occur when the source query takes longer than the configured limit (default 10 minutes). Fix by optimizing the source query, adding indexes, using incremental refresh, or increasing the dataset query timeout setting in Power BI Desktop.
Monitor ADF run status via Azure Monitor or the ADF REST API, correlate failed runs with the Power BI datasets that depend on their output tables, and alert when a pipeline failure will cause a downstream dataset to serve stale data on its next refresh.
Databricks jobs write Delta tables that Power BI datasets read from via Direct Lake or import mode. Monitoring both layers means detecting a Databricks job failure before the dependent Power BI refresh runs, so stale data doesn't reach users.
dbt Cloud integration connects dbt job run status, model-level errors, and manifest lineage to the Power BI datasets that depend on dbt model outputs. When a dbt model fails, you see which Power BI datasets will be affected before those datasets refresh.
Use the Fabric REST API to poll Fabric Data Pipeline run status, detect failures, and correlate them with the Fabric semantic models or Direct Lake datasets they feed. Fabric pipeline failures immediately affect Direct Lake reports — there is no import buffer.
Azure Monitor can collect Power BI activity logs via Log Analytics (Premium workspaces only), but requires KQL expertise to write alert queries and doesn't include data-specific signals like schema drift, volume anomalies, or cross-tool lineage to non-Azure sources.
The Power BI Admin Portal provides workspace governance, gateway cluster status, capacity utilization, and usage metrics. It cannot send automated refresh failure alerts across workspaces, detect upstream pipeline failures, or trace incidents to non-Power BI data sources.
APM tools measure request latency, error rates, and service availability — not data quality signals like refresh delays, volume anomalies, or schema drift. A data pipeline can be 'up' (executing without errors) while delivering wrong data, which APM tools cannot detect.
Build when requirements are narrow (refresh failure alerts only), engineering capacity is available, and the data stack is simple. Buy when the stack is multi-tool, incidents have business impact, or the monitoring needs to cover schema drift, volume anomalies, and lineage.
A data pipeline incident response playbook covers five elements: detection thresholds and escalation criteria, a triage checklist for the first 15 minutes, impact assessment steps, remediation paths by error type, and stakeholder communication templates.
Enterprise Power BI monitoring covers five layers: refresh status (all workspaces), data quality signals (volume and watermarks), upstream pipeline correlation (ADF, dbt, Databricks), gateway health, and structured incident response. Each layer addresses different failure modes.
During migration, monitor both environments simultaneously: SSIS job status (push to a monitoring table), ADF pipeline runs, gateway health, and Power BI refresh outcomes. Compare outputs from both environments during parallel running to validate parity before decommissioning on-premises jobs.
Configure alerts via Power BI's built-in email notifications (dataset owner only), the Power BI REST API with a polling script and webhook integration, Azure Monitor alert rules (Premium only), or a purpose-built monitoring tool. Each approach has different scope and complexity.
MetricSign puts these answers into practice — monitoring Power BI, ADF, Databricks, Fabric, and dbt in one incident feed.
Get started free →