Volume monitoring answers a fundamental question: does this dataset have approximately the right amount of data after each refresh? Setting it up requires three components: a mechanism to capture row counts, a baseline to compare against, and an alerting rule.
Capturing row counts
There are two main approaches to capturing row counts after a Power BI refresh:
Via the Power BI REST API: The API doesn't return row counts directly, but you can execute DAX queries against a dataset using the executeQueries endpoint. A simple EVALUATE ROW("Count", COUNTROWS(TableName)) returns the current row count without loading the full dataset.
Via direct database queries: If the dataset sources from a database you control, you can query the source table directly after the pipeline loads it and before the Power BI refresh runs. This is simpler and more reliable but requires database access.
Building the baseline
A simple absolute row count threshold ("alert if under 40,000 rows") works for stable datasets but generates false positives on datasets with natural variation. A better approach uses a rolling baseline:
- Calculate the average row count over the last 7–14 refreshes
- For datasets with strong day-of-week patterns (e.g., transaction tables with lower weekend volumes), use a day-of-week-aware baseline: compare Monday's count against previous Monday averages, not a general weekly mean
- Start with a 15% deviation threshold and tune based on actual variation you observe
Setting the alert threshold per dataset
Different datasets warrant different thresholds:
- Large fact tables (millions of rows): a 10% drop is significant — alert at 10–15%
- Medium transactional tables: natural variation is higher — start at 20% and tune
- Dimension tables and lookups: these should be very stable — a drop of more than 5% is almost always a problem
- Snapshot tables (fully reloaded): compare to the previous refresh, not a rolling average
Handling expected volume changes
Some volume changes are expected: end-of-month reporting periods, seasonal business patterns, data migrations. Your alerting system should support snoozing or acknowledging alerts for known volume changes, and your baseline calculation should weight recent refreshes higher to adapt to genuine growth trends.
What volume monitoring catches that refresh status misses
Volume monitoring catches: zero-row copies from ADF (pipeline succeeded but source was empty), soft deletes that wiped a partition before the refresh ran, incremental refresh policy misconfiguration that loaded the wrong partition, and source system failures that produced no export file. All of these produce a successful refresh status while loading incorrect data.