Data Quality Monitoring Tools: What They Catch, What They Miss, and How to Choose One

A data quality monitoring tool runs assertions against your data on a schedule, fails when an assertion is violated, and emits an alert. The assertions look like this:

• customer_id should never be null
• order_total should be between 0 and 1.000.000
• country_code should be one of {NL, DE, FR, BE, ...}
• The number of rows ingested today should be within 20% of yesterday's count
• email should be unique within the active customer set

If you are familiar with unit tests for code, the mental model is the same. You write expectations. The tool runs them. You get told when reality breaks them.

This is genuinely useful and surprisingly underused. According to a 2023 BARC survey, only 38% of organisations have systematic data quality monitoring in place, despite 92% reporting data quality is a top concern.

Forrester research summarised that bad data costs the average enterprise $15 million per year, with measurement and monitoring being the most common gap in remediation programmes. (Forrester, 2024)

You can usually get the first 30 to 50 useful checks running in a day. The improvement to incident detection is immediate.

Three families of data quality monitoring tools dominate the market, and they answer slightly different questions.

For most teams the right answer is whichever fits the existing stack with the least friction. dbt-native if you already use dbt. A framework if you have engineers and complex sources. An observability platform if you need lineage and freshness alongside DQ.

The wrong answer is picking the tool with the longest expectation library and then never having time to write the expectations.

Every data quality monitoring tool, by design, catches what you wrote a rule for. That is the strength and the limit.

The failures that slip through:

1. The unknown unknowns Your rules cover what you have seen go wrong before. They miss the failures that happen for the first time. A new upstream system goes live on Monday and starts emitting a column with 80% empty strings instead of nulls. The null check passes. The dashboards are wrong by Tuesday.

2. Volume and freshness Most rule engines treat each row independently. They do not natively detect that today's batch arrived four hours late or contained 60% fewer rows than yesterday. You can approximate this with row-count checks comparing windows, but it is brittle.

3. Schema drift A column gets renamed upstream. The rule references the old name. The tool reports zero rows checked, which often is silently treated as zero violations. This is one of the most common ways a quality monitoring system creates false confidence.

4. Cross-table and cross-tool issues A referential check between two tables in the same warehouse is straightforward. Detecting that a Power BI dashboard is showing yesterday's data because an ADF pipeline failed three layers upstream requires lineage, not rules.

A 2024 Wakefield Research / Monte Carlo industry survey reported 70 data incidents per 1.000 tables per year on average, with the majority not caught by predefined data quality rules. (Monte Carlo, 2024)

The practical conclusion: rule-based tools cover the known half. You need observability tooling for the rest, or you need to accept that the unknown failures will continue to reach your business users.

These two terms get conflated and they should not be. The distinction shapes which tool you should buy first.

A practical heuristic: if you have not yet written a single data quality rule, start with a quality monitoring tool. The leverage is enormous and the cost is low. If you have hundreds of rules and incidents are still reaching your business users, you have hit the limit of what rules can do. That is when you add observability.

Most evaluations focus on the wrong things. Here is what actually matters once you have used a few tools in production.

Where the rules live. SQL-native tools keep rules close to the data, in version control, reviewable in pull requests. Framework tools often store expectations in YAML or JSON files separate from the data layer. The closer rules live to the transformations, the easier they are to maintain — and the more likely they will be updated when the data changes.

How easy it is to scope. Can you target a check at one column, one table, or 50 tables matching a pattern? Tools that force you to write each check explicitly do not scale past a few hundred checks. Tools that let you express "every staging table should have non-null primary keys" once cover thousands.

Cost of false positives. A tool that fires on every weekend dip in row counts will train your team to mute the channel within a week. Look for tools with built-in baseline awareness (rolling windows, day-of-week patterns) or anomaly thresholds.

Integration with your alert routing. A check that fires into the void is wasted work. The tool needs to integrate with your incident workflow: PagerDuty, Slack with ownership tags, or your data observability platform.

Performance impact. Some tools execute every check as a separate query. On a billion-row table, that is expensive fast. SQL-native tools that compile checks into a single query per table are usually cheaper at scale.

An IDC InfoBrief reports data engineers spend up to 30% of their time on incident triage and root cause. (IDC, 2023) The right data quality monitoring tool reduces the trivial portion of that time. The wrong one increases it.

Patterns we have seen repeated across teams.

Mistake 1: Writing too many rules too fast. A team imports 1.000 dbt tests on day one and gets 1.000 alerts on day two. Start with the 20 most-watched datasets. Add rules until you have meaningful coverage. Resist the urge to systemically test every column.

Mistake 2: Treating zero-row checks as zero violations. A rule that runs against a renamed or dropped column will return zero rows. Most tools will report zero failures, which is misleading. Configure the tool to flag empty result sets as suspicious, not safe.

Mistake 3: Forgetting to maintain rules when the data changes. Rules are code. Code rots. Build the rule update into the same PR workflow as model changes.

Mistake 4: Stopping at quality. Quality monitoring is necessary. It is not sufficient. Once you have decent rule coverage, the next bottleneck is everything quality monitoring cannot see: freshness, lineage, volume, the unknowns. That is the moment to add a data observability tool on top, not a moment too soon.

MetricSign is not primarily a data quality monitoring tool. It is a data observability tool that includes quality checks as one of its capabilities, alongside freshness, volume, schema drift, and cross-tool lineage.

If you are starting from zero, dbt tests or Soda will probably get you further faster on pure rule-based DQ, especially in a Snowflake-only stack. MetricSign is the right addition once you have rule coverage and the failures still reaching business users are the unknowns: stale Power BI refreshes, ADF pipeline drift, schema changes upstream of your dbt models, anomalies you did not write a rule for.

We are designed to run alongside your existing data quality monitoring tool, not replace it.