A migration script with DROP COLUMN or DROP INDEX was already applied in a previous run. Re-executing it (e.g. after a rollback + retry in ADF) triggers 1091 because the column or index no longer exists.. The column or index name in the migration script contains a typo or case mismatch — MySQL object names are case-sensitive on Linux filesystems, so `DROP COLUMN Status` fails if the actual column is `status`.. Schema drift between environments: the migration was developed against staging (where the column exists) but the production database is at a different schema version where it was already dropped.. A dbt pre-hook or post-hook drops a temporary index that was only created conditionally. On runs where the index was never created, the DROP INDEX raises 1091.. Multiple migration scripts target the same column. For example, migration V5 renames column `old_name` to `new_name`, and migration V7 tries to DROP `old_name` — which no longer exists after V5.

Step 1: Confirm the column or index actually exists before dropping. Run `SELECT COLUMN_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = 'your_db' AND TABLE_NAME = 'your_table';` and `SHOW INDEX FROM your_db.your_table;` to see what is currently present.. Step 2: If the column/index is already gone, mark the migration as applied without re-running it. In Flyway: `flyway repair` to fix the checksum, then insert the migration record into `flyway_schema_history`. In Liquibase: `liquibase changelogSync` for the specific changeset.. Step 3: Make your DROP idempotent. MySQL does not support IF EXISTS for DROP COLUMN, so wrap it in a stored procedure: `DROP PROCEDURE IF EXISTS drop_col_if_exists; DELIMITER // CREATE PROCEDURE drop_col_if_exists() BEGIN IF EXISTS (SELECT 1 FROM information_schema.COLUMNS WHERE TABLE_SCHEMA='your_db' AND TABLE_NAME='your_table' AND COLUMN_NAME='target_col') THEN ALTER TABLE your_table DROP COLUMN target_col; END IF; END // DELIMITER ; CALL drop_col_if_exists(); DROP PROCEDURE drop_col_if_exists;`. Step 4: For indexes, MySQL 8.0 does not support DROP INDEX IF EXISTS either (MariaDB does). Use the same stored procedure pattern, querying `information_schema.STATISTICS` instead of `COLUMNS`.. Step 5: If running in ADF, set the Stored Procedure activity retry count to 0 for migration activities — retrying a failed DDL migration will hit the same 1091 error and waste pipeline compute time.. Step 6: Sync schema state across environments. Run `mysqldump --no-data your_db > schema.sql` on the source environment and diff it against the target to identify which migrations are missing or already applied.. Step 7: After fixing, validate with a dry-run: execute the migration script in a transaction (`START TRANSACTION; ... ROLLBACK;`) on a staging copy to confirm it completes without errors before applying to production.

Low severityschema

MySQL Error:
1091, Column or Key Does Not Exist

What does this error mean?

MySQL raises error 1091 (ER_CANT_DROP_FIELD_OR_KEY) when an ALTER TABLE statement tries to DROP a column or index that does not exist in the target table. In data pipelines this typically surfaces during automated schema migrations: a dbt pre-hook, Flyway migration, or ADF Stored Procedure activity executes a DROP COLUMN or DROP INDEX on a column or key that was already removed in a previous run, or that never existed in this environment. The engineer sees the pipeline activity fail with the message "Can't DROP '<name>'; check that column/key exists". Because the ALTER TABLE is not idempotent, any re-execution of the same migration script triggers the error again, blocking the entire pipeline run until the migration is fixed or manually marked as applied.

Common causes

1A migration script with DROP COLUMN or DROP INDEX was already applied in a previous run. Re-executing it (e.g. after a rollback + retry in ADF) triggers 1091 because the column or index no longer exists.
2The column or index name in the migration script contains a typo or case mismatch — MySQL object names are case-sensitive on Linux filesystems, so `DROP COLUMN Status` fails if the actual column is `status`.
3Schema drift between environments: the migration was developed against staging (where the column exists) but the production database is at a different schema version where it was already dropped.
4A dbt pre-hook or post-hook drops a temporary index that was only created conditionally. On runs where the index was never created, the DROP INDEX raises 1091.
5Multiple migration scripts target the same column. For example, migration V5 renames column `old_name` to `new_name`, and migration V7 tries to DROP `old_name` — which no longer exists after V5.

How to fix it

1Step 1: Confirm the column or index actually exists before dropping. Run `SELECT COLUMN_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA = 'your_db' AND TABLE_NAME = 'your_table';` and `SHOW INDEX FROM your_db.your_table;` to see what is currently present.
2Step 2: If the column/index is already gone, mark the migration as applied without re-running it. In Flyway: `flyway repair` to fix the checksum, then insert the migration record into `flyway_schema_history`. In Liquibase: `liquibase changelogSync` for the specific changeset.
3Step 3: Make your DROP idempotent. MySQL does not support IF EXISTS for DROP COLUMN, so wrap it in a stored procedure: `DROP PROCEDURE IF EXISTS drop_col_if_exists; DELIMITER // CREATE PROCEDURE drop_col_if_exists() BEGIN IF EXISTS (SELECT 1 FROM information_schema.COLUMNS WHERE TABLE_SCHEMA='your_db' AND TABLE_NAME='your_table' AND COLUMN_NAME='target_col') THEN ALTER TABLE your_table DROP COLUMN target_col; END IF; END // DELIMITER ; CALL drop_col_if_exists(); DROP PROCEDURE drop_col_if_exists;`
4Step 4: For indexes, MySQL 8.0 does not support DROP INDEX IF EXISTS either (MariaDB does). Use the same stored procedure pattern, querying `information_schema.STATISTICS` instead of `COLUMNS`.
5Step 5: If running in ADF, set the Stored Procedure activity retry count to 0 for migration activities — retrying a failed DDL migration will hit the same 1091 error and waste pipeline compute time.
6Step 6: Sync schema state across environments. Run `mysqldump --no-data your_db > schema.sql` on the source environment and diff it against the target to identify which migrations are missing or already applied.
7Step 7: After fixing, validate with a dry-run: execute the migration script in a transaction (`START TRANSACTION; ... ROLLBACK;`) on a staging copy to confirm it completes without errors before applying to production.

Example log output

ERROR 1091 (42000) at line 14: Can't DROP 'idx_customer_email'; check that column/key exists
2026-05-11 08:22:17 [ERROR] ADF activity 'Migrate_MySQL_Schema' failed: ErrorCode=UserErrorFailedToExecuteMySqlCommand, Message='Can't DROP idx_customer_email; check that column/key exists (MySQL errno 1091)'
2026-05-11 08:22:18 [WARN] dbt pre-hook on model staging.stg_customers failed: mysql.connector.errors.ProgrammingError: 1091 (42000): Can't DROP 'legacy_status'; check that column/key exists

Frequently asked questions

How do I safely drop a column that may or may not exist in MySQL?

MySQL does not support DROP COLUMN IF EXISTS (MariaDB 10.0.2+ does). The standard workaround is a stored procedure that queries information_schema.COLUMNS for the column's existence and only executes the ALTER TABLE if the column is found. Wrap the procedure creation, call, and cleanup in your migration script so it runs as a single unit.

What migration tools handle idempotency automatically for MySQL?

Flyway and Liquibase both track applied migrations in a metadata table (flyway_schema_history / databasechangelog). They skip already-applied migrations, preventing duplicate DROP errors. If you cannot use a migration tool, add a manual check against information_schema before every DDL statement.

Can I retry a pipeline that failed with error 1091?

Retrying without fixing the migration will hit the same error — the column or index is still absent. First resolve the schema state (either apply the missing prerequisite migration or mark the failing migration as complete), then retry. In ADF, set retry to 0 on DDL activities to avoid wasting time on automatic retries.

Does MySQL 8.0 support IF EXISTS for DROP INDEX?

No. As of MySQL 8.0, neither DROP COLUMN IF EXISTS nor DROP INDEX IF EXISTS are supported. MariaDB supports both since version 10.0.2. On MySQL, use a stored procedure that checks information_schema.STATISTICS before executing DROP INDEX.

Source · dev.mysql.com/doc/mysql-errors/8.0/en/server-error-reference.html