fix(datasets): bypass soft-delete in dashboard_utils.get_table for tests

CI test-postgres (current) on PR apache#40130 cascade-failed with:

  psycopg2.errors.UniqueViolation: duplicate key value violates unique
  constraint "_customer_location_uc"
  DETAIL: Key (database_id, schema, table_name)=(1, public, birth_names)
          already exists.

Root cause: dashboard_utils.get_table queries SqlaTable through the
ORM, which the soft-delete listener now filters. When a prior test in
the same session soft-deletes the birth_names row (any test exercising
the new soft-delete DELETE behavior on datasets), the row stays in the
DB but is hidden from the helper. create_table_metadata's "doesn't
exist, INSERT" path then collides with the underlying (database_id,
schema, table_name) unique constraint that survives soft-delete (the
constraint is enforced even though the SQLAlchemy comment claims it's
not physical — Postgres test environments enforce it via a named
constraint that an old migration tried to drop).

Two-part fix in tests/integration_tests/dashboard_utils.py:

1. get_table attaches a per-query SKIP_VISIBILITY_FILTER_CLASSES
   execution option so it sees soft-deleted leftovers. Scoped to the
   single query — no leak to anything else in the session.

2. create_table_metadata restores any soft-deleted row it finds before
   mutating it for the new test's setup. Without this, the test would
   succeed in finding the row but then operate on a row whose
   `deleted_at` is still set, which the listener would hide from any
   subsequent API queries inside the same test.

Mirrors the pattern the dashboards branch (sc-106190) uses for
insert_dashboard's defensive cleanup, scoped narrowly to the test
helper and limited to SqlaTable.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: Mike Bridge

tests/integration_tests/dashboard_utils.py

@@ -24,6 +24,7 @@
 from superset.connectors.sqla.models import SqlaTable
 from superset.models.core import Database
 from superset.models.dashboard import Dashboard
+from superset.models.helpers import SKIP_VISIBILITY_FILTER_CLASSES
 from superset.models.slice import Slice
 from superset.utils import json
 from superset.utils.core import DatasourceType, get_example_default_schema
@@ -35,8 +36,14 @@ def get_table(
     schema: Optional[str] = None,
 ):
     schema = schema or get_example_default_schema()
+    # Bypass the soft-delete listener so the helper finds rows previously
+    # soft-deleted by other tests in the same session. Without the
+    # bypass, the listener hides them and a subsequent INSERT collides
+    # with the underlying (database_id, schema, table_name) unique
+    # constraint that survives soft-delete.
     return (
         db.session.query(SqlaTable)
+        .execution_options(**{SKIP_VISIBILITY_FILTER_CLASSES: {SqlaTable}})
         .filter_by(database_id=database.id, schema=schema, table_name=table_name)
         .one_or_none()
     )
@@ -60,6 +67,12 @@ def create_table_metadata(
             always_filter_main_dttm=False,
         )
         db.session.add(table)
+    elif table.deleted_at is not None:
+        # Restore a soft-deleted leftover from a prior test so the row is
+        # usable for this setup. Cleaning up via re-create-then-collide
+        # would fail on the underlying unique constraint that survives
+        # soft-delete.
+        table.deleted_at = None
     if fetch_values_predicate:
         table.fetch_values_predicate = fetch_values_predicate
     table.database = database