system.iceberg_history and system.iceberg_files list nothing for Glue catalog tables

[DimensionWieldr]

✅ I checked the Altinity Stable Builds lifecycle table, and the Altinity Stable Build version I'm using is still supported.

Type of problem

Bug report - something's broken

Describe the situation

system.iceberg_history and system.iceberg_files return no rows for an Iceberg table reached through a Glue DataLakeCatalog database. SELECT against the same table returns its rows.

Reproduced on the latest public Antalya release, 26.6.4.20001.altinityantalya (commit 2073b1f88dec885444fac97edb8179ad1753c1ad). On that same binary, a REST DataLakeCatalog table lists its snapshot and its position-delete files.

This issue:

• system.iceberg_history stays empty after INSERT, while SELECT shows the inserted rows
• system.iceberg_files reports 0 POSITION_DELETE files after DELETE removed rows; the REST catalog on the same server reports 2
• Reproduced against LocalStack Glue, not AWS Glue

---

How to reproduce the behavior

Environment

• Version: 26.6.4.20001.altinityantalya
• Build type: release, public Antalya package
• Commit: 2073b1f88dec885444fac97edb8179ad1753c1ad (v26.6.4.20001.altinityantalya)
• Package: clickhouse-common-static_26.6.4.20001.altinityantalya_amd64.deb
• Catalog: LocalStack Glue at http://localstack:4566, warehouse on MinIO

Steps

1. Create a Glue catalog database and an Iceberg table.

SET allow_experimental_database_glue_catalog = 1;

CREATE DATABASE glue_db
ENGINE = DataLakeCatalog('http://localstack:4566')
SETTINGS
    catalog_type = 'glue',
    storage_endpoint = 'http://minio:9000/warehouse',
    region = 'us-east-1',
    aws_access_key_id = '...',
    aws_secret_access_key = '...';

CREATE TABLE glue_db.`ns.t` (event_date Date, id Int64)
ENGINE = IcebergS3('http://minio:9000/warehouse/data/ns/t/', '...', '...')
PARTITION BY event_date
ORDER BY id;

2. Insert rows and read history.

SET allow_insert_into_iceberg = 1;

INSERT INTO glue_db.`ns.t` VALUES
    ('2024-01-15', 1), ('2024-01-15', 2),
    ('2024-01-16', 3), ('2024-01-17', 4);

SELECT count() FROM glue_db.`ns.t`;

SELECT count()
FROM system.iceberg_history
WHERE database = 'glue_db' AND table = 'ns.t';

3. Delete two rows and read delete files.

DELETE FROM glue_db.`ns.t` WHERE event_date = '2024-01-16';
DELETE FROM glue_db.`ns.t` WHERE id = 1;

SELECT count() FROM glue_db.`ns.t`;

SELECT count()
FROM system.iceberg_files
WHERE database = 'glue_db' AND table = 'ns.t' AND content = 'POSITION_DELETE';

The same steps against a REST DataLakeCatalog database on this build list one APPEND snapshot after the insert and return 2 position-delete files after the deletes.

---

Expected behavior

After the insert, system.iceberg_history lists that table's snapshots. After the deletes, system.iceberg_files lists the position-delete files. SELECT count() returns 4 after the insert and 2 after the deletes.

Actual behavior

On release builds

SELECT count() returns 4, then 2. Both system-table counts return 0. system.iceberg_history and system.iceberg_files have no rows for that database at all.

system.iceberg_history count: 0
system.iceberg_files POSITION_DELETE count: 0

On REST, the same server returns:

system.iceberg_history count: 1   (operation APPEND)
system.iceberg_files POSITION_DELETE count: 2
system.iceberg_files DATA count: 3

---

Root cause analysis

StorageSystemIcebergHistory::fillData reads each Iceberg storage's metadata and, on any exception, logs Ignoring broken table and omits the table. system.iceberg_files does the same per manifest (Ignoring broken manifest). SELECT uses another read path and succeeds, so a metadata read that the system table rejects disappears from the result instead of surfacing an error.

---

Additional context

Seen with an explicit IcebergS3 table created inside the Glue database. The same gap showed up for tables created in Glue by PyIceberg and then read through DataLakeCatalog, on a #2361 build that printed this version string but was commit 953220d7166c8c7e464ffdd687872fc5eb805ce1, not the published release.

Regression coverage is iceberg/tests/iceberg_engine/drop_partition.py in clickhouse-regression. Those scenarios are skipped until DROP PARTITION is in a released build. DROP PARTITION is not in this public release; the system-table failure does not depend on it.