SQL Catalog: Filter on iceberg_type in commit_table (#3337) - #3846
SQL Catalog: Filter on iceberg_type in commit_table (#3337)#3846hedger9487 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes SqlCatalog.commit_table so it consistently filters SQL catalog rows by iceberg_type (matching TABLE or NULL), preventing accidental operations on VIEW rows created by other Iceberg implementations.
Changes:
- Apply
self._iceberg_type_filter()to thecommit_tableUPDATE statement (rowcount-based path) and theSELECT ... FOR UPDATEquery (fallback path) inpyiceberg/catalog/sql.py. - Add a regression unit test ensuring commits fail (and do not modify the row) when the underlying row has
iceberg_type='VIEW'.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| pyiceberg/catalog/sql.py | Adds the iceberg_type predicate to commit_table’s update/locking queries to avoid touching VIEW rows. |
| tests/catalog/test_sql.py | Adds a unit test covering the commit_table behavior when the backing row is tampered into a VIEW. |
Suppressed comments (1)
tests/catalog/test_sql.py:413
- The assertion query filters only on table_name, which can become ambiguous if additional rows with the same name exist (e.g., different namespaces). Include catalog_name and table_namespace in the WHERE clause to ensure the test checks the intended row.
with catalog.engine.connect() as conn:
row = conn.execute(text("SELECT iceberg_type FROM iceberg_tables WHERE table_name = 'a_view'")).fetchone()
assert row is not None
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # Tamper the table row into a VIEW (simulating external writer) | ||
| with catalog.engine.connect() as conn: | ||
| conn.execute(text("UPDATE iceberg_tables SET iceberg_type = 'VIEW' WHERE table_name = 'a_view'")) | ||
| conn.commit() |
| assert row[0] == "VIEW" | ||
|
|
||
|
|
||
| def test_commit_table_ignores_view_rows(warehouse: Path) -> None: |
There was a problem hiding this comment.
I don't think this is a valid regression test. It passes even if I revert sql.py's change.
…pe_filter regression
|
Good catch! @ebyhr The previous test was failing early in I have updated |
Description
Fixes #3337.
In
SqlCatalog,_iceberg_type_filter()was added in #3263 to filter oniceberg_type(matchingTABLEorNULL) and avoid operating onVIEWrows written by other Iceberg implementations (e.g., Java or Rust). Whileload_table,drop_table,rename_table, andlist_tablesincorporate this filter,commit_tableomittedtype_filterin its SQL update statement.This PR applies
type_filter = self._iceberg_type_filter()to thecommit_tableupdate query inpyiceberg/catalog/sql.py.Testing
test_commit_table_ignores_view_rowsintests/catalog/test_sql.py.