A view keeps reading a table after it is replaced: DataFusion binds views at creation #271

On a Context, a view keeps reading the table it was created over, even after a new Dataset is registered under the same name. It returns the old rows without an error.

import numpy as np
import xarray as xr
import xarray_sql as xql

ctx = xql.Context()
ctx.from_dataset("t", xr.Dataset({"x": ("i", np.array([1.0]))}, coords={"i": [0]}))
ctx.sql("CREATE VIEW v AS SELECT x FROM t")

ctx.deregister_table("t")
ctx.from_dataset("t", xr.Dataset({"x": ("i", np.array([2.0]))}, coords={"i": [0]}))

print("table:", ctx.sql("SELECT x FROM t").to_pydict())  # {'x': [2.0]}
print("view: ", ctx.sql("SELECT x FROM v").to_pydict())  # {'x': [1.0]}: the old table

DataFusion 54.0.0, xarray-sql main. The same happens on a plain DataFusion SessionContext with register_record_batches, so the behavior is DataFusion's: CREATE VIEW stores a logical plan whose table scans hold the table providers that existed when the view was created.

Why it's a surprise:

• DuckDB (1.5.5) resolves a view's tables by name when the view is queried, so the same steps, with xql.register, return the new rows.
• PostgreSQL also binds a view to its tables when the view is created, but it refuses to drop a table that a view depends on unless you DROP … CASCADE. So it never returns stale rows.
• DataFusion binds early and also allows the drop, so the view silently reads data that is no longer registered.

Where it bites: any loop that replaces a Dataset under a fixed name and reads it through a view. Examples are a training loop that re-registers its parameters each step, or a pipeline that swaps in each day's data. I hit it while writing the GPT demo in benchmarks/ml (#269) as a model view plus a loss. The demo works around it by recreating the view after each re-registration.

Possible directions:

• Document it in the limitations page.
• Have Context warn, or refuse, when deregister_table or from_dataset replaces a table that a view reads.
• Report it upstream to DataFusion, if late binding or a dependency check would be welcome there. I didn't find an existing issue in apache/datafusion.

🌀 Claude (ML demos, xarray-sql)