Return 404 instead of 500 for Dags whose serialized definition cannot be loaded#69099
Open
itej13 wants to merge 1 commit into
Open
Return 404 instead of 500 for Dags whose serialized definition cannot be loaded#69099itej13 wants to merge 1 commit into
itej13 wants to merge 1 commit into
Conversation
… be loaded A Dag can have run history and a serialized row in the metadata DB while its definition can no longer be reconstructed -- for example when it was written under a synthetic bundle by dag.test()/system tests against a shared DB, or when its serialized blob references an operator class that is not importable in the API server. The serving DagBag only handled the missing-row case, so a deserialization error escaped uncaught and surfaced as repeated HTTP 500s on the Dag detail page. Treat an undeserializable definition the same as a missing one so read-only API callers return a clean 404.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A Dag can have run history and a serialized row in the metadata DB while its definition can no longer be reconstructed — e.g. when it was written under a synthetic bundle by
dag.test()/ system tests against a shared DB, or when its serialized blob references an operator class that is not importable in the API server.The serving
DBDagBagonly handled the missing-row case (None→ 404). A deserialization error fromfrom_dict()escaped uncaught throughDBDagBag._read_dag, surfacing as repeated HTTP 500s and infinite loading on the Dag detail page (while the grid endpoint stayed at 200). This change catches the failure at that single read chokepoint and treats an undeserializable definition the same as a missing one, so every read-only Dag detail/version/run API endpoint returns a clean 404.The guard is scoped to the serving
DBDagBag, not theSerializedDagModel.dagproperty, so the scheduler/writer paths still surface deserialization errors loudly.closes: #69035
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.8) following the guidelines