docs(deps): document Airflow logs PVC ReadWriteMany requirement#494
docs(deps): document Airflow logs PVC ReadWriteMany requirement#494safayavatsal wants to merge 1 commit into
Conversation
The upstream Airflow chart (v1.18.0) hardcodes ReadWriteMany on the logs PVC, which causes provisioning failures on clusters that only support ReadWriteOnce (e.g., GKE Autopilot with standard-rwo storage class). - Add WARNING comment in values.yaml above logs persistence config - Update README with RWX storage guidance for both DAGs and logs - Add cloud provider RWX storage class reference table - Include logs storageClassName in production example values Closes open-metadata#468
|
Taking up issue #468. Root cause: The upstream Apache Airflow Helm chart (v1.18.0 through v1.20.0) hardcodes Since this is an upstream issue that cannot be patched from the parent chart (Helm does not allow overriding subchart templates), this PR adds clear documentation and warnings so users can apply the appropriate workaround for their environment. A proper fix would require the Apache Airflow Helm chart to templatize |
Summary
Fixes #468
The upstream Apache Airflow Helm chart (v1.18.0) hardcodes
ReadWriteManyas the access mode on the logs PersistentVolumeClaim (logs-persistent-volume-claim.yaml:41). This causes PVC provisioning failures on clusters whose default storage class only supportsReadWriteOnce, such as GKE Autopilot withstandard-rwo:Unlike the DAGs PVC (which uses a configurable
dags.persistence.accessMode), the logs PVC access mode is not configurable — even in the latest upstream chart version (v1.20.0).Changes
charts/deps/values.yaml: Added aWARNINGcomment above theairflow.logs.persistencesection explaining the RWX requirement and two workarounds (use an RWX storage class, or disable logs persistence)charts/deps/README.md:logs.persistence.storageClassNameRoot Cause Analysis
ReadWriteOnce(default)dags.persistence.accessMode)ReadWriteMany(hardcoded)ReadWriteOnceWorkarounds for affected users
Use an RWX-capable storage class:
Disable logs persistence:
Test plan
helm template --dry-runrenders cleanly with default values