Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docker-stack-docs/build.rst
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,28 @@ Note that similarly when adding individual packages, you need to use the ``airfl
:language: text


Example of adding a Python venv for ``ExternalPythonOperator``
..............................................................

The following example creates a separate Python virtualenv inside the image and installs packages
from ``requirements.txt`` into it, leaving Airflow's own Python environment untouched. This is the
typical setup for the
:ref:`ExternalPythonOperator <apache-airflow-providers-standard:howto/operator:externalpythonoperator>`
(or the ``@task.external_python`` decorator), which executes tasks in a pre-existing, immutable
Python environment. Use ``/opt/airflow/venv/bin/python`` as the operator's ``python`` argument
to run tasks in this venv.

The venv is created without ``--system-site-packages``, so the packages installed inside it
are isolated from Airflow's own dependencies.

.. exampleinclude:: docker-examples/extending/add-python-venv/Dockerfile
:language: Dockerfile
:start-after: [START Dockerfile]
:end-before: [END Dockerfile]

.. exampleinclude:: docker-examples/extending/add-python-venv/requirements.txt
:language: text

Example when writable directory is needed
.........................................

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

# This is an example Dockerfile. It is not intended for PRODUCTION use
# [START Dockerfile]
FROM apache/airflow:3.4.0
COPY requirements.txt /
RUN python -m venv /opt/airflow/venv \
&& /opt/airflow/venv/bin/pip install --no-cache-dir -r /requirements.txt
# [END Dockerfile]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
colorama==0.4.0