python - 将 Python 诗歌与 Docker 集成

Question

你能给我一个例子Dockerfile，我可以在其中安装我需要的所有软件包，poetry.lock从pyproject.tomlDocker 安装到我的图像/容器中吗？

score 267 · Accepted Answer

poetry与一起使用时要记住几件事docker。

安装

官方安装方式poetry是：

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -

这种方式允许poetry将其依赖项与您的依赖项隔离开来。但是，在我看来，这不是一件好事，原因有两个：

poetry版本可能会得到更新，它会破坏你的构建。在这种情况下，您可以指定POETRY_VERSION环境变量。安装人员会尊重它
我不喜欢将互联网上的东西通过管道传输到我的容器中而没有任何可能的文件修改保护的想法

所以，我用pip install 'poetry==$POETRY_VERSION'. 如您所见，我仍然建议固定您的版本。

此外，将此版本也固定在您pyproject.toml的：

[build-system]
# Should be the same as `$POETRY_VERSION`:
requires = ["poetry>=1.0"]
build-backend = "poetry.masonry.api"

它将保护您免受本地和docker环境之间的版本不匹配。

缓存依赖项

我们希望缓存我们的需求，并且仅在文件更改pyproject.toml时重新安装它们。poetry.lock否则构建会很慢。为了实现工作缓存层，我们应该放：

COPY poetry.lock pyproject.toml /code/

在poetry安装之后，但在添加任何其他文件之前。

虚拟环境

接下来要记住的是virtualenv创造。我们不需要它docker。它已经被隔离了。因此，我们使用poetry config virtualenvs.create false设置将其关闭。

开发与生产

如果你Dockerfile像我一样在开发和生产中使用相同的，你将需要根据一些环境变量安装不同的依赖集：

poetry install $(test "$YOUR_ENV" == production && echo "--no-dev")

这种方式$YOUR_ENV将控制将安装哪些依赖项集：全部（默认）或仅带有--no-dev标志的生产。

您可能还想添加更多选项以获得更好的体验：

--no-interaction不要问任何互动问题
--no-ansi标记以使您的输出对日志更友好

结果

你最终会得到类似的东西：

FROM python:3.6.6-alpine3.7

ARG YOUR_ENV

ENV YOUR_ENV=${YOUR_ENV} \
  PYTHONFAULTHANDLER=1 \
  PYTHONUNBUFFERED=1 \
  PYTHONHASHSEED=random \
  PIP_NO_CACHE_DIR=off \
  PIP_DISABLE_PIP_VERSION_CHECK=on \
  PIP_DEFAULT_TIMEOUT=100 \
  POETRY_VERSION=1.0.0

# System deps:
RUN pip install "poetry==$POETRY_VERSION"

# Copy only requirements to cache them in docker layer
WORKDIR /code
COPY poetry.lock pyproject.toml /code/

# Project initialization:
RUN poetry config virtualenvs.create false \
  && poetry install $(test "$YOUR_ENV" == production && echo "--no-dev") --no-interaction --no-ansi

# Creating folders, and files for a project:
COPY . /code

您可以在这里找到一个完整的真实示例：wemake-django-template

2019-12-17 更新

更新poetry到 1.0

score 109 · Accepted Answer

使用 Poetry 和 venv 构建多阶段 Docker

不要禁用 virtualenv 创建。Virtualenvs 在 Docker 构建中发挥作用，因为它们提供了一种利用多阶段构建的优雅方式。简而言之，您的构建阶段将所有内容安装到 virtualenv 中，最后阶段只是将 virtualenv 复制到一个小图像中。

在复制代码之前，首先使用poetry export并安装您的固定需求。这将允许您使用 Docker 构建缓存，并且永远不会仅仅因为您更改了代码中的一行而重新安装依赖项。

不要poetry install用于安装您的代码，因为它将执行可编辑的安装。相反，使用poetry build构建一个轮子，然后将其 pip-install 到您的 virtualenv 中。（感谢PEP 517 ，整个过程也可以用一个简单pip install .的.

这是一个将 Flask 应用程序安装到 Alpine 映像中的示例 Dockerfile，它依赖于 Postgres。此示例使用入口点脚本来激活 virtualenv。但一般来说，没有入口点脚本应该没问题，因为您可以/venv/bin/python在CMD指令中简单地引用 Python 二进制文件。

Dockerfile

FROM python:3.7.6-alpine3.11 as base

ENV PYTHONFAULTHANDLER=1 \
    PYTHONHASHSEED=random \
    PYTHONUNBUFFERED=1

WORKDIR /app

FROM base as builder

ENV PIP_DEFAULT_TIMEOUT=100 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PIP_NO_CACHE_DIR=1 \
    POETRY_VERSION=1.0.5

RUN apk add --no-cache gcc libffi-dev musl-dev postgresql-dev
RUN pip install "poetry==$POETRY_VERSION"
RUN python -m venv /venv

COPY pyproject.toml poetry.lock ./
RUN poetry export -f requirements.txt | /venv/bin/pip install -r /dev/stdin

COPY . .
RUN poetry build && /venv/bin/pip install dist/*.whl

FROM base as final

RUN apk add --no-cache libffi libpq
COPY --from=builder /venv /venv
COPY docker-entrypoint.sh wsgi.py ./
CMD ["./docker-entrypoint.sh"]

码头入口点.sh

#!/bin/sh

set -e

. /venv/bin/activate

while ! flask db upgrade
do
     echo "Retry..."
     sleep 1
done

exec gunicorn --bind 0.0.0.0:5000 --forwarded-allow-ips='*' wsgi:app

wsgi.py

import your_app

app = your_app.create_app()

score 17 · Accepted Answer

TL;博士

我已经能够poetry使用. 在做了一些研究之后，我得到了以下结果：DjangopostgresDockerfile

FROM python:slim

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE 1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED 1

# Install and setup poetry
RUN pip install -U pip \
    && apt-get update \
    && apt install -y curl netcat \
    && curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="${PATH}:/root/.poetry/bin"

WORKDIR /usr/src/app
COPY . .
RUN poetry config virtualenvs.create false \
  && poetry install --no-interaction --no-ansi

# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]

这是的内容entrypoint.sh：

#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
    echo "Waiting for postgres..."

    while ! nc -z $SQL_HOST $SQL_PORT; do
      sleep 0.1
    done

    echo "PostgreSQL started"
fi

python manage.py migrate

exec "$@"

详细说明

需要注意的几点：

我决定使用slim而不是alpine作为python图像的标签，因为即使alpine图像应该减小 Docker 图像的大小并加快构建速度，但使用 Python，您实际上可以得到一个更大的图像，这需要一段时间构建（阅读本文了解更多信息）。
使用此配置构建容器比使用 alpine 映像更快，因为我不需要添加一些额外的包来正确安装 Python 包。
我poetry直接从文档中提供的 URL 安装。我知道sobolevn. 但是，我认为从长远来看，poetry默认情况下使用最新版本比依赖我应该定期更新的环境变量更好。
更新环境变量PATH至关重要。否则，你会得到一个错误，说没有找到诗歌。
依赖项直接安装在容器的 python 解释器中。它不会poetry在安装依赖项之前创建虚拟环境。

如果您需要此alpine版本Dockerfile：

FROM python:alpine

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE 1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED 1

# Install dev dependencies
RUN apk update \
    && apk add curl postgresql-dev gcc python3-dev musl-dev openssl-dev libffi-dev

# Install poetry
RUN pip install -U pip \
    && curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
ENV PATH="${PATH}:/root/.poetry/bin"

WORKDIR /usr/src/app
COPY . .
RUN poetry config virtualenvs.create false \
  && poetry install --no-interaction --no-ansi

# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]

请注意，该alpine版本需要一些依赖项postgresql-dev gcc python3-dev musl-dev openssl-dev libffi-dev才能正常工作。

score 16 · Accepted Answer

这是对@Claudio 提供的答案的小修改，它使用了 @sobolevn 在他的答案poetry install --no-root中描述的新功能。

为了强制诗歌将依赖项安装到特定的虚拟环境中，首先需要启用它。

. /path/to/virtualenv/bin/activate && poetry install

因此将这些添加到@Claudio的答案中，我们有

FROM python:3.9-slim as base

ENV PYTHONFAULTHANDLER=1 \
    PYTHONHASHSEED=random \
    PYTHONUNBUFFERED=1

RUN apt-get update && apt-get install -y gcc libffi-dev g++
WORKDIR /app

FROM base as builder

ENV PIP_DEFAULT_TIMEOUT=100 \
    PIP_DISABLE_PIP_VERSION_CHECK=1 \
    PIP_NO_CACHE_DIR=1 \
    POETRY_VERSION=1.1.3

RUN pip install "poetry==$POETRY_VERSION"
RUN python -m venv /venv

COPY pyproject.toml poetry.lock ./
RUN . /venv/bin/activate && poetry install --no-dev --no-root

COPY . .
RUN . /venv/bin/activate && poetry build

FROM base as final

COPY --from=builder /venv /venv
COPY --from=builder /app/dist .
COPY docker-entrypoint.sh ./

RUN . /venv/bin/activate && pip install *.whl
CMD ["./docker-entrypoint.sh"]

如果您需要将其用于开发目的，您可以--no-dev通过替换此行来添加或删除

RUN . /venv/bin/activate && poetry install --no-dev --no-root

像@sobolevn's answer中所示的这样的事情

RUN . /venv/bin/activate && poetry install --no-root $(test "$YOUR_ENV" == production && echo "--no-dev")

添加适当的环境变量声明后。

该示例使用 debian-slim 作为基础，但是，将其调整为基于 alpine 的图像应该是一项微不足道的任务。

score 16 · Accepted Answer

这是对我有用的最小配置：

FROM python:3.7

ENV PIP_DISABLE_PIP_VERSION_CHECK=on

RUN pip install poetry

WORKDIR /app
COPY poetry.lock pyproject.toml /app/

RUN poetry config virtualenvs.create false
RUN poetry install --no-interaction

COPY . /app

请注意，它不如@sobolevn 的配置安全。

作为琐事，我将补充一点，如果可以为pyproject.tomlprojects进行可编辑安装，则可以删除一两行：

FROM python:3.7

ENV PIP_DISABLE_PIP_VERSION_CHECK=on

WORKDIR /app
COPY poetry.lock pyproject.toml /app/

RUN pip install -e .

COPY . /app

score 10 · Accepted Answer

这是一个剥离的示例，其中首先将具有依赖关系的层（仅在这些更改时才构建），然后将具有完整源代码的层添加到图像中。设置poetry安装到全局site-packages会留下一个也可以删除的配置工件。

FROM python:alpine

WORKDIR /app

COPY poetry.lock pyproject.toml ./
RUN pip install --no-cache-dir --upgrade pip \
 && pip install --no-cache-dir poetry \
 \
 && poetry config settings.virtualenvs.create false \
 && poetry install --no-dev \
 \
 && pip uninstall --yes poetry \

COPY . ./

score 8 · Accepted Answer

我使用锁包（取决于锁文件中的所有版本的包）创建了一个解决方案。这会导致没有要求文件的纯 pip 安装。

步骤是：构建包，构建锁包，将两个轮子复制到您的容器中，使用 pip 安装两个轮子。

安装是：poetry add --dev poetry-lock-package

docker build 之外的步骤是：

poetry build
poetry run poetry-lock-package --build

那么你Dockerfile应该包含：

FROM python:3-slim

COPY dist/*.whl /

RUN pip install --no-cache-dir /*.whl \
    && rm -rf /*.whl

CMD ["python", "-m", "entry_module"]

score 5 · Accepted Answer

我看到这里所有的答案都是使用 pip 方式安装 Poetry 以避免版本问题。安装诗歌的官方方式读取 POETRY_VERSION 环境变量，如果定义为安装最合适的版本。

这里的 github 有一个问题，我认为这张票的解决方案很有趣：

# `python-base` sets up all our shared environment variables
FROM python:3.8.1-slim as python-base

    # python
ENV PYTHONUNBUFFERED=1 \
    # prevents python creating .pyc files
    PYTHONDONTWRITEBYTECODE=1 \
    \
    # pip
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    \
    # poetry
    # https://python-poetry.org/docs/configuration/#using-environment-variables
    POETRY_VERSION=1.0.3 \
    # make poetry install to this location
    POETRY_HOME="/opt/poetry" \
    # make poetry create the virtual environment in the project's root
    # it gets named `.venv`
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    # do not ask any interactive question
    POETRY_NO_INTERACTION=1 \
    \
    # paths
    # this is where our requirements + virtual environment will live
    PYSETUP_PATH="/opt/pysetup" \
    VENV_PATH="/opt/pysetup/.venv"


# prepend poetry and venv to path
ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"


# `builder-base` stage is used to build deps + create our virtual environment
FROM python-base as builder-base
RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        # deps for installing poetry
        curl \
        # deps for building python deps
        build-essential

# install poetry - respects $POETRY_VERSION & $POETRY_HOME
RUN curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

# copy project requirement files here to ensure they will be cached.
WORKDIR $PYSETUP_PATH
COPY poetry.lock pyproject.toml ./

# install runtime deps - uses $POETRY_VIRTUALENVS_IN_PROJECT internally
RUN poetry install --no-dev


# `development` image is used during development / testing
FROM python-base as development
ENV FASTAPI_ENV=development
WORKDIR $PYSETUP_PATH

# copy in our built poetry + venv
COPY --from=builder-base $POETRY_HOME $POETRY_HOME
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH

# quicker install as runtime deps are already installed
RUN poetry install

# will become mountpoint of our code
WORKDIR /app

EXPOSE 8000
CMD ["uvicorn", "--reload", "main:app"]


# `production` image used for runtime
FROM python-base as production
ENV FASTAPI_ENV=production
COPY --from=builder-base $PYSETUP_PATH $PYSETUP_PATH
COPY ./app /app/
WORKDIR /app
CMD ["gunicorn", "-k", "uvicorn.workers.UvicornWorker", "main:app"]

score 3 · Accepted Answer

这是一种不同的方法，它使 Poetry 保持完整，因此您仍然可以使用poetry addetc。如果您使用的是 VS Code devcontainer，这很好。

总之，安装 Poetry，让 Poetry 创建虚拟环境，然后每次启动新的 shell 时通过修改.bashrc.

FROM ubuntu:20.04

RUN apt-get update && apt-get install -y python3 python3-pip curl

# Use Python 3 for `python`, `pip`
RUN    update-alternatives --install /usr/bin/python  python  /usr/bin/python3 1 \
    && update-alternatives --install /usr/bin/pip     pip     /usr/bin/pip3    1

# Install Poetry
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python3 -
ENV PATH "$PATH:/root/.local/bin/"

# Install Poetry packages (maybe remove the poetry.lock line if you don't want/have a lock file)
COPY pyproject.toml ./
COPY poetry.lock ./
RUN poetry install --no-interaction

# Provide a known path for the virtual environment by creating a symlink
RUN ln -s $(poetry env info --path) /var/my-venv

# Clean up project files. You can add them with a Docker mount later.
RUN rm pyproject.toml poetry.lock

# Hide virtual env prompt
ENV VIRTUAL_ENV_DISABLE_PROMPT 1

# Start virtual env when bash starts
RUN echo 'source /var/my-venv/bin/activate' >> ~/.bashrc

提醒一下，没有必要避免使用 virtualenv。它不会影响性能，而且 Poetry 并不是真正设计为没有它们就可以工作的。

pyproject.toml编辑：@Davos 指出，除非您已经有一个andpoetry.lock文件，否则这不起作用。如果您需要处理这种情况，您也许可以使用这种解决方法，无论这些文件是否存在，它都应该有效。

COPY pyproject.toml* ./
COPY poetry.lock* ./
RUN poetry init --no-interaction; (exit 0) # Does nothing if pyproject.toml exists
RUN poetry install --no-interaction

score 3 · Accepted Answer

您可以在两个项目中了解如何正确执行此操作，或者您可以使用这些项目来构建您自己的镜像，因为它们只是基础镜像：

基础镜像的 Dockerfile：https ://github.com/max-pfeiffer/uvicorn-poetry/blob/main/build/Dockerfile

ARG OFFICIAL_PYTHON_IMAGE
FROM ${OFFICIAL_PYTHON_IMAGE}

ENV PYTHONUNBUFFERED=1 \
    PYTHONDONTWRITEBYTECODE=1 \
    PIP_NO_CACHE_DIR=off \
    PIP_DISABLE_PIP_VERSION_CHECK=on \
    PIP_DEFAULT_TIMEOUT=100 \
    POETRY_VERSION=1.1.11 \
    POETRY_HOME="/opt/poetry" \
    POETRY_VIRTUALENVS_IN_PROJECT=true \
    PYTHONPATH=/application_root \
    VIRTUAL_ENVIRONMENT_PATH="/application_root/.venv"

ENV PATH="$POETRY_HOME/bin:$VIRTUAL_ENVIRONMENT_PATH/bin:$PATH"

# https://python-poetry.org/docs/#osx--linux--bashonwindows-install-instructions
RUN apt-get update \
    && apt-get install --no-install-recommends -y \
        build-essential \
        curl \
    && curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python - \
    && apt-get purge --auto-remove -y \
      build-essential \
      curl

COPY ./scripts/start_uvicorn.sh /application_server/
RUN chmod +x /application_server/start_uvicorn.sh

COPY ./scripts/pytest_entrypoint.sh ./scripts/black_entrypoint.sh /entrypoints/
RUN chmod +x /entrypoints/pytest_entrypoint.sh
RUN chmod +x /entrypoints/black_entrypoint.sh

EXPOSE 80

CMD ["/application_server/start_uvicorn.sh"]

示例项目镜像的 Dockerfile：https ://github.com/max-pfeiffer/uvicorn-poetry/blob/main/examples/fast_api_multistage_build/Dockerfile

ARG BASE_IMAGE_NAME_AND_TAG=pfeiffermax/uvicorn-poetry:1.0.1-python3.9.8-slim-bullseye
FROM ${BASE_IMAGE_NAME_AND_TAG} as base-image

WORKDIR /application_root

# install [tool.poetry.dependencies]
# this will install virtual environment into /.venv because of POETRY_VIRTUALENVS_IN_PROJECT=true
# see: https://python-poetry.org/docs/configuration/#virtualenvsin-project
COPY ./poetry.lock ./pyproject.toml /application_root/
RUN poetry install --no-interaction --no-root --no-dev

FROM base-image as test-base-image
ENV LOG_LEVEL="debug"

COPY --from=base-image $VIRTUAL_ENVIRONMENT_PATH $VIRTUAL_ENVIRONMENT_PATH

# install [tool.poetry.dev-dependencies]
RUN poetry install --no-interaction --no-root

COPY /app /application_root/app/
COPY /tests /application_root/tests/

# image for running pep8 checks
FROM test-base-image as black-test-image

ENTRYPOINT /entrypoints/black_entrypoint.sh $0 $@

CMD ["--target-version py39", "--check", " --line-length 80", "app"]

# image for running unit tests
FROM test-base-image as unit-test-image

ENTRYPOINT /entrypoints/pytest_entrypoint.sh $0 $@

# You need to use pytest-cov as pytest plugin. Makes life very simple.
# tests directory is configured in pyproject.toml
# https://github.com/pytest-dev/pytest-cov
CMD ["--cov=app", "--cov-report=xml:/test_coverage_reports/unit_tests_coverage.xml"]

FROM base-image as development-image
ENV RELOAD="true" \
    LOG_LEVEL="debug"

COPY --from=base-image $VIRTUAL_ENVIRONMENT_PATH $VIRTUAL_ENVIRONMENT_PATH

# install [tool.poetry.dev-dependencies]
RUN poetry install --no-interaction --no-root

COPY . /application_root/

FROM base-image as production-image

COPY --from=base-image $VIRTUAL_ENVIRONMENT_PATH $VIRTUAL_ENVIRONMENT_PATH

# This RUN statement fixes an issue while running the tests with GitHub Actions.
# Tests work reliable locally on my machine or running GitHub Actions using act.
# There is a bug with multistage builds in GitHub Actions which I can also reliable reproduce
# see: https://github.com/moby/moby/issues/37965
# Will also check if I can fix that annoying issue with some tweaks to docker build args
# see: https://gist.github.com/UrsaDK/f90c9632997a70cfe2a6df2797731ac8
RUN true

COPY /app /application_root/app/

score 1 · Accepted Answer

使用 docker 多阶段构建和 python slim image，将诗歌锁导出到 requirements.txt，然后通过 pip 在 virtualenv 中安装。

它具有最小的尺寸，不需要在运行时图像中添加诗歌，固定所有内容的版本。

FROM python:3.9.7 as base
ENV PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app

FROM base as poetry
RUN pip install poetry==1.1.12
COPY poetry.lock pyproject.toml /app/
RUN poetry export -o requirements.txt

FROM base as build
COPY --from=poetry /app/requirements.txt /tmp/requirements.txt
RUN python -m venv .venv && \
    .venv/bin/pip install 'wheel==0.36.2' && \
    .venv/bin/pip install -r /tmp/requirements.txt

FROM python:3.9.7-slim as runtime
ENV PIP_DISABLE_PIP_VERSION_CHECK=1
WORKDIR /app
ENV PATH=/app/.venv/bin:$PATH
COPY --from=build /app/.venv /app/.venv
COPY . /app