python - 下载大文件并将它们流式传输到 S3？

Question

我需要下载大文件并将它们写入 S3。是否可以将文件直接流式传输到 S3，而不是将文件下载到本地硬盘并复制到 S3？

我从https://www.python-httpx.org/advanced/找到了以下代码。如何将块写入 S3？

import tempfile

import httpx
from tqdm import tqdm

with tempfile.NamedTemporaryFile() as download_file:
    url = "https://speed.hetzner.de/100MB.bin"
    with httpx.stream("GET", url) as response:
        total = int(response.headers["Content-Length"])

        with tqdm(total=total, unit_scale=True, unit_divisor=1024, unit="B") as progress:
            num_bytes_downloaded = response.num_bytes_downloaded
            for chunk in response.iter_bytes():
                download_file.write(chunk)
                progress.update(response.num_bytes_downloaded - num_bytes_downloaded)
                num_bytes_downloaded = response.num_bytes_downloaded