我正在尝试从我的云存储桶中读取 5 个基本文本文件,将它们压缩,然后写入 BlobStore。
from google.appengine.api import files
from google.appengine.ext import blobstore
from google.appengine.ext.webapp import blobstore_handlers
import StringIO
import zipfile
class FactoryHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self):
""" SERVE THE BLOB, IF KEY AVAILABLE """
k = self.request.get('key')
if k:
self.send_blob(k)
return
""" TAKES FROM CLOUD STORAGE , ZIPS IT """
zipstream = StringIO.StringIO()
zfile = zipfile.ZipFile(file=zipstream, mode='w')
objects = files.listdir('/gs/test-bucket')
for o in objects:
with files.open(o, 'r') as f:
data = f.read(1)
while data != "":
zfile.writestr(o.encode('utf-8'),data)
data = f.read(1)
zfile.close()
zipstream.seek()
""" NOW, ADD ZIP TO BLOBSTORE """
zip_file = files.blobstore.create(mime_type='application/zip',_blobinfo_uploaded_filename='test.zip')
zip_data = zipstream.getvalue()
with files.open(zip_file, 'a') as f:
f.write(zip_data)
files.finalize(zip_file)
blob_key = files.blobstore.get_blob_key(zip_file)
self.response.out.write(blob_key)
不知何故,我总是只得到每个文本文件中的最后一个字符。我怀疑这是因为我在调用f.read(1)
,但代码基本上是遍历每个字节,然后将其写入zfile
对象。
我尝试连接数据:
for o in objects:
with files.open(o, 'r') as f:
data = f.read(1)
while data != "":
data += f.read(1)
""" once completed, write """
zfile.writestr(o.encode('utf-8'),data)
但 App Engine 开发服务器挂起。可能是因为我们无法连接数据。
任何解决方案,这是否适用于非文本文件?(图像、mp3 等)
编辑:
所以我在我的生产应用程序 + Google Storage 帐户上运行了答案。
得到这个错误
ApplicationError: 8
Traceback (most recent call last):
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", line
710, in __call__
handler.get(*groups)
File "/base/data/home/apps/s~app-name/v1-55-app-
proto.363439297871242967/factory.py", line 98, in get
with files.open(o, 'r') as f:
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 520,
in open
exclusive_lock=exclusive_lock)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 276,
in __init__
self._open()
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 423,
in _open
self._make_rpc_call_with_retry('Open', request, response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 427,
in _make_rpc_call_with_retry
_make_call(method, request, response)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 252,
in _make_call
_raise_app_error(e)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/files/file.py", line 210,
in _raise_app_error
raise PermissionDeniedError(e)
PermissionDeniedError: ApplicationError: 8
我的 ACL 设置是正确的(我之前收到了 ACL 访问被拒绝错误并修复了它)
ACL 设置添加到原始设置,特定于存储桶
<Entry>
<Scope type="UserByEmail">
<EmailAddress>
app-name@appspot.gserviceaccount.com
</EmailAddress>
</Scope>
<Permission>
FULL_CONTROL
</Permission>
</Entry>
任何提示?根据来自https://developers.google.com/appengine/docs/python/googlestorage/exceptions的文档
exception PermissionDeniedError()
The application does not have permission to perform this operation.
更新 - 我注意到当我将文件设置为 public_read 时,我可以通过我的应用程序读取它们。这意味着我的应用程序未正确配置为以私有模式访问。有什么提示吗?修复它的唯一方法(我知道)是通过 ACL,我已经配置了该部分。