0

I was wondering if Google App Engine supports reading and writing large files ( example, text files of size more than 2GB ) into and from Google Drive or Cloud Storage ?

What problems can I expect?

I'm using the Python 2,7 Google App Engine SDK by the way.

Updates.

I intend to read up to a million rows of data from Google's Data Store ( or maybe the new NDB ), and save the data into a text file for further processing either on Google Cloud Compute or some third party services like PiCloud.

The data is basically a network relationship and it goes like this:

A -> B
B -> C
A -> D

The above means that A is linked to B, B is linked to C and A is linked to D and so on...

As i have over a million edges, i think i might have to use the task queue or cron job to do this?

So after i processed the relationships, i'll have another text file which contains some scores between each pair of nodes, which i will than write back into the database.

Best Regards.

4

2 回答 2

1

是的 - 问题是您打算如何编写文件？

您可以使用gsutil或create_upload_url直接上传到 Cloud Storage，也可以使用此文件 API从您的应用程序写入。

如果您使用的是文件 API，那么您需要以不大于 32MB 的块读取或写入。

于 2012-10-01T09:51:12.600 回答
1

应用引擎有一个 API，或者我想你可以使用 REST 云 API。我刚开始，但这里是 api 页面 https://developers.google.com/appengine/docs/python/googlestorage/

于 2012-10-01T03:46:18.507 回答