我正在使用批量加载程序从一台 GAE 服务器下载我的评论类型并上传到另一台 GAE 服务器。此评论类型包含大约 30,000 个实体。
我使用命令上传：
appcfg.py upload_data --kind=Review --filename=review.csv --application=<MY_APP_ID> --url=http://<MY_APP_ID>.appspot.com/remote_api --rps_limit=500 --bandwidth_limit=2500000 --batch_size=100
注意：整个错误很长。所以我把它包括在这个问题的底部。
上传根本没有开始并停止在该行：[INFO ] Opening database: bulkloader-progress-20110611.222101.sql3
我在整个跟踪的底部看到：lambda val: _EPOCH + datetime.timedelta(microseconds=val), OverflowError: date value out of range
我怀疑这是由于在我的发布日期列中发现的日期格式错误造成的（请参阅下面的整个评论类型）。
我已经尝试解决的问题
我在 GAE 管理门户中运行了 GQL：，SELECT * FROM Review where publisheddate = null发现大约 15 个实体的已发布日期列具有空值。所以我删除了那些，重新下载并尝试重新上传评论类型。我仍然得到同样的错误。
我在想，因为我正在使用没有配置文件参数的上传命令，例如--config_file=bulkloader.yaml（参见上面的上传，我没有指定--config_file）
我依靠数据存储统计信息来确定下载到我的review.csv文件的内容。
现在，由于我刚刚删除了已发布日期为空的实体，因此重新下载然后立即重新上传与我最初的下载和上传没有任何不同。由于尚未更新数据存储统计信息。
我的问题
由于有 30, 000 个实体，我不确定如何找到日期格式错误的实体。是否可以快速定位具有错误发布日期值的实体？
有谁知道如何解决这个问题？从错误跟踪中我不确定真正的问题是什么。
点评种类
我正在使用 objectify3.0.jar 来操作我的 GAE 数据存储。所以我的评论类型看起来像这样：
public class Review {
@Id private Long id; //This is my key property, auto generated by objectify
private String reviewlink;
private Date publisheddate;
private double rating;
private Long gameid; //to make querying easier
//ommitting getters and setters
}
整个错误
10:20:48>appcfg.py upload_data --kind=Review --filename=review.csv --application=<MY_APP_ID> --url=http://<MY_APP_ID>.appspot.com/remote_api --rps_limit=500 --bandwidth_limit=2500000 --batch_size=100
Uploading data records.
[INFO ] Logging to bulkloader-log-20110611.222101
[INFO ] Throttling transfers:
[INFO ] Bandwidth: 2500000 bytes/second
[INFO ] HTTP connections: 8/second
[INFO ] Entities inserted/fetched/modified: 500/second
[INFO ] Batch Size: 100
[INFO ] Opening database: bulkloader-progress-20110611.222101.sql3
Traceback (most recent call last):
File "/usr/local/bin/appcfg.py", line 76, in <module>
run_file(__file__, globals())
File "/usr/local/bin/appcfg.py", line 72, in run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 3592, in <module>
main(sys.argv)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 3583, in main
result = AppCfgApp(argv).Run()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 2229, in Run
self.action(self)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 3368, in __call__
return method()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 3195, in PerformUpload
run_fn(args)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/appcfg.py", line 3087, in RunBulkloader
sys.exit(bulkloader.Run(arg_dict))
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 4360, in Run
return _PerformBulkload(arg_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 4204, in _PerformBulkload
loader.initialize(filename, loader_opts)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 2879, in initialize
self.high_id_table = self._find_high_id(self.generate_records(filename))
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 2896, in _find_high_id
for values in record_generator:
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/bulkloader.py", line 2920, in generate_records
yield datastore.Entity._FromPb(fixed_entity_proto)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore.py", line 958, in _FromPb
value = datastore_types.FromPropertyPb(prop)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore_types.py", line 1792, in FromPropertyPb
value = conversion(value)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/api/datastore_types.py", line 1722, in <lambda>
lambda val: _EPOCH + datetime.timedelta(microseconds=val),
OverflowError: date value out of range