14

我转储了一个只有 1 个集合和两个索引的数据库。该集合大约有 650 万份文件。当我试图恢复它时,令我惊讶的是,这个过程非常缓慢。一些统计数据如下:

Wed Aug 14 12:34:08.346         Progress: 333818/1378309050 0%  (bytes)
Wed Aug 14 12:34:11.077         Progress: 1530246/1378309050    0%  (bytes)
Wed Aug 14 12:34:26.177         Progress: 2714544/1378309050    0%  (bytes)
Wed Aug 14 12:34:30.145         Progress: 3355627/1378309050    0%  (bytes)
Wed Aug 14 12:34:34.504         Progress: 3895668/1378309050    0%  (bytes)
Wed Aug 14 12:34:53.246         Progress: 4334159/1378309050    0%  (bytes)
Wed Aug 14 12:34:56.318         Progress: 4963878/1378309050    0%  (bytes)
Wed Aug 14 12:34:59.545         Progress: 5617794/1378309050    0%  (bytes)
Wed Aug 14 12:35:08.042         Progress: 6923804/1378309050    0%  (bytes)
Wed Aug 14 12:35:16.424         Progress: 7342576/1378309050    0%  (bytes)
Wed Aug 14 12:35:23.168         Progress: 7987560/1378309050    0%  (bytes)
Wed Aug 14 12:35:29.703         Progress: 9295140/1378309050    0%  (bytes)
Wed Aug 14 12:35:38.582         Progress: 9943758/1378309050    0%  (bytes)
Wed Aug 14 12:35:43.574         Progress: 11128693/1378309050   0%  (bytes)
Wed Aug 14 12:35:46.008         Progress: 11982044/1378309050   0%  (bytes)
Wed Aug 14 12:35:50.134         Progress: 12421241/1378309050   0%  (bytes)
Wed Aug 14 12:35:54.548         Progress: 13166696/1378309050   0%  (bytes)
Wed Aug 14 12:35:58.152         Progress: 13837935/1378309050   1%  (bytes)

从上面的数据可以推断,总转储(在 bson 中)大约是 1.3 Gigs。mongorestore 需要 110 秒才能恢复 1%,即 13 MB。

如果有人对此有解释,请告诉我。我希望我无意中做错了什么,因为与本世纪的计算标准相比,这些数字太慢了。

编辑


我使用以下两个选项再次运行该命令,希望它们能加快该过程:

--noobjcheck --noIndexRestore

但令我惊讶的是,这个过程现在更慢了!以下是一些统计数据。

Wed Aug 14 13:13:53.750     going into namespace [temp_raw_tweet_db.tweets]
Wed Aug 14 13:14:00.258         Progress: 871186/1378309050 0%  (bytes)
Wed Aug 14 13:14:04.424         Progress: 2070390/1378309050    0%  (bytes)
Wed Aug 14 13:14:07.482         Progress: 2921304/1378309050    0%  (bytes)
Wed Aug 14 13:14:11.895         Progress: 3647526/1378309050    0%  (bytes)
Wed Aug 14 13:14:57.028         Progress: 4984815/1378309050    0%  (bytes)
Wed Aug 14 13:15:01.015         Progress: 6202286/1378309050    0%  (bytes)
Wed Aug 14 13:15:05.051         Progress: 6797800/1378309050    0%  (bytes)
Wed Aug 14 13:15:08.402         Progress: 8133842/1378309050    0%  (bytes)
Wed Aug 14 13:15:12.712         Progress: 8872607/1378309050    0%  (bytes)
Wed Aug 14 13:15:15.259         Progress: 9964997/1378309050    0%  (bytes)
Wed Aug 14 13:15:19.266         Progress: 14684145/1378309050   1%  (bytes)
Wed Aug 14 13:15:22.364         Progress: 16154567/1378309050   1%  (bytes)
Wed Aug 14 13:15:29.627         Progress: 16754495/1378309050   1%  (bytes)
Wed Aug 14 13:15:35.225         Progress: 17726291/1378309050   1%  (bytes)
Wed Aug 14 13:15:39.447         Progress: 18333902/1378309050   1%  (bytes)
Wed Aug 14 13:15:43.717         Progress: 19055308/1378309050   1%  (bytes)
Wed Aug 14 13:15:46.481         Progress: 19305912/1378309050   1%  (bytes)
Wed Aug 14 13:15:49.902         Progress: 20038391/1378309050   1%  (bytes)
Wed Aug 14 13:15:53.868         Progress: 20389108/1378309050   1%  (bytes)
Wed Aug 14 13:15:58.578         Progress: 21127296/1378309050   1%  (bytes)
Wed Aug 14 13:16:03.706         Progress: 21837923/1378309050   1%  (bytes)
Wed Aug 14 13:16:56.512         Progress: 22092536/1378309050   1%  (bytes)
Wed Aug 14 13:16:59.035         Progress: 22583057/1378309050   1%  (bytes)
Wed Aug 14 13:17:02.313         Progress: 22817464/1378309050   1%  (bytes)
Wed Aug 14 13:17:05.044         Progress: 23178521/1378309050   1%  (bytes)
Wed Aug 14 13:17:26.023         Progress: 23434010/1378309050   1%  (bytes)
Wed Aug 14 13:17:39.161         Progress: 23670222/1378309050   1%  (bytes)
Wed Aug 14 13:17:42.846         Progress: 24049639/1378309050   1%  (bytes)
Wed Aug 14 13:17:59.125         Progress: 24284177/1378309050   1%  (bytes)
Wed Aug 14 13:18:02.722         Progress: 24515270/1378309050   1%  (bytes)
Wed Aug 14 13:18:06.827         Progress: 25018013/1378309050   1%  (bytes)
Wed Aug 14 13:18:09.234         Progress: 25253850/1378309050   1%  (bytes)
Wed Aug 14 13:18:14.282         Progress: 25617812/1378309050   1%  (bytes)
Wed Aug 14 13:18:46.296         Progress: 25983107/1378309050   1%  (bytes)
Wed Aug 14 13:18:51.303         Progress: 26604320/1378309050   1%  (bytes)
Wed Aug 14 13:18:55.500         Progress: 26971559/1378309050   1%  (bytes)
Wed Aug 14 13:19:00.656         Progress: 27444735/1378309050   1%  (bytes)
Wed Aug 14 13:19:04.100         Progress: 28064675/1378309050   2%  (bytes)

从 1% 到 2%大约需要4 分钟。这里肯定有严重的问题。

4

2 回答 2

17

这是一个非常古老的线程,但我最近遇到了类似的问题,可能是出于不同的原因,并且遇到了这个问题。

如果您碰巧在 AWS 上运行 mongo,请务必使用正确的实例和卷类型。

T 型实例有 CPU 积分,在运行大 mongorestore 进程时用完。您的恢复将快速开始,然后缓慢爬行......它永远不会真正停止,但需要几天才能完成。

如果您尝试使用磁性 EBS 卷来节省一些钱,那也是个坏主意。SC1 尤其错误,因为它在每秒操作数方面具有相似的信用... mongorestore 将消耗您所有的 IOPs 信用,无论您在几分钟内有多少,之后速度将降低到每秒 3 次操作并且恢复可能需要几周完成。

我希望这对某人有所帮助。

于 2016-05-24T07:44:35.693 回答
11

不幸的是,这些数字并不罕见。mongorestore 收集 3 亿个数据需要几天时间。

你基本上有两个选择。

首先,只需进行长时间运行的还原并在一夜之间运行它:

nohup mongorestore [args] &

其次,将文件复制到数据库目录本身(默认/data/db)而不是使用 mongodump/mongorestore。如果您可以关闭数据库一段时间,这是最好的。否则,您必须使用文件系统快照或类似的东西。有关更多官方 Mongo 简介,请参阅此内容:http: //docs.mongodb.org/manual/core/backups/

于 2013-12-19T23:11:57.340 回答