0

在我的一个导入脚本完成导入所有数据后，我尝试重新启动它以获取任何更新的数据。它做的第一件事是获取最近更新的记录：

db.select().from(newClass).order('updatedAt desc').limit(1).one()

但是，这导致我的节点脚本出现以下错误：

Possibly unhandled OrientDB.RequestError: Java heap space
    at Operation.parseError (/Users/gsquare567/node_modules/oriento/lib/transport/binary/protocol/operation.js:779:13)
    at Operation.consume (/Users/gsquare567/node_modules/oriento/lib/transport/binary/protocol/operation.js:369:35)
    at Connection.process (/Users/gsquare567/node_modules/oriento/lib/transport/binary/connection.js:324:17)
    at Connection.handleSocketData (/Users/gsquare567/node_modules/oriento/lib/transport/binary/connection.js:250:17)
    at Socket.emit (events.js:95:17)
    at Socket.<anonymous> (_stream_readable.js:748:14)
    at Socket.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:410:10)
    at emitReadable (_stream_readable.js:406:5)
    at readableAddChunk (_stream_readable.js:168:9)

我收到以下服务器输出：

java.lang.OutOfMemoryError: GC overhead limit exceeded
Dumping heap to java_pid1694.hprof ...
Heap dump file created [2055557443 bytes in 37.799 secs]
Error on fetching record during browsing. The record has been skipped
Error on retrieving record #11:1023466 (cluster: user)
-> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:252)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:1017)
-> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:65)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:264)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:40)
-> com.orientechnologies.orient.core.iterator.OIdentifiableIterator.readCurrentRecord(OIdentifiableIterator.java:285)
-> com.orientechnologies.orient.core.iterator.ORecordIteratorClusters.hasNext(ORecordIteratorClusters.java:139)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:913)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:397)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:358)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:60)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:94)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:83)
-> com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1181)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:340)
-> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169)
-> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45)
GC overhead limit exceeded

由于以下原因，我的另一个同时运行的导入脚本也停止了：

"error":{"name":"OrientDB.RequestError","message":"Java heap space","data":{},"previous":[],"id":1,"type":"java.lang.OutOfMemoryError","hasMore":0}

再次尝试运行原始脚本后，我在服务器中得到以下输出：

Exception in thread "OrientDB WAL Flush Task (pumpup)" Error on client connection
Java heap spacejava.lang.OutOfMemoryError: Java heap space

节点脚本输出：

Possibly unhandled OrientDB.RequestError: Java heap space
    at Operation.parseError (/Users/gsquare567/node_modules/oriento/lib/transport/binary/protocol/operation.js:779:13)
    at Operation.consume (/Users/gsquare567/node_modules/oriento/lib/transport/binary/protocol/operation.js:369:35)
    at Connection.process (/Users/gsquare567/node_modules/oriento/lib/transport/binary/connection.js:324:17)
    at Connection.handleSocketData (/Users/gsquare567/node_modules/oriento/lib/transport/binary/connection.js:250:17)
    at Socket.emit (events.js:95:17)
    at Socket.<anonymous> (_stream_readable.js:748:14)
    at Socket.emit (events.js:92:17)
    at emitReadable_ (_stream_readable.js:410:10)
    at emitReadable (_stream_readable.js:406:5)
    at readableAddChunk (_stream_readable.js:168:9)

编辑

将内存限制增加到 2GB 后，我能够插入 5M 记录（而不是之前的 2M 记录），但仍然遇到此错误。

GC overhead limit exceeded
-> com.orientechnologies.orient.core.db.raw.ODatabaseRaw.read(ODatabaseRaw.java:252)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.executeReadRecord(ODatabaseRecordAbstract.java:1017)
-> com.orientechnologies.orient.core.tx.OTransactionNoTx.loadRecord(OTransactionNoTx.java:65)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:264)
-> com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.load(ODatabaseRecordTx.java:40)
-> com.orientechnologies.orient.core.iterator.OIdentifiableIterator.readCurrentRecord(OIdentifiableIterator.java:285)
-> com.orientechnologies.orient.core.iterator.ORecordIteratorClusters.hasNext(ORecordIteratorClusters.java:139)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:913)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:397)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:358)
-> com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:60)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.executeCommand(OStorageEmbedded.java:94)
-> com.orientechnologies.orient.core.storage.OStorageEmbedded.command(OStorageEmbedded.java:83)
-> com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:59)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.command(ONetworkProtocolBinary.java:1181)
-> com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:340)
-> com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169)
-> com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45)
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "Timer-0"
4

1 回答 1

2

这里的问题是您正在尝试对所有 5M 记录进行排序。

该操作必须将整个数据集加载到内存中才能对其进行排序（实际上我们有计划对其进行优化以避免在这种情况下出现 OOM，但尚未实现）。

因此，即使您指定限制 1，您也会加载整批记录，查询会很慢并且会消耗大量内存。

要优化该查询，请在updatedAt字段上建立索引。

于 2014-07-08T06:57:55.417 回答