1

我在这里尝试实现的是获取特定节点的关系数量,而其他线程同时向它添加新关系。我使用TestGraphDatabaseFactory().newImpermanentDatabase()图形服务在单元测试中运行我的代码 。

我的代码由大约 50 个线程执行,它看起来像这样:

int numOfRels = 0;
try {
    Iterable<Relationship> rels = parentNode.getRelationships(RelTypes.RUNS, Direction.OUTGOING);
    while (rels.iterator().hasNext()) {
        numOfRels++;
        rels.iterator().next();
    }
}
catch(Exception e) {
    throw e;
}

// Enforce relationship limit
if (numOfRels > 10) {
    // do something
}

Transaction tx = graph.beginTx();
try {
    Node node = createMyNodeAndConnectToParentNode(...);

    tx.success();

    return node;
}
catch (Exception e) {
    tx.failure();
}
finally {
    tx.finish();
}

问题是有时我在上面的 try-catch 块(围绕getRelationships()的块)中得到一个“ArrayIndexOutOfBoundsException:1” 。如果我理解正确 Iterable 不是线程安全的并导致此问题。

我的问题是使用 Neo4j 的 Java API 迭代不断变化的关系和节点的最佳方法是什么?

我收到以下错误:

Exception in thread "Thread-14" org.neo4j.helpers.ThisShouldNotHappenError: Developer: Stefan/Jake claims that: A property key id disappeared under our feet
    at org.neo4j.kernel.impl.core.NodeProxy.setProperty(NodeProxy.java:188)
    at com.inbiza.connio.neo4j.server.extensions.graph.AppEntity.createMyNodeAndConnectToParentNode(AppEntity.java:546)
    at com.inbiza.connio.neo4j.server.extensions.graph.AppEntity.create(AppEntity.java:305)
    at com.inbiza.connio.neo4j.server.extensions.TestEmbeddedConnioGraph$appCreatorThread.run(TestEmbeddedConnioGraph.java:61)
    at java.lang.Thread.run(Thread.java:722)
Exception in thread "Thread-92" java.lang.ArrayIndexOutOfBoundsException: 1
    at org.neo4j.kernel.impl.core.RelationshipIterator.fetchNextOrNull(RelationshipIterator.java:72)
    at org.neo4j.kernel.impl.core.RelationshipIterator.fetchNextOrNull(RelationshipIterator.java:36)
    at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:55)
    at com.inbiza.connio.neo4j.server.extensions.graph.AppEntity.create(AppEntity.java:243)
    at com.inbiza.connio.neo4j.server.extensions.TestEmbeddedConnioGraph$appCreatorThread.run(TestEmbeddedConnioGraph.java:61)
    at java.lang.Thread.run(Thread.java:722)
Exception in thread "Thread-12" java.lang.ArrayIndexOutOfBoundsException: 1
    at org.neo4j.kernel.impl.core.RelationshipIterator.fetchNextOrNull(RelationshipIterator.java:72)
    at org.neo4j.kernel.impl.core.RelationshipIterator.fetchNextOrNull(RelationshipIterator.java:36)
    at org.neo4j.helpers.collection.PrefetchingIterator.hasNext(PrefetchingIterator.java:55)
    at com.inbiza.connio.neo4j.server.extensions.graph.AppEntity.create(AppEntity.java:243)
    at com.inbiza.connio.neo4j.server.extensions.TestEmbeddedConnioGraph$appCreatorThread.run(TestEmbeddedConnioGraph.java:61)
    at java.lang.Thread.run(Thread.java:722)
Exception in thread "Thread-93" java.lang.ArrayIndexOutOfBoundsException
Exception in thread "Thread-90" java.lang.ArrayIndexOutOfBoundsException

下面是负责创建节点的方法:

static Node createMyNodeAndConnectToParentNode(GraphDatabaseService graph, final Node ownerAccountNode, final String suggestedName, Map properties) {

  final String accountId = checkNotNull((String)ownerAccountNode.getProperty("account_id"));

  Node appNode = graph.createNode();
  appNode.setProperty("urn_name", App.composeUrnName(accountId, suggestedName.toLowerCase().trim()));

  int nextId = nodeId.addAndGet(1); // I normally use getOrCreate idiom but to simplify I replaced it with an atomic int - that would do for testing 

  String urn = App.composeUrnUid(accountId,  nextId);
  appNode.setProperty("urn_uid", urn);
  appNode.setProperty("id", nextId);
  appNode.setProperty("name", suggestedName);

  Index<Node> indexUid =  graph.index().forNodes("EntityUrnUid");
  indexUid.add(appNode, "urn_uid", urn);

  appNode.addLabel(LabelTypes.App);

  appNode.setProperty("version", properties.get("version"));
  appNode.setProperty("description", properties.get("description"));

  Relationship rel = ownerAccountNode.createRelationshipTo(appNode, RelTypes.RUNS);
  rel.setProperty("date_created", fmt.print(new DateTime()));

  return appNode;
}

我在看 org.neo4j.kernel.impl.core.RelationshipIterator.fetchNextOrNull()

看起来我的测试生成了一个条件,其中 else if ( (status = fromNode.getMoreRelationships( nodeManager )).loaded() || lastTimeILookedThereWasMoreToLoad ) 没有执行,并且currentTypeIterator状态在两者之间发生了变化。

RelIdIterator currentTypeIterator = rels[currentTypeIndex];  //<-- this is where is crashes
do
{
  if ( currentTypeIterator.hasNext() )
  ...
  ... 

  while ( !currentTypeIterator.hasNext() )
  {
    if ( ++currentTypeIndex < rels.length )
    {
        currentTypeIterator = rels[currentTypeIndex];
    }
    else if ( (status = fromNode.getMoreRelationships( nodeManager )).loaded()
            // This is here to guard for that someone else might have loaded
            // stuff in this relationship chain (and exhausted it) while I
            // iterated over my batch of relationships. It will only happen
            // for nodes which have more than <grab size> relationships and
            // isn't fully loaded when starting iterating.
            || lastTimeILookedThereWasMoreToLoad )
    {
        ....
    }
  }
} while ( currentTypeIterator.hasNext() );

我还测试了几个锁定场景。下面的一个解决了这个问题。不确定每次基于此迭代关系时是否应该使用锁。

Transaction txRead = graph.beginTx();
try {
  txRead.acquireReadLock(parentNode);

  long numOfRels = 0L;
  Iterable<Relationship> rels = parentNode.getRelationships(RelTypes.RUNS, Direction.OUTGOING);
  while (rels.iterator().hasNext()) {
    numOfRels++;
    rels.iterator().next();
  }

  txRead.success();
}
finally {
  txRead.finish();
}

我对 Neo4j 及其源代码库非常陌生;只是作为我们产品的潜在数据存储进行测试。如果有人从里到外了解 Neo4j 解释这里发生的事情,我将不胜感激。

4

2 回答 2

2

这是一个错误。此拉取请求中捕获了修复:https ://github.com/neo4j/neo4j/pull/1011

于 2013-07-28T23:42:08.940 回答
0

好吧,我认为这是一个错误。Iterable返回的 by意味着getRelationships()是不可变的。当这个方法被调用时,Nodes直到那个时刻所有可用的都将在迭代器中可用。(您可以从org.neo4j.kernel.IntArrayIterator验证这一点)

我尝试通过让 250 个线程尝试将关系从一个节点插入到其他节点来复制它。并且有一个主线程在第一个节点的迭代器上循环。仔细分析,迭代器只包含getRelationship()上次调用时添加的关系。这个问题从来没有出现在我身上。

您能否输入您的完整代码,IMO 可能会出现一些愚蠢的错误。它不会发生的原因是在添加关系时写入锁已经到位,因此读取是同步的。

于 2013-07-26T12:53:42.903 回答