当尝试使用 SparkGraphComputer 计算集群上泰坦图上的顶点数时，我收到一个错误，我不知道如何处理。我在我的代码和集群上使用 tinkerpop 3.1.1-incubating 和 Titan 1.1.0-SNAPSHOT 我安装了 datastax community edition 2.1.11 和 spark 1.5.2-bin-hadoop2.6
我整理了一个最小的 Java 示例来重现我的问题：
private void strippedDown() {
// a normal titan cluster
String titanClusterConfig = "titan-cassandra-test-cluster.properties";
// a hadoop graph with cassandra as input and gryo as output
String sparkClusterConfig = "titan-cassandra-test-spark.properties";
String edgeLabel = "blank";
// add a graph
int n = 100;
Graph titanGraph = GraphFactory.open(titanClusterConfig);
Vertex superNode = titanGraph.addVertex(T.label, String.valueOf(0));
for (int i=1;i<n;i++) {
Vertex currentNode = titanGraph.addVertex(T.label, String.valueOf(i));
currentNode.addEdge(edgeLabel,superNode);
}
titanGraph.tx().commit();
//count with titan
Long count = titanGraph.traversal().V().count().next();
System.out.println("The number of vertices in the graph is: "+count);
// count the graph using titan graph computer
count = titanGraph.traversal(GraphTraversalSource.computer(FulgoraGraphComputer.class)).V().count().next();
System.out.println("The number of vertices in the graph is: "+count);
// count the graph using spark graph computer
Graph sparkGraph = GraphFactory.open(sparkClusterConfig);
count = sparkGraph.traversal(GraphTraversalSource.computer(SparkGraphComputer.class)).V().count().next();
System.out.println("The number of vertices in the graph is: "+count);
}
使用 OLTP 和使用带有 FulgoraGraphComputer 的 OLAP 的计数返回正确答案。然而，使用 SparkGraphComputer 的 OLAP 计数会抛出 org.apache.spark.SparkException: Job aborted due to stage failure：
有趣的是，如果我通过与 Titan 打包在一起的 gremlin 控制台运行类似的脚本，我会因为似乎是相同的算法而得到不同的错误：
graph = GraphFactory.open('titan-cassandra-test-cluster.properties')
graph.addVertex(T.label,"0")
graph.addVertex(T.label,"1")
graph.addVertex(T.label,"2")
graph.tx().commit()
sparkGraph = GraphFactory.open('titan-cassandra-test-spark.properties')
sparkGraph.traversal(computer(SparkGraphComputer)).V().count()
这会抛出org.apache.thrift.protocol.TProtocolException: Required field 'keyspace' was not present! Struct: set_keyspace_args(keyspace:null)两次，但会完成并返回 0，这是不正确的。
我知道邮件列表中有这篇文章，但我无法理解它或解决问题。谁能向我解释发生了什么以及如何解决这个问题？我在下面粘贴了我的配置。
gremlin.graph=com.thinkaurelius.titan.core.TitanFactory
storage.backend=cassandrathrift
storage.hostname=node1
storage.cassandra.keyspace=mindmapstest
storage.cassandra.replication-factor=3
cache.db-cache = true
cache.db-cache-clean-wait = 20
cache.db-cache-time = 180000
cache.db-cache-size = 0.5
和
gremlin.graph=org.apache.tinkerpop.gremlin.hadoop.structure.HadoopGraph
gremlin.hadoop.graphInputFormat=com.thinkaurelius.titan.hadoop.formats.cassandra.CassandraInputFormat
gremlin.hadoop.graphOutputFormat=org.apache.hadoop.mapreduce.lib.output.NullOutputFormat
gremlin.hadoop.jarsInDistributedCache=true
gremlin.hadoop.inputLocation=none
gremlin.hadoop.outputLocation=none
####################################
# Cassandra Cluster Config #
####################################
titanmr.ioformat.conf.storage.backend=cassandrathrift
titanmr.ioformat.conf.storage.cassandra.keyspace=mindmapstest
titanmr.ioformat.conf.storage.hostname=node1,node2,node3
####################################
# SparkGraphComputer Configuration #
####################################
spark.master=spark://node1:7077
spark.executor.memory=250m
spark.serializer=org.apache.spark.serializer.KryoSerializer
####################################
# Apache Cassandra InputFormat configuration
####################################
cassandra.input.partitioner.class=org.apache.cassandra.dht.Murmur3Partitioner
编辑：此脚本将重现错误
graph = TitanFactory.open('titan-cassandra-test-cluster.properties')
superNode = graph.addVertex(T.label,"0")
for(i in 1..100) {
currentNode = graph.addVertex(T.label,i.toString())
currentNode.addEdge("blank",superNode)
}
graph.tx().commit()
graph.traversal().V().count()
graph.traversal(computer()).V().count()
sparkGraph = GraphFactory.open('titan-cassandra-test-spark.properties')
sparkGraph.traversal(computer(SparkGraphComputer)).V().count()