spark-shell --packages com.databricks:spark-csv_2.11:1.5.0
spark > val flightData2015 = spark.read.option("inferSchema", "true").option("header","true").csv("/Users/me/Desktop/2015-summary.csv")
我得到了这个例外。
java.io.IOException：无法连接到 org.apache.spark.network.client.TransportClientFactory 的 org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:232) 的 /192.168.1.112:51351。 createClient(TransportClientFactory.java:182) at org.apache.spark.rpc.netty.NettyRpcEnv.downloadClient(NettyRpcEnv.scala:366) at org.apache.spark.rpc.netty.NettyRpcEnv.openChannel(NettyRpcEnv.scala:332)在 org.apache.spark.util.Utils$.doFetchFile(Utils.scala:654) 在 org.apache.spark.util.Utils$.fetchFile(Utils.scala:480) 在 org.apache.spark.executor.Executor $$anonfun$org$apache$spark$executor$Executor$$updateDependencies$5.apply(Executor.scala:696) at org.apache.spark.executor.Executor$$anonfun$org$apache$spark$executor$Executor$ $updateDependencies$5.apply(Executor.scala:688）在 scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733) 在 scala.collection.Iterator$class.foreach(Iterator.scala:893) 在 scala.collection.AbstractIterator.foreach (Iterator.scala:1336) 在 scala.collection.IterableLike$class.foreach(IterableLike.scala:72) 在 scala.collection.AbstractIterable.foreach(Iterable.scala:54) 在 scala.collection.TraversableLike$WithFilter.foreach( TraversableLike.scala:732) at org.apache.spark.executor.Executor.org$apache$spark$executor$Executor$$updateDependencies(Executor.scala:688) at org.apache.spark.executor.Executor$TaskRunner.run (Executor.scala:308) 在 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 在 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 在 java.lang.Thread.run(Thread.java:748) 引起：io.netty.channel.AbstractChannel$AnnotatedConnectException：操作超时：/192.168.1.112:51351 在 sun.nio.ch.SocketChannelImpl.checkConnect （本机方法）在 sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) 在 io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:257) 在 io.netty.channel.nio .AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:291) 在 io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:631) 在 io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:566) ) 在 io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java: io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:480)442) 在 io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) 在 io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144) ... 还有 1 个