0

我正在尝试使 Mahout 0.13.0 与 spark 1.6.3 一起使用，

  • 我已经有 spark 1.6.3 和 Hadoop 2.7 工作。

  • 我从主页mahout_download下载了最后一个版本。

  • 在 /opt/mahout 上解包。

  • 尝试从网络上执行 spark-shell 上的示例。

在 scala 提示符下输入

MASTER=local mahout spark-shell

执行：

:load /opt/mahout/examples/bin/SparseSparseDrmTimer.mscala
timeSparseDRMMMul(1000,1000,1000,1,.02,1234L)

我得到跟随错误：

18/08/08 15:22:23 ERROR executor.Executor: Exception in task 0.0 in stage 3.0 (TID 3)
java.lang.UnsatisfiedLinkError: no jniViennaCL in java.library.path
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
    at java.lang.Runtime.loadLibrary0(Runtime.java:870)
    at java.lang.System.loadLibrary(System.java:1122)
    at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:726)
    at org.bytedeco.javacpp.Loader.load(Loader.java:501)
    at org.bytedeco.javacpp.Loader.load(Loader.java:434)
    at org.apache.mahout.viennacl.opencl.javacpp.Context$.loadLib(Context.scala:63)
    at org.apache.mahout.viennacl.opencl.javacpp.Context$.<init>(Context.scala:65)
    at org.apache.mahout.viennacl.opencl.javacpp.Context$.<clinit>(Context.scala)
    at org.apache.mahout.viennacl.opencl.GPUMMul$.org$apache$mahout$viennacl$opencl$GPUMMul$$gpuSparseRWRW(GPUMMul.scala:282)
    at org.apache.mahout.viennacl.opencl.GPUMMul$.org$apache$mahout$viennacl$opencl$GPUMMul$$jvmSparseRWCW(GPUMMul.scala:343)
    at org.apache.mahout.viennacl.opencl.GPUMMul$$anonfun$18.apply(GPUMMul.scala:86)
    at org.apache.mahout.viennacl.opencl.GPUMMul$$anonfun$18.apply(GPUMMul.scala:86)
    at org.apache.mahout.viennacl.opencl.GPUMMul$.apply(GPUMMul.scala:127)
    at org.apache.mahout.viennacl.opencl.GPUMMul$.apply(GPUMMul.scala:33)
    at org.apache.mahout.math.scalabindings.RLikeMatrixOps.$percent$times$percent(RLikeMatrixOps.scala:37)
    at org.apache.mahout.sparkbindings.blas.ABt$.org$apache$mahout$sparkbindings$blas$ABt$$mmulFunc$1(ABt.scala:98)
    at org.apache.mahout.sparkbindings.blas.ABt$$anonfun$6.apply(ABt.scala:113)
    at org.apache.mahout.sparkbindings.blas.ABt$$anonfun$6.apply(ABt.scala:113)
    at org.apache.mahout.sparkbindings.blas.ABt$$anonfun$pairwiseApply$1.apply(ABt.scala:209)
    at org.apache.mahout.sparkbindings.blas.ABt$$anonfun$pairwiseApply$1.apply(ABt.scala:209)
    at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
    at org.apache.spark.util.collection.ExternalSorter.insertAll(ExternalSorter.scala:191)
    at org.apache.spark.shuffle.sort.SortShuffleWriter.write(SortShuffleWriter.scala:64)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:73)
    at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:41)
    at org.apache.spark.scheduler.Task.run(Task.scala:89)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:227)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsatisfiedLinkError: /tmp/javacpp106197113868261/libjniViennaCL.so: libOpenCL.so.1: cannot open shared object file: No such file or directory
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
    at java.lang.Runtime.load0(Runtime.java:809)
    at java.lang.System.load(System.java:1086)
    at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:709)
    ... 27 more

搜索问题后，我安装了网上显示的 ViennaCL 库。

mkdir ~/tmp
cd ~/tmp && git clone https://github.com/viennacl/viennacl-dev.git
cp -r viennacl/ /usr/local/
cp -r CL/ /usr/local/

另外，我尝试在路径上复制：/usr/local/include/

但是什么都没有改变...

4

1 回答 1

0

使用 ViennaCL 时，您必须从源代码构建。请尝试提取 mahout 0.13.0 标签并从源代码构建（ -Pviennacl激活配置文件）。

于 2018-08-08T19:18:27.330 回答