1

我正在尝试让 CaffeOnSpark 在本地运行，我在 CaffeOnSpark wiki 上遵循此过程：https ://github.com/yahoo/CaffeOnSpark/wiki/GetStarted_local

前 4 项对我有好处。但是，当我在第 5 步中进行 Make 构建时。我在 protoc 上收到此错误：

整个错误堆栈：

jiaqi@jiaqi-ThinkPad-L450:~/github/CaffeOnSpark$ make build
cd caffe-public; make proto; make -j4 -e distribute; cd ..
make[1]: Entering directory '/home/jiaqi/github/CaffeOnSpark/caffe-public'
make[1]: Nothing to be done for 'proto'.
make[1]: Leaving directory '/home/jiaqi/github/CaffeOnSpark/caffe-public'
make[1]: Entering directory '/home/jiaqi/github/CaffeOnSpark/caffe-public'
CXX/LD -o .build_release/tools/convert_imageset.bin
CXX/LD -o .build_release/tools/upgrade_net_proto_binary.bin
CXX/LD -o .build_release/tools/compute_image_mean.bin
CXX/LD -o .build_release/tools/caffe.bin
.build_release/tools/compute_image_mean.o: In function `std::string* google::MakeCheckOpString<int, int>(int const&, int const&, char const*)':
compute_image_mean.cpp:(.text._ZN6google17MakeCheckOpStringIiiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringIiiEEPSsRKT_RKT0_PKc]+0x50): undefined reference to `google::base::CheckOpMessageBuilder::NewString()'
.build_release/tools/compute_image_mean.o: In function `std::string* google::MakeCheckOpString<unsigned long, int>(unsigned long const&, int const&, char const*)':
compute_image_mean.cpp:(.text._ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc]+0x50): undefined reference to `google::base::CheckOpMessageBuilder::NewString()'
.build_release/tools/compute_image_mean.o: In function `main':
compute_image_mean.cpp:(.text.startup+0x168): undefined reference to `google::SetUsageMessage(std::string const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::Status::ToString() const'
collect2: error: ld returned 1 exit status
Makefile:613: recipe for target '.build_release/tools/compute_image_mean.bin' failed
make[1]: *** [.build_release/tools/compute_image_mean.bin] Error 1
make[1]: *** Waiting for unfinished jobs....
.build_release/tools/convert_imageset.o: In function `std::string* google::MakeCheckOpString<unsigned long, int>(unsigned long const&, int const&, char const*)':
convert_imageset.cpp:(.text._ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc]+0x50): undefined reference to `google::base::CheckOpMessageBuilder::NewString()'
.build_release/tools/convert_imageset.o: In function `main':
convert_imageset.cpp:(.text.startup+0x347): undefined reference to `google::SetUsageMessage(std::string const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::Status::ToString() const'
collect2: error: ld returned 1 exit status
Makefile:613: recipe for target '.build_release/tools/convert_imageset.bin' failed
make[1]: *** [.build_release/tools/convert_imageset.bin] Error 1
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)'
.build_release/lib/libcaffe.so: undefined reference to `google::base::CheckOpMessageBuilder::NewString()'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::Status::ToString() const'
collect2: error: ld returned 1 exit status
Makefile:613: recipe for target '.build_release/tools/upgrade_net_proto_binary.bin' failed
make[1]: *** [.build_release/tools/upgrade_net_proto_binary.bin] Error 1
.build_release/tools/caffe.o: In function `std::string* google::MakeCheckOpString<unsigned long, int>(unsigned long const&, int const&, char const*)':
caffe.cpp:(.text._ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc[_ZN6google17MakeCheckOpStringImiEEPSsRKT_RKT0_PKc]+0x50): undefined reference to `google::base::CheckOpMessageBuilder::NewString()'
.build_release/tools/caffe.o: In function `main':
caffe.cpp:(.text.startup+0x50): undefined reference to `google::SetVersionString(std::string const&)'
caffe.cpp:(.text.startup+0x83): undefined reference to `google::SetUsageMessage(std::string const&)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::DB::Open(leveldb::Options const&, std::string const&, leveldb::DB**)'
.build_release/lib/libcaffe.so: undefined reference to `cv::imdecode(cv::_InputArray const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `leveldb::Status::ToString() const'
collect2: error: ld returned 1 exit status
Makefile:613: recipe for target '.build_release/tools/caffe.bin' failed
make[1]: *** [.build_release/tools/caffe.bin] Error 1
make[1]: Leaving directory '/home/jiaqi/github/CaffeOnSpark/caffe-public'
export LD_LIBRARY_PATH="/home/y/lib64:/home/y/lib64/mkl/intel64:/home/jiaqi/github/CaffeOnSpark/caffe-public/distribute/lib:/home/jiaqi/github/CaffeOnSpark/caffe-distri/distribute/lib:/usr/lib64:/lib64 "; mvn -B package
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for com.yahoo.ml:caffe-grid:jar:0.1-SNAPSHOT
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] caffe
[INFO] caffe-distri
[INFO] caffe-grid
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building caffe 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building caffe-distri 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (proto) @ caffe-distri ---
[INFO] Executing tasks

protoc:
     [exec] make[1]: Entering directory '/home/jiaqi/github/CaffeOnSpark/caffe-distri'
     [exec] make[1]: Leaving directory '/home/jiaqi/github/CaffeOmake[1]: *** No rule to make target '../caffe-public/distribute/proto/caffe.proto', needed by 'src/main/java/caffe/Caffe.java'. nSpark/caffe-distri'
     [exec]  Stop.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] caffe .............................................. SUCCESS [  0.019 s]
[INFO] caffe-distri ....................................... FAILURE [  2.425 s]
[INFO] caffe-grid ......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.144 s
[INFO] Finished at: 2016-05-13T12:33:40+08:00
[INFO] Final Memory: 9M/238M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...<exec failonerror="true" dir="/home/jiaqi/github/CaffeOnSpark/caffe-distri" executable="make">... @ 5:97 in /home/jiaqi/github/CaffeOnSpark/caffe-distri/target/antrun/build-protoc.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :caffe-distri
Makefile:9: recipe for target 'build' failed
make: *** [build] Error 1

似乎 protoc 找不到它的规则。你有没有遇到过这个问题？请帮帮我！

我的协议版本是 2.5.0。

4

3 回答 3

2

我之前遇到过 CaffeOnSpark 的构建错误。我知道这篇文章很旧，但无论如何它可能会对某人有所帮助。

从日志中我认为您有一些依赖性问题。CaffeOnSpark 需要满足 Caffe 的所有依赖项。所以我想问的第一个问题是 Caffe 是否正常工作（如果已安装）。

  1. 如果没有安装 Caffe，您可以尝试安装 Caffe 并通过打开 python 并输入来验证它的工作import caffe。如果 Caffe 不工作，请转到步骤 5。

  2. 如果 caffe 正常工作，请验证您是否执行了下载 git 的确切命令：

    git clone https://github.com/yahoo/CaffeOnSpark.git --recursive

  3. 然后请检查你Makefile.config的CaffeOnSpark/caffe-public。确保正确给出所有标志。检查它们后，运行这些：

    cd CaffeOnSpark make clean make build

  4. 如果您遇到此错误：

    make[1]: Leaving directory `/home/das/CaffeOnSpark/caffe-public'
export LD_LIBRARY_PATH="/usr/local/cuda-7.5/lib64:/home/das/CaffeOnSpark/caffe-public/distribute/lib:/home/das/CaffeOnSpark/caffe-distri/distribute/lib:/usr/lib64:/lib64 "; mvn -B package -DskipTests
/bin/sh: 1: mvn: not found
make: *** [build] Error 127

    运行这个命令来安装 Maven：

    sudo apt-get install maven

  5. 如果您仍然收到错误，您可能需要重新安装/再次检查依赖项。我会发布说明以防万一。请注意，其中许多实际上并不是必需的。我安装这些来设置我的深度学习机器：

    sudo apt-get update
sudo apt-get install -y build-essential cmake g++ gfortran git pkg-config python-dev software-properties-common
sudo apt-get install -y python-pip python-dev python-numpy python-scipy python-nose python-h5py python-skimage python-matplotlib python-pandas python-sklearn python-sympy
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler libgflags-dev libgoogle-glog-dev liblmdb-dev

    并从 Caffe 根文件夹安装要求：

    sudo pip install -r python/requirements.txt

  6. 如果存在 BLAS 问题，请正确安装：

    git clone https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS
make FC=gfortran -j 4
sudo make PREFIX=/usr/local install

  7. 请确保该.bashrc文件的所有路径都正确。

    export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
export CAFFE_ROOT=/home/das/Caffe
export PYTHONPATH=$CAFFE_ROOT/python:$PYTHONPATH
export CAFFE_ON_SPARK=/home/das/CaffeOnSpark
export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export SPARK_HOME=/usr/local/spark
export LD_LIBRARY_PATH=${CAFFE_ON_SPARK}/caffe-public/distribute/lib:${CAFFE_ON_SPARK}/caffe-distri/distribute/lib

..如果仍然有错误，请将其发布，以便我们进行调查。

谢谢。

于 2016-08-23T02:27:02.703 回答
0

早期应该有一条错误消息，指出找不到 blas 库。默认情况下，Caffe 使用可以在 Makefile.config: 中找到的 atlas 库BLAS := atlas，如果你安装了其他 blas 库，比如 mkl，你应该使用BLAS := mkl

于 2016-05-12T19:27:28.733 回答
-1

我尝试在 Ubuntu 16.04 上编译 CaffeOnSpark，在GetStarted_standalone参考之前，我从相应的网站安装了 maven-3.0.5 和 hadoop-2.7.1 和 spark-2.0.0-bin-hadoop2.7。我遇到了类似的问题，错误如下：

[INFO] Compiling 10 source files to /home/bdg/usr/CaffeOnSpark/caffe-distri/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/bdg/usr/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[9919,31] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf
[ERROR] /home/bdg/usr/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[8280,24] cannot find symbol
  symbol:   class ProtocolStringList
  location: package com.google.protobuf
...
[ERROR] /home/bdg/usr/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[5655,26] cannot find symbol
  symbol:   method getUnmodifiableView()
  location: variable input_ of type com.google.protobuf.LazyStringList
[ERROR] /home/bdg/usr/CaffeOnSpark/caffe-distri/src/main/java/caffe/Caffe.java:[6395,26] cannot find symbol
  symbol:   method getUnmodifiableView()
  location: variable input_ of type com.google.protobuf.LazyStringList
...
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :caffe-distri
Makefile:19: recipe for target 'build' failed
make: *** [build] Error 1
于 2016-11-25T07:43:01.877 回答