1 
Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)
        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:61)
Caused by: java.lang.RuntimeException: native-lzo library not available
        at com.hadoop.compression.lzo.LzoCodec.getCompressorType(LzoCodec.java:155)
        at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:150)
        at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:168)
        at org.apache.parquet.hadoop.CodecFactory$HeapBytesCompressor.<init>(CodecFactory.java:144)
        at org.apache.parquet.hadoop.CodecFactory.createCompressor(CodecFactory.java:206)
        at org.apache.parquet.hadoop.CodecFactory.getCompressor(CodecFactory.java:189)
        at org.apache.parquet.hadoop.ParquetWriter.<init>(ParquetWriter.java:287)
        at org.apache.parquet.hadoop.ParquetWriter$Builder.build(ParquetWriter.java:569)

我正在尝试在我的镶木地板中使用 lzo 压缩编解码器。我不需要 Hadoop，所以我的 Hadoop_Home 路径中只有 hadoop.dll 和 winutils.exe。在我的 Windows 系统中，我尝试按照文档中的说明构建 lzo2.dll，但不知道如何使用它。我浏览了 twitter fork hadoop-lzo，但 Windows 的说明不清楚。请告诉我如何让这个编解码器在 Windows 中运行。我下载了hadoop-3.2.1、lzo-2.1.0、hadoop-lzo。

在 hadoop-lzo 中使用 mvn clean install 会给我 gplcompression.dll not found 错误。

[exec] "P:\hadoop\hadoop-lzo\src\main\native\gplcompression.sln" (default target) (1) ->
     [exec] "P:\hadoop\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj" (default target) (2) ->
     [exec] (Build target) -> 
     [exec]   P:\hadoop\hadoop-lzo\src\main\native\gplcompression.vcxproj.metaproj : warning MSB4078: The project file "gplcompression.vcxproj" is not supported by MSBuild and cannot be built.
     [exec] 
     [exec]     1 Warning(s)
     [exec]     0 Error(s)
     [exec] 
     [exec] Time Elapsed 00:00:01.57
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  15.603 s
[INFO] Finished at: 2020-05-28T17:16:24+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (build-native-win) on project hadoop-lzo: An Ant BuildException has occured: Warning: Could not find file P:\hadoop\hadoop-lzo\target\native\Windows_NT-${env.PLATFORM}\gplcompression.dll to copy.
[ERROR] around Ant part ...<copy file="${build.native}/gplcompression.dll" todir="${build.native}/lib"/>... @ 19:80 in P:\hadoop\hadoop-lzo\target\antrun\build-build-native-win.xml
[ERROR] -> [Help 1]
4

1 回答 1

0

它在 Windows 中不起作用。如果您尝试为 Windows 构建 Hadoop-LZO，那么您在每一步都会遇到构建错误。问题出现是因为我没有包含正确构建的 Hadoop-LZO (twitter)。LZO 有一些本机（系统相关）代码和许可问题，因此您需要按照“https://www.oberhumer.com/opensource/lzo/”中的 LZO 使用“ https://www.oberhumer.com/opensource/lzo/ ”中的说明构建特定的 Hadoop-LZO 。 /github.com/twitter/hadoop-lzo “。我为 linux 构建了它并在 linux 中运行了代码，它工作正常。我无法为 Windows 成功构建它。我面临各种错误，我认为 Hadoop-LZO 中的 Windows 本机代码存在问题。

于 2020-06-04T15:02:54.787 回答