0

我想使用我想从 cloudera 站点下载的 hadoop cdh jar 运行字数统计程序。要连接到 cloudera 站点并下载,我使用以下 pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>uk.co.arunhorne</groupId>
    <artifactId>hadoop-wordcount</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>

    <properties>
        <jdkLevel>1.6</jdkLevel>
        <requiredMavenVersion>[2.1,)</requiredMavenVersion>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.build.outputEncoding>UTF-8</project.build.outputEncoding>
    </properties>

    <repositories>
        <repository>
            <id>cloudera-releases</id>
            <!-- <url>http://10.118.200.138/cloudera-jars/repodata/</url> -->
            <!--<url>https://repository.cloudera.com/content/repositories/releases/</url> -->
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>0.20.2-cdh3u3</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.1</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>2.3.1</version>
                <configuration>
                    <source>${jdkLevel}</source>
                    <target>${jdkLevel}</target>
                    <showDeprecation>true</showDeprecation>
                    <showWarnings>true</showWarnings>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.2.1</version>
                <configuration>
                    <descriptors>
                        <descriptor>src/main/assembly/hadoop-job.xml</descriptor>
                    </descriptors>
                    <archive>
                        <manifest>
                            <mainClass>uk.co.arunhorne.mapred.wordcount.WordCount</mainClass>
                        </manifest>
                    </archive>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

</project>

但是当我构建它时,我在控制台中收到以下错误:-

[WARNING] 
[WARNING] Some problems were encountered while building the effective settings
[WARNING] Unrecognised tag: 'repositories' (position: START_TAG seen ...</offline>\r\n<repositories>... @9:15)  @ D:\installations\apache-maven-3.0.4\conf\settings.xml, line 9, column 15
[WARNING] Unrecognised tag: 'repositories' (position: START_TAG seen ...</offline>\r\n<repositories>... @9:15)  @ C:\Users\Sailaja_Sahoo\.m2\settings.xml, line 9, column 15
[WARNING] 
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building hadoop-wordcount 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/hadoop/hadoop-core/0.20.2-cdh3u3/hadoop-core-0.20.2-cdh3u3.pom
Feb 4, 2013 5:55:04 PM org.apache.maven.wagon.providers.http.httpclient.client.protocol.RequestProxyAuthentication process
SEVERE: Proxy authentication error: Invalid name provided (Mechanism level: Could not load configuration file C:\Windows\krb5.ini (The system cannot find the file specified))
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.152s
[INFO] Finished at: Mon Feb 04 17:55:04 IST 2013
[INFO] Final Memory: 3M/15M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hadoop-wordcount: Could not resolve dependencies for project uk.co.arunhorne:hadoop-wordcount:jar:1.0-SNAPSHOT: Failed to collect dependencies for [org.apache.hadoop:hadoop-core:jar:0.20.2-cdh3u3 (provided), org.apache.commons:commons-lang3:jar:3.1 (compile)]: Failed to read artifact descriptor for org.apache.hadoop:hadoop-core:jar:0.20.2-cdh3u3: Could not transfer artifact org.apache.hadoop:hadoop-core:pom:0.20.2-cdh3u3 from/to cloudera-releases (https://repository.cloudera.com/artifactory/cloudera-repos/): Not authorized by proxy, ReasonPhrase:Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  ). -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

当我尝试创建本地目录并下载 jar 时,它构建成功。但是在尝试连接到 cloudera url 时出错,因为我只需要使用 cloudera url,而不是任何其他 url

4

1 回答 1

0

尝试从 Maven Repo 使用它:

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-core</artifactId>
    <version>1.1.1</version>
</dependency>

更新

我认为您使用此页面创建您的应用程序。

我创建了类似的项目并运行mvn package. Maven 从 cloudera 存储库下载所有必要的库,您可以看到完整的 Maven 日志

我也给你链接到我的项目(也许对你有帮助)

所以看看这一切,我可以假设你的 Maven 配置有问题。

于 2013-02-04T12:52:38.123 回答