2

我正在尝试创建一个 Dataflow 作业以便在 BigTable 中插入行，但是当我在本地测试 Dataflow 作业时，我收到以下错误：

Exception in thread "main" com.google.cloud.dataflow.sdk.Pipeline$PipelineExecutionException: java.lang.IllegalStateException: Neither Jetty ALPN nor OpenSSL via netty-tcnative were properly configured.
at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:186)

您可以在下面找到我的主要代码：

    CloudBigtableOptions options =
            PipelineOptionsFactory.fromArgs(args).withValidation().create().as(CloudBigtableOptions.class);

    options.setProject("xxxxxxxxx");
    options.setBigtableProjectId("xxxxxxxxx");
    options.setBigtableInstanceId("xxxxxxxxx");
    options.setBigtableTableId("xxxxxxxxx");
    options.setZone("europe-west1-b");
    options.setRunner(DirectPipelineRunner.class);

    CloudBigtableTableConfiguration config =
            CloudBigtableTableConfiguration.fromCBTOptions(options);

    Pipeline p = Pipeline.create(options);
    CloudBigtableIO.initializeForWrite(p);

    FixedWindows window = FixedWindows.of(Duration.standardMinutes(1));

    p
            .apply(Create.of("Hello"))
            .apply(Window.into(window))
            .apply(ParDo.of(MUTATION_TRANSFORM))
            .apply(CloudBigtableIO.writeToTable(config));

    p.run();

另一种尝试是使用以下代码：

    CloudBigtableTableConfiguration config =
            new CloudBigtableTableConfiguration.Builder()
                    .withProjectId("xxxxxxxxx")
                    .withInstanceId("xxxxxxxxx")
                    .withTableId("xxxxxxxxx")
                    .build();

    Pipeline p = Pipeline.create(options);
    CloudBigtableIO.initializeForWrite(p);

    FixedWindows window = FixedWindows.of(Duration.standardMinutes(1));

    p
            .apply(Create.of("Hello"))
            .apply(Window.into(window))
            .apply(ParDo.of(MUTATION_TRANSFORM))
            .apply(CloudBigtableIO.writeToTable(config));

    p.run();

但我得到了同样的错误。

难道我做错了什么？

编辑：

完整错误：

Exception in thread "main" com.google.cloud.dataflow.sdk.Pipeline$PipelineExecutionException: java.lang.IllegalStateException: Neither Jetty ALPN nor OpenSSL via netty-tcnative were properly configured.
    at com.google.cloud.dataflow.sdk.Pipeline.run(Pipeline.java:186)
    at HubCache.main(HubCache.java:75)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.IllegalStateException: Neither Jetty ALPN nor OpenSSL via netty-tcnative were properly configured.
    at com.google.bigtable.repackaged.com.google.cloud.grpc.BigtableSession.<init>(BigtableSession.java:236)
    at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:123)
    at org.apache.hadoop.hbase.client.AbstractBigtableConnection.<init>(AbstractBigtableConnection.java:91)
    at com.google.cloud.bigtable.hbase1_0.BigtableConnection.<init>(BigtableConnection.java:33)
    at com.google.cloud.bigtable.dataflow.CloudBigtableConnectionPool$1.<init>(CloudBigtableConnectionPool.java:72)
    at com.google.cloud.bigtable.dataflow.CloudBigtableConnectionPool.createConnection(CloudBigtableConnectionPool.java:72)
    at com.google.cloud.bigtable.dataflow.CloudBigtableConnectionPool.getConnection(CloudBigtableConnectionPool.java:64)
    at com.google.cloud.bigtable.dataflow.CloudBigtableConnectionPool.getConnection(CloudBigtableConnectionPool.java:57)
    at com.google.cloud.bigtable.dataflow.AbstractCloudBigtableTableDoFn.getConnection(AbstractCloudBigtableTableDoFn.java:96)
    at com.google.cloud.bigtable.dataflow.CloudBigtableIO$CloudBigtableSingleTableBufferedWriteFn.getBufferedMutator(CloudBigtableIO.java:941)
    at com.google.cloud.bigtable.dataflow.CloudBigtableIO$CloudBigtableSingleTableBufferedWriteFn.processElement(CloudBigtableIO.java:966)

pom.xml：

 <dependencies>
        <dependency>
            <groupId>com.google.cloud.dataflow</groupId>
            <artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
            <version>LATEST</version>
        </dependency>
        <dependency>
            <groupId>com.google.cloud.bigtable</groupId>
            <artifactId>bigtable-hbase-dataflow</artifactId>
            <version>LATEST</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-api -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.21</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-simple -->
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>LATEST</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/io.netty/netty-tcnative-boringssl-static -->
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-tcnative-boringssl-static</artifactId>
            <version>1.1.33.Fork13</version>
            <classifier>${os.detected.classifier}</classifier>
        </dependency>
    </dependencies>

    <build>
        <pluginManagement>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.5.1</version>
                    <configuration>
                        <source>1.8</source>
                        <target>1.8</target>
                    </configuration>
                </plugin>
            </plugins>
        </pluginManagement>
        <extensions>
            <!-- Use os-maven-plugin to initialize the "os.detected" properties -->
            <extension>
                <groupId>kr.motd.maven</groupId>
                <artifactId>os-maven-plugin</artifactId>
                <version>1.4.0.Final</version>
            </extension>
        </extensions>
    </build>
4

2 回答 2

3

我们刚刚发布了与 Netty tcnative-boringssl-static配对的0.9.1 客户端，我们使用 Fork19，如连接器示例中所示， pardo hello 示例不再需要使用.os.detected

于 2016-07-19T18:46:45.367 回答
1

问题是用于初始化“os.detected”属性的扩展os-maven-plugin没有正确初始化该属性。

我已经正确设置了属性，并且测试已经执行，没有问题。

于 2016-07-14T09:04:00.333 回答