4

对于我想扩展 Elasticsearch 的项目,因此需要使用包Symja。在 Symja 的 Github 中,提供了使用 Maven 的手册

由于 Elasticsearch 存储库是使用 Gradle 构建的,因此我还需要使用 Gradle 而不是 Maven。测试建议的示例 Symja project,以下build.gradle(我基本上是通过使用gradle init和调整生成的)完美地导入了库:

apply plugin: 'java'
apply plugin: 'maven-publish'

repositories {
    mavenLocal()
    maven {
        url = uri('https://oss.sonatype.org/content/repositories/snapshots')
    }
    maven {
        url = uri('https://repo.maven.apache.org/maven2/')
    }
}

dependencies {
    implementation 'org.matheclipse:matheclipse-core:1.0.0-SNAPSHOT'
}

publishing {
    publications {
        maven(MavenPublication) {
            from(components.java)
        }
    }
}

现在,我想将 Symja 添加到我的 Elasticsearch fork 到server模块中。

但是,当我尝试构建它时,导入

import org.matheclipse.parser.client.SyntaxError;
import org.matheclipse.parser.client.math.MathException;

未检测到 Symja 示例:java: package org.matheclipse.parser.client does not exist. 可能出了什么问题?

我更新build.gradleserver模块如下所示:

...
apply plugin: 'java'
apply plugin: 'maven-publish'

publishing {
    publications {
        ...
        maven(MavenPublication) {
            from(components.java)
        }
    }
}

repositories {
    mavenLocal()
    maven {
        url = uri('https://oss.sonatype.org/content/repositories/snapshots')
    }
    maven {
        url = uri('https://repo.maven.apache.org/maven2/')
    }
}

...

dependencies {
    ...
    // Symja
    implementation "org.matheclipse:matheclipse-core:1.0.0-SNAPSHOT"
    ...
}

...

编辑: 在遵循@axelclk 的添加建议之后implementation "org.matheclipse:matheclipse-frontend:1.0.0-SNAPSHOT",我的 IDE 现在检测到了这些包。虽然,如果我构建build.gradle(同时包含 Symja 示例中的方法org.elasticsearch.index.query.BoolQueryBuilder.java),会出现以下错误:

C:\{path here...}\elasticsearch\server\src\main\java\org\elasticsearch\index\query\BoolQueryBuilder.java:454: error: cannot access GcdRingElem
            System.out.println("Out[2]: " + result.toString());
                                                  ^
  class file for edu.jas.structure.GcdRingElem not found
C:\{path here...}\elasticsearch\server\src\main\java\org\elasticsearch\index\query\BoolQueryBuilder.java:459: error: cannot access FieldElement
            System.out.println("Out[3]: " + result.toString());
                                                  ^
  class file for org.hipparchus.FieldElement not found

似乎不知何故,该toString方法无法正常工作。此外,如果我使用 运行 Elasticsearch fork gradlew run -Dtests.es.xpack.security.enabled=false -Dtests.heap.size=1G,构建将失败并显示以下输出:

> Configure project :modules:reindex
Disabling reindex-from-old tests because we can't get the pid file on windows

> Configure project :plugins:repository-hdfs
hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH

> Configure project :x-pack:plugin:searchable-snapshots:qa:hdfs
hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH
=======================================
Elasticsearch Build Hamster says Hello!
  Gradle Version        : 6.8
  OS Info               : Windows 10 10.0 (amd64)
  JDK Version           : 15 (Oracle)
  JAVA_HOME             : C:\Program Files\Java\jdk-15.0.1
  Random Testing Seed   : C5A79F8CABA0D6FA
  In FIPS 140 mode      : false
=======================================

> Task :server:compileJava
C:\{path here ...}\elasticsearch\server\src\main\java\org\elasticsearch\index\query\BoolQueryBuilder.java:454: error: cannot access GcdRingElem
            System.out.println("Out[2]: " + result.toString());
                                                  ^
  class file for edu.jas.structure.GcdRingElem not found
C:\{path here ...}\elasticsearch\server\src\main\java\org\elasticsearch\index\query\BoolQueryBuilder.java:459: error: cannot access FieldElement
            System.out.println("Out[3]: " + result.toString());
                                                  ^
  class file for org.hipparchus.FieldElement not found
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
2 errors

> Task :server:compileJava FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':server:compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 58s
228 actionable tasks: 8 executed, 220 up-to-date

编辑2:我按照@axelclk 的建议添加implementation "org.matheclipse:matheclipse-external:1.0.0-SNAPSHOT"和替换mavenLocal()mavenCentral()这不会导致任何不同的结果。

编辑 3:我再次根据@axelclk 的建议添加了 - 由 和 的文件pom.xml描述的实现:matheclipse-externalmatheclipse-core

...
  // Symja
  implementation "org.matheclipse:matheclipse-external:1.0.0-SNAPSHOT"
  //implementation "org.apache.logging.log4j:log4j-core:2.13.2"
  //implementation "org.apache.logging.log4j:log4j-api:2.13.2"
  testImplementation "org.slf4j:slf4j-simple:1.7.25"
  implementation "org.slf4j:log4j-over-slf4j:1.7.25"
  //implementation "com.google.guava:failureaccess:1.0.1"
  //implementation "com.google.guava:guava:30.1.1-jre"
  implementation "com.fasterxml.jackson.core:jackson-databind:2.11.0"
  implementation "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.0"
  implementation "io.pebbletemplates:pebble:3.1.5"
  implementation "org.jheaps:jheaps:0.13"
  implementation "org.antlr:antlr4-runtime:4.8-1"
  implementation "org.apache.commons:commons-text:1.8"
  //testImplementation "junit:junit:4.13.1"

  implementation "org.matheclipse:matheclipse-frontend:1.0.0-SNAPSHOT"

  implementation "org.matheclipse:matheclipse-core:1.0.0-SNAPSHOT"
  implementation "org.commonmark:commonmark:0.17.1"
  implementation "org.commonmark:commonmark-ext-gfm-tables:0.17.1"
  implementation "org.apfloat:apfloat:1.10.0-SNAPSHOT"
  //implementation "org.apache.logging.log4j:log4j-core:2.13.2"
  //implementation "org.apache.logging.log4j:log4j-api:2.13.2"
  //testImplementation "org.slf4j:slf4j-simple:1.7.25"
  //implementation "org.slf4j:log4j-over-slf4j:1.7.25"
  implementation "org.hipparchus:hipparchus-core:1.8"
  implementation "org.hipparchus:hipparchus-clustering:1.8"
  implementation "org.hipparchus:hipparchus-fft:1.8"
  implementation "org.hipparchus:hipparchus-fitting:1.8"
  implementation "org.hipparchus:hipparchus-ode:1.8"
  implementation "org.hipparchus:hipparchus-optim:1.8"
  implementation "org.hipparchus:hipparchus-stat:1.8"
  //implementation "commons-codec:commons-codec:1.14"
  implementation "org.apache.commons:commons-lang3:3.11"
  implementation "com.google.code.findbugs:jsr305:3.0.0"
  implementation "org.logicng:logicng:2.0.2"
  implementation "org.gavaghan:geodesy:1.1.3"
  implementation "it.unimi.dsi:fastutil:8.5.2"
  implementation "org.codehaus.janino:janino:3.1.3"
  implementation "com.ibm.icu:icu4j:68.2"
  implementation "com.univocity:univocity-parsers:2.8.4"
  testImplementation "org.junit.jupiter:junit-jupiter-api:5.4.2"
  testImplementation "org.junit.jupiter:junit-jupiter-engine:5.4.2"
  testImplementation "org.junit.vintage:junit-vintage-engine:5.4.2"

  //implementation "org.matheclipse:matheclipse-api:1.0.0-SNAPSHOT"
  //implementation "org.matheclipse:matheclipse-io:1.0.0-SNAPSHOT"
  //implementation "org.matheclipse:matheclipse-beakerx:1.0.0-SNAPSHOT"
...

现在,server构建似乎工作。但是,如果我现在使用 using 运行gradlew run -Dtests.es.xpack.security.enabled=false -Dtests.heap.size=1G,则会有不同的任务失败:

> Configure project :modules:reindex
Disabling reindex-from-old tests because we can't get the pid file on windows

> Configure project :plugins:repository-hdfs
hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH

> Configure project :x-pack:plugin:searchable-snapshots:qa:hdfs
hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH
=======================================
Elasticsearch Build Hamster says Hello!
  Gradle Version        : 6.8
  OS Info               : Windows 10 10.0 (amd64)
  JDK Version           : 15 (Oracle)
  JAVA_HOME             : C:\Program Files\Java\jdk-15.0.1
  Random Testing Seed   : 266D7D1003A7C996
  In FIPS 140 mode      : false
=======================================

> Task :modules:kibana:bundlePlugin FAILED
> Task :modules:ingest-user-agent:bundlePlugin FAILED
> Task :modules:aggs-matrix-stats:bundlePlugin FAILED
> Task :modules:ingest-common:bundlePlugin FAILED
> Task :modules:lang-mustache:bundlePlugin FAILED
> Task :modules:ingest-geoip:bundlePlugin FAILED
> Task :modules:analysis-common:bundlePlugin FAILED
> Task :modules:geo:bundlePlugin FAILED

FAILURE: Build completed with 8 failures.

1: Task failed with an exception.
-----------
* What went wrong:
...

* Get more help at https://help.gradle.org

BUILD FAILED in 13s
204 actionable tasks: 9 executed, 195 up-to-date

我觉得这很好奇,因为我没有对我的 Elasticsearch 分支的这些部分进行任何更改。这里发生了什么?

编辑 4:在 Elasticsearch 的“jar hell”功能出现一些问题后,我设法在@IanGabes 的帮助下通过repositoriesbuild.gradle. 问题是,Symja 类似乎还没有工作。我有一个normalize当前执行以下操作的方法:

private void normalize(String queryString) {
    System.out.println("I'm here!");
    String eString = "";
    ExprEvaluator util = new ExprEvaluator(false, (short) 100);
    IExpr e;
}

该方法在调用BoolQueryBuilder.doToQuery()之前在 Elasticsearch 中addBooleanClauses调用。现在我正在POST学习一个BooleanQuery. "I'm here!"正如预期的那样,有输出。之后,会发生以下情况:

...
╗ [2021-04-19T22:54:29,779][INFO ][stdout ] [runTask-0] I'm here!
╗ [2021-04-19T20:54:30.497923300Z] [BUILD] Stopping node

=== Standard error of node `node{::runTask-0}` ===
╗ ? last 40 non error or warning messages from C:\{path...}\elasticsearch\build\testclusters\runTask-0\logs\es.stderr.log ?
╗ fatal error in thread [elasticsearch[runTask-0][search][T#1]], exiting
╗ java.lang.NoClassDefFoundError: com/google/common/util/concurrent/UncheckedTimeoutException
╗ at org.elasticsearch.index.query.BoolQueryBuilder.normalize(BoolQueryBuilder.java:440)
╗ at org.elasticsearch.index.query.BoolQueryBuilder.doToQuery(BoolQueryBuilder.java:332)
╗ at org.elasticsearch.index.query.AbstractQueryBuilder.toQuery(AbstractQueryBuilder.java:103)
╗ at org.elasticsearch.index.query.SearchExecutionContext.lambda$toQuery$3(SearchExecutionContext.java:464)
╗ at org.elasticsearch.index.query.SearchExecutionContext.toQuery(SearchExecutionContext.java:476)
╗ at org.elasticsearch.index.query.SearchExecutionContext.toQuery(SearchExecutionContext.java:463)
╗ at org.elasticsearch.search.SearchService.parseSource(SearchService.java:912)
╗ at org.elasticsearch.search.SearchService.createContext(SearchService.java:750)
╗ at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:438)
╗ at org.elasticsearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:406)
╗ at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58)
╗ at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)
╗ at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
╗ at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44)
╗ at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:739)
╗ at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
╗ at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
╗ at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
╗ at java.base/java.lang.Thread.run(Thread.java:832)
╗ Caused by: java.lang.ClassNotFoundException: com.google.common.util.concurrent.UncheckedTimeoutException
╗ at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:606)
╗ at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:168)
╗ at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
╗ ... 19 more

> Task :run FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':run'.
> Elasticsearch cluster died

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Get more help at help.gradle.org

BUILD FAILED in 3m 34s
429 actionable tasks: 13 executed, 416 up-to-date

我看不出用于初始化 Symja 类的几个命令如何在这里造成任何伤害。但是当我将它们注释掉时,一切正常,所以问题似乎出在这个地方。目前包含以下依赖gradle.build项(或分别注释掉,如果它们与不同模块的其他依赖项发生冲突):

...
  implementation "org.matheclipse:matheclipse-external:1.0.0-SNAPSHOT"
  //implementation "org.apache.logging.log4j:log4j-core:2.13.2"
  //implementation "org.apache.logging.log4j:log4j-api:2.13.2"
  testImplementation "org.slf4j:slf4j-simple:1.7.25"
  //implementation "org.slf4j:log4j-over-slf4j:1.7.25"
  //implementation "com.google.guava:failureaccess:1.0.1"
  //implementation "com.google.guava:guava:30.1.1-jre"
  //implementation "com.fasterxml.jackson.core:jackson-databind:2.11.0"
  implementation "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.0"
  implementation "io.pebbletemplates:pebble:3.1.5"
  implementation "org.jheaps:jheaps:0.13"
  //implementation "org.antlr:antlr4-runtime:4.8-1"
  implementation "org.apache.commons:commons-text:1.8"
  //testImplementation "junit:junit:4.13.1"

  implementation "org.matheclipse:matheclipse-frontend:1.0.0-SNAPSHOT"

  implementation "org.matheclipse:matheclipse-core:1.0.0-SNAPSHOT"
  implementation "org.commonmark:commonmark:0.17.1"
  implementation "org.commonmark:commonmark-ext-gfm-tables:0.17.1"
  implementation "org.apfloat:apfloat:1.10.0-SNAPSHOT"
  //implementation "org.apache.logging.log4j:log4j-core:2.13.2"
  //implementation "org.apache.logging.log4j:log4j-api:2.13.2"
  //testImplementation "org.slf4j:slf4j-simple:1.7.25"
  //implementation "org.slf4j:log4j-over-slf4j:1.7.25"
  implementation "org.hipparchus:hipparchus-core:1.8"
  implementation "org.hipparchus:hipparchus-clustering:1.8"
  implementation "org.hipparchus:hipparchus-fft:1.8"
  implementation "org.hipparchus:hipparchus-fitting:1.8"
  implementation "org.hipparchus:hipparchus-ode:1.8"
  implementation "org.hipparchus:hipparchus-optim:1.8"
  implementation "org.hipparchus:hipparchus-stat:1.8"
  //implementation "commons-codec:commons-codec:1.14"
  implementation "org.apache.commons:commons-lang3:3.11"
  implementation "com.google.code.findbugs:jsr305:3.0.0"
  implementation "org.logicng:logicng:2.0.2"
  implementation "org.gavaghan:geodesy:1.1.3"
  implementation "it.unimi.dsi:fastutil:8.5.2"
  implementation "org.codehaus.janino:janino:3.1.3"
  //implementation "com.ibm.icu:icu4j:68.2"
  implementation "com.univocity:univocity-parsers:2.8.4"
  testImplementation "org.junit.jupiter:junit-jupiter-api:5.4.2"
  testImplementation "org.junit.jupiter:junit-jupiter-engine:5.4.2"
  testImplementation "org.junit.vintage:junit-vintage-engine:5.4.2"
...
4

1 回答 1

2

为了完整起见,我想至少包含@axelclk 和@IanGabes 给出的有效解决方案的一部分。首先,似乎有必要手动将所有隐式依赖项以及它们源自的存储库添加server'sbuild.gradle,对应pom.xml文件:matheclipse-corematheclipse-external

...
apply plugin: 'java'
apply plugin: 'maven-publish'

publishing {
    publications {
        ...
        maven(MavenPublication) {
            from(components.java)
        }
    }
}

repositories {
  mavenCentral()
  maven {
    url = uri('https://oss.sonatype.org/content/repositories/snapshots')
  }
  maven {
    url = uri('https://repo.maven.apache.org/maven2/')
  }
}

...

  // Symja
  implementation "org.matheclipse:matheclipse-external:1.0.0-SNAPSHOT"
  //implementation "org.apache.logging.log4j:log4j-core:2.13.2"
  //implementation "org.apache.logging.log4j:log4j-api:2.13.2"
  testImplementation "org.slf4j:slf4j-simple:1.7.25"
  //implementation "org.slf4j:log4j-over-slf4j:1.7.25"
  implementation "com.google.guava:failureaccess:1.0.1"
  //implementation "com.google.guava:guava:30.1.1-jre"
  //implementation "com.fasterxml.jackson.core:jackson-databind:2.11.0"
  implementation "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.11.0"
  implementation "io.pebbletemplates:pebble:3.1.5"
  implementation "org.jheaps:jheaps:0.13"
  //implementation "org.antlr:antlr4-runtime:4.8-1"
  implementation "org.apache.commons:commons-text:1.8"
  //testImplementation "junit:junit:4.13.1"

  implementation "org.matheclipse:matheclipse-frontend:1.0.0-SNAPSHOT"

  implementation "org.matheclipse:matheclipse-core:1.0.0-SNAPSHOT"
  implementation "org.commonmark:commonmark:0.17.1"
  implementation "org.commonmark:commonmark-ext-gfm-tables:0.17.1"
  implementation "org.apfloat:apfloat:1.10.0-SNAPSHOT"
  //implementation "org.apache.logging.log4j:log4j-core:2.13.2"
  //implementation "org.apache.logging.log4j:log4j-api:2.13.2"
  //testImplementation "org.slf4j:slf4j-simple:1.7.25"
  //implementation "org.slf4j:log4j-over-slf4j:1.7.25"
  implementation "org.hipparchus:hipparchus-core:1.8"
  implementation "org.hipparchus:hipparchus-clustering:1.8"
  implementation "org.hipparchus:hipparchus-fft:1.8"
  implementation "org.hipparchus:hipparchus-fitting:1.8"
  implementation "org.hipparchus:hipparchus-ode:1.8"
  implementation "org.hipparchus:hipparchus-optim:1.8"
  implementation "org.hipparchus:hipparchus-stat:1.8"
  //implementation "commons-codec:commons-codec:1.14"
  implementation "org.apache.commons:commons-lang3:3.11"
  implementation "com.google.code.findbugs:jsr305:3.0.0"
  implementation "org.logicng:logicng:2.0.2"
  implementation "org.gavaghan:geodesy:1.1.3"
  implementation "it.unimi.dsi:fastutil:8.5.2"
  implementation "org.codehaus.janino:janino:3.1.3"
  //implementation "com.ibm.icu:icu4j:68.2"
  implementation "com.univocity:univocity-parsers:2.8.4"
  testImplementation "org.junit.jupiter:junit-jupiter-api:5.4.2"
  testImplementation "org.junit.jupiter:junit-jupiter-engine:5.4.2"
  testImplementation "org.junit.vintage:junit-vintage-engine:5.4.2"
...

但是,如上面的代码所示,一些依赖项被注释掉了。这是因为 Elasticsearch 具有引发构建错误的功能,因为jar hell当构建的其他部分已经使用了特定依赖项的另一个版本时。

此外,由于server包依赖于其父包,因此有必要将存储库添加到's的subprojects环境中elasticsearchbuild.gradle

...
subprojects {
  ...
  repositories {
    mavenCentral()
    maven {
      url = uri('https://oss.sonatype.org/content/repositories/snapshots')
    }
    maven {
      url = uri('https://repo.maven.apache.org/maven2/')
    }
  }
}
...

完成此操作后,当将示例添加到模块中现有的 Elasticsearch 类时,会在我的 IDE(对我来说:IntelliJ IDEA)中检测到 Symja 示例的包server。(在我的例子中,我调用了一个org.elasticsearch.index.query.BoolQueryBuilder包含Symja 示例的方法)。

如果我尝试使用例如从我的 Elasticsearch 根目录构建/运行所有内容gradlew run -Dtests.es.xpack.security.enabled=false -Dtests.heap.size=1G,那么一切都会首先完美构建。也可以使用PUT.... 在我的例子中,尝试运行一个布尔查询(然后使用 Symja 调用添加的方法)是一个更大的问题。在这种情况下,Elasticsearch 会立即崩溃并显示以下输出:

...
╗ [2021-04-22T08:34:21.178673600Z] [BUILD] Stopping node

=== Standard error of node `node{::runTask-0}` ===
╗   ? last 40 non error or warning messages from C:\...\elasticsearch\build\testclusters\runTask-0\logs\es.stderr.log ?
╗ fatal error in thread [elasticsearch[runTask-0][search][T#1]], exiting
╗  java.lang.NoClassDefFoundError: com/google/common/util/concurrent/UncheckedTimeoutException
...

> Task :run FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':run'.
> Elasticsearch cluster died

...

BUILD FAILED in 7m 37s
429 actionable tasks: 19 executed, 410 up-to-date

这是因为 Symja 在运行时依赖于Guava包的存在,需要从build.gradle. 一方面,无法使用该implementation选项将 Guava 添加到任何包中,因为它在文件中被 Elasticsearch 定义为“禁止依赖项” gradle\forbidden-dependencies.gradle

// we do not want any of these dependencies on the compilation classpath
// because they could then be used within Elasticsearch
List<String> FORBIDDEN_DEPENDENCIES = [
  'guava'
]
...

另一方面,也不能通过将其添加runtimeOnly "com.google.guava:guava:30.1.1-jre"server模块中。这再次导致构建失败jar hell,因为 Elasticsearch 的一些测试部分也以这种方式使用 Guava。尝试将版本调整为现有版本也无效。

总而言之,将 Symja 内容添加到 Elasticsearch 的server模块中似乎是不可能的。在 Elastic Stack 社区中,声明如下:

[...] 您在依赖项等方面面临的问题是尝试向server.

这似乎是最佳做法。我会走这条路,因为无论如何我的目标是创建一个插件。

更新:由于我设法安装了我正在编写的插件,因此使用上述工作流程确实可以正常工作(通过runtimeOnly并使用build.gradle我的插件包含 Guava)。

于 2021-04-22T08:59:10.363 回答