我在公司防火墙后面的 Windows 服务器中运行 MiNiFi。我的 NiFi 在 Hortonworks 集群上运行（3 个节点，例如：sj46、sj47、sj48）。
我正在尝试将数据从 MiNiFi 发送到 NiFi。
- MiNiFi 版本：0.4
- NiFi 版本：1.5
在 MiNiFi 中，我有 ListFile 处理器、FetchFile 处理器和 RemoteProcessGroup。
远程进程组：
- 网址： https ://sjxx46.xxx:6689/nifi,https://sjxx47.xxx:6689/nifi,https://sjxx48.xxx:6689/nifi
- 传输协议：HTTP
NiFi.属性：
nifi.remote.input.host=
nifi.remote.input.http.enabled=true
nifi.remote.input.http.transaction.ttl=30 sec
nifi.remote.input.secure=True
nifi.remote.input.socket.port=6698
nifi.cluster.node.protocol.port=6699
nifi.web.http.port=
nifi.web.https.host=sj46.xxxx
nifi.web.https.port=6689
MiNiFi config.yml
MiNiFi Config Version: 3
Flow Controller:
name: min2510
comment: ''
Core Properties:
flow controller graceful shutdown period: 10 sec
flow service write delay interval: 500 ms
administrative yield duration: 30 sec
bored yield duration: 10 millis
max concurrent threads: 1
variable registry properties: ''
FlowFile Repository:
partitions: 256
checkpoint interval: 2 mins
always sync: false
Swap:
threshold: 20000
in period: 5 sec
in threads: 1
out period: 5 sec
out threads: 4
Content Repository:
content claim max appendable size: 10 MB
content claim max flow files: 100
always sync: false
Provenance Repository:
provenance rollover time: 1 min
implementation: org.apache.nifi.provenance.MiNiFiPersistentProvenanceRepository
Component Status Repository:
buffer size: 1440
snapshot frequency: 1 min
Security Properties:
keystore: ''
keystore type: ''
keystore password: ''
key password: ''
truststore: ''
truststore type: ''
truststore password: ''
ssl protocol: ''
Sensitive Props:
key:
algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
provider: BC
Processors:
- id: 50420aac-ef66-30ac-0000-000000000000
name: FetchFile
class: org.apache.nifi.processors.standard.FetchFile
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list:
- failure
- not.found
- permission.denied
Properties:
Completion Strategy: None
File to Fetch: ${absolute.path}/${filename}
Log level when file not found: ERROR
Log level when permission denied: ERROR
Move Conflict Strategy: Rename
Move Destination Directory:
- id: ae4799d5-073c-3dd6-0000-000000000000
name: ListFile
class: org.apache.nifi.processors.standard.ListFile
max concurrent tasks: 1
scheduling strategy: TIMER_DRIVEN
scheduling period: 0 sec
penalization period: 30 sec
yield period: 1 sec
run duration nanos: 0
auto-terminated relationships list: []
Properties:
File Filter: '[^\.].*'
Ignore Hidden Files: 'true'
Include File Attributes: 'true'
Input Directory: C:\minifi to EFT
Input Directory Location: Local
Maximum File Age:
Maximum File Size:
Minimum File Age: 0 sec
Minimum File Size: 0 B
Path Filter:
Recurse Subdirectories: 'true'
target-system-timestamp-precision: auto-detect
Controller Services: []
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: 95dde9e7-96a4-3bea-0000-000000000000
name: FetchFile/success/eef78a5f-cc55-1d33-0000-0000481921e3
source id: 50420aac-ef66-30ac-0000-000000000000
source relationship names:
- success
destination id: eef78a5f-cc55-1d33-0000-0000481921e3
max work queue size: 10000
max work queue data size: 1 GB
flowfile expiration: 0 sec
queue prioritizer class: ''
- id: 2ba13196-5ac6-3966-0000-000000000000
name: ListFile/success/FetchFile
source id: ae4799d5-073c-3dd6-0000-000000000000
source relationship names:
- success
destination id: 50420aac-ef66-30ac-0000-000000000000
max work queue size: 10000
max work queue data size: 1 GB
flowfile expiration: 0 sec
queue prioritizer class: ''
Remote Process Groups:
- id: 6bd4b280-646b-3b93-0000-000000000000
name: ''
url: https://sj46.xxxx:6689/nifi
comment: ''
timeout: 30 sec
yield period: 10 sec
transport protocol: HTTP
proxy host: ''
proxy port: ''
proxy user: ''
proxy password: ''
local network interface: ''
Input Ports:
- id: eef78a5f-cc55-1d33-0000-0000481921e3
name: From MiNiFi
comment: ''
max concurrent tasks: 1
use compression: false
Output Ports: []
NiFi Properties Overrides: {}
在MiNiFi windows server中，打开了6688、6689、6699等端口。
但如果我尝试在 MiNiFi 和 NiFi 上远程登录，它就没有建立起来。
minifi-app.log
2019-03-27 15:50:09,152 WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://sj46.xxxx:6689/nifi-api due to org.apache.http.conn.HttpHostConnectException: Connect to sj46.xxxx:6689 [sj46.xxxx/x.x.x.x] failed: Connection timed out: connect
2019-03-27 15:50:09,160 WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.remote.StandardRemoteProcessGroup Unable to communicate with remote instance RemoteProcessGroup[https://sj46.xxxx:6689/nifi]
org.apache.nifi.controller.exception.CommunicationsException: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://sj46.xxxx:6689/nifi due to: Connect to sj46.xxxx:6689 [sj46.xxxx/x.x.x.x] failed: Connection timed out: connect
at org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:882)
at org.apache.nifi.remote.StandardRemoteProcessGroup.lambda$initialize$0(StandardRemoteProcessGroup.java:193)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.nifi.controller.exception.CommunicationsException: Unable to communicate with Remote NiFi at URI https://sj46.xxxx:6689/nifi due to: Connect to sj46.xxxx:6689 [sj46.xxxx/x.x.x.x] failed: Connection timed out: connect
at org.apache.nifi.remote.StandardRemoteProcessGroup.refreshFlowContents(StandardRemoteProcessGroup.java:844)
... 8 common frames omitted
有关如何解决此问题的任何指示？
通过从 MiNiFi 服务器禁用防火墙解决了上述问题。现在我遇到了另一个问题，如下所示：
WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.r.util.SiteToSiteRestApiClient Failed to get controller from https://sj46.xxxx:6689/nifi-api due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
WARN [Remote Process Group 6bd4b280-646b-3b93-0000-000000000000 Thread-1] o.a.n.remote.StandardRemoteProcessGroup Unable to connect to RemoteProcessGroup[https://sj46.xxxx:6689/nifi] due to javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
上述 SSLHandshakeException 问题已通过将 SSL 证书（您可以在 NiFi.properties 尤其是 keystore.jks 和 truststore.jks 中引用）导入 MiNiFi 得到解决。我们需要手动修改 MiNiFi 中的 config.yml。
在这个问题之后，我面临以下问题：
ERROR [Timer-Driven Process Thread-6] o.a.n.c.t.ContinuallyRunConnectableTask RemoteGroupPort[name=From MiNiFi,targets=https://sj46.xxxx:6689/nifi] failed to process session due to java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
ERROR [Timer-Driven Process Thread-6] o.a.n.c.t.ContinuallyRunConnectableTask
java.lang.RuntimeException: java.lang.NoSuchMethodError: org.apache.http.impl.client.HttpClientBuilder.setSSLContext(Ljavax/net/ssl/SSLContext;)Lorg/apache/http/impl/client/HttpClientBuilder;
SSLContext 问题已通过将 MiNiFi 版本从 0.4 更改为 0.3 得到解决。
现在我能够成功地从 MiNiFi 服务器获取数据到 NiFi。
谢谢。