2

我已经像下面这样配置了我的水槽代理。不知何故,水槽代理无法正常运行。它一直挂着,没有任何错误。下面的配置有没有问题。

仅供参考:我有一个名为“country”的文件,其中包含硬编码标题作为状态

#Define sources, sink and channels
foo.sources =  s1
foo.channels = chn-az  chn-oth
foo.sinks   =  sink-az sink-oth
#
### # # Define a source on agent and connect to channel memory-channel.
foo.sources.s1.type = exec
foo.sources.s1.command = cat /home/hadoop/flume/country.txt
foo.sources.s1.batchSize = 1
foo.sources.s1.channels = chn-ca  chn-oth

#selector configuration
foo.sources.s1.selector.type = multiplexing
foo.sources.s1.selector.header = state
foo.sources.s1.selector.mapping.AZ = chn-az
foo.sources.s1.selector.default    = chn-oth
#
#
### Define a memory channel on agent called memory-channel.
foo.channels.chn-az.type = memory
foo.channels.chn-oth.type = memory
#
#
##Define sinks that outputs to hdfs.
foo.sinks.sink-az.channel = chn-az
foo.sinks.sink-az.type = hdfs
foo.sinks.sink-az.hdfs.path = hdfs://master:9099/user/hadoop/flume
foo.sinks.sink-az.hdfs.filePrefix = statefilter
foo.sinks.sink-az.hdfs.fileType = DataStream
foo.sinks.sink-az.hdfs.writeFormat = Text
foo.sinks.sink-az.batchSize    = 1
foo.sinks.sink-az.rollInterval = 0
#
foo.sinks.sink-oth.channel = chn-oth
foo.sinks.sink-oth.type = hdfs
foo.sinks.sink-oth.hdfs.path = hdfs://master:9099/user/hadoop/flume
foo.sinks.sink-oth.hdfs.filePrefix = statefilter
foo.sinks.sink-oth.hdfs.fileType = DataStream
foo.sinks.sink-oth.batchSize    = 1
foo.sinks.sink-oth.rollInterval = 0

谢谢,维诺特

4

1 回答 1

0

关于源头配置的频道列表:

foo.sources.s1.channels = chn-ca  chn-oth

我认为chn-ca应该是chn-az

尽管如此,我认为这样的配置永远不会起作用,因为选择器使用的“状态”标头不是由任何 Flume 组件创建的。您必须为此引入一个拦截器,通常是Regex Extractor Interceptor

于 2015-09-23T13:28:25.680 回答