2

这是一个 cygnus 代理日志：

16 Sep 2015 12:30:19,820 INFO  [521330370@qtp-1739580287-1] (com.telefonica.iot.cygnus.handlers.OrionRestHandler.getEvents:236)  - Received data (<notifyContextRequest><subscriptionId>55f932e6c06c4173451bbe1c</subscriptionId><originator>localhost</originator>...<contextAttribute><name>utctime</name><type>string</type><contextValue>2015-9-16 9:37:52</contextValue></contextAttribute></contextAttributeList></contextElement><statusCode><code>200</code><reasonPhrase>OK</reasonPhrase></statusCode></contextElementResponse></contextResponseList></notifyContextRequest>)
16 Sep 2015 12:30:19,820 INFO  [521330370@qtp-1739580287-1] (com.telefonica.iot.cygnus.handlers.OrionRestHandler.getEvents:258)  - Event put in the channel (id=1145647744, ttl=0)
16 Sep 2015 12:30:19,820 WARN  [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionSink.process:184)  - 
16 Sep 2015 12:30:19,820 INFO  [SinkRunner-PollingRunner-DefaultSinkProcessor] (com.telefonica.iot.cygnus.sinks.OrionSink.process:193)  - Finishing transaction (1442395508-572-0000013907)

我们保留与此问题相同的配置： Fiware Cygnus Error。

尽管 Cygnus 代理从 Context Broker 订阅中正确接收数据，但 Cosmos 没有收到任何数据。

再次感谢您:)

4

1 回答 1

0

与导致您评论令人讨厌的规则部分的原因无关（尽管如此，我认为这是因为我自己在https://jira.fiware.org/browse/HELC-986的错误建议:)），该部分无法评论并且必须添加到配置中：

# Source interceptors, do not change
cygnusagent.sources.http-source.interceptors = ts gi
# TimestampInterceptor, do not change
cygnusagent.sources.http-source.interceptors.ts.type = timestamp
# GroupinInterceptor, do not change
cygnusagent.sources.http-source.interceptors.gi.type = com.telefonica.iot.cygnus.interceptors.GroupingInterceptor$Builder
# Grouping rules for the GroupingInterceptor, put the right absolute path to the file if necessary
# See the doc/design/interceptors document for more details
cygnusagent.sources.http-source.interceptors.gi.grouping_rules_conf_file = /usr/cygnus/conf/grouping_rules.conf

一旦添加了该部分，很可能会出现另一个问题：Cygnus 的性能将非常差（这就是我错误地建议用户在https://jira.fiware.org/browse/HELC-986评论分组的原因功能，试图通过删除处理步骤来提高性能）。原因是最新版本的 Cygnus（0.8.2）还没有准备好处理运行在 Cosmos 端的 HiveServer2（这台服务器最近从旧的 HiveServer1 升级到 HiveServer2）并且每次持久化操作延迟很多。例如：

time=2015-09-21T12:42:30.405CEST | lvl=INFO | trans=1442832138-143-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[150] : Starting transaction (1442832138-143-0000000000)
time=2015-09-21T12:42:30.407CEST | lvl=INFO | trans=1442832138-143-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[236] : Received data ({  "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",  "originator" : "localhost",  "contextResponses" : [    {      "contextElement" : {        "attributes" : [          {            "name" : "temperature",            "type" : "centigrade",            "value" : "26.5"          }        ],        "type" : "Room",        "isPattern" : "false",        "id" : "Room1"      },      "statusCode" : {        "code" : "200",        "reasonPhrase" : "OK"      }    }  ]})
time=2015-09-21T12:42:30.409CEST | lvl=INFO | trans=1442832138-143-0000000000 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[258] : Event put in the channel (id=1966649489, ttl=10)
time=2015-09-21T12:42:30.462CEST | lvl=INFO | trans=1442832138-143-0000000000 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[128] : Event got from the channel (id=1966649489, headers={fiware-servicepath=rooms, destination=other_rooms, content-type=application/json, fiware-service=deleteme2, ttl=10, transactionId=1442832138-143-0000000000, timestamp=1442832150410}, bodyLength=460)
time=2015-09-21T12:42:30.847CEST | lvl=INFO | trans=1442832138-143-0000000000 | function=persist | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionHDFSSink[330] : [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (deleteme2/rooms/other_rooms/other_rooms.txt), Data ({"recvTimeTs":"1442832150","recvTime":"2015-09-21T10:42:30.410Z","entityId":"Room1","entityType":"Room","attrName":"temperature","attrType":"centigrade","attrValue":"26.5","attrMd":[]})
time=2015-09-21T12:42:31.529CEST | lvl=INFO | trans=1442832138-143-0000000000 | function=provisionHiveTable | comp=Cygnus | msg=com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl[185] : Creating Hive external table=frb_deleteme2_rooms_other_rooms_row

(a big timeout)

一种解决方法是配置为hive_host无法访问的 IP 地址，例如fake.cosmos.lab.fiware.org：

time=2015-09-21T12:44:58.278CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[150] : Starting transaction (1442832280-746-0000000001)
time=2015-09-21T12:44:58.280CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[236] : Received data ({  "subscriptionId" : "51c0ac9ed714fb3b37d7d5a8",  "originator" : "localhost",  "contextResponses" : [    {      "contextElement" : {        "attributes" : [          {            "name" : "temperature",            "type" : "centigrade",            "value" : "26.5"          }        ],        "type" : "Room",        "isPattern" : "false",        "id" : "Room1"      },      "statusCode" : {        "code" : "200",        "reasonPhrase" : "OK"      }    }  ]})
time=2015-09-21T12:44:58.280CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=getEvents | comp=Cygnus | msg=com.telefonica.iot.cygnus.handlers.OrionRestHandler[258] : Event put in the channel (id=1640732993, ttl=10)
time=2015-09-21T12:44:58.283CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[128] : Event got from the channel (id=1640732993, headers={fiware-servicepath=rooms, destination=other_rooms, content-type=application/json, fiware-service=deleteme3, ttl=10, transactionId=1442832280-746-0000000001, timestamp=1442832298280}, bodyLength=460)
time=2015-09-21T12:44:58.527CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=persist | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionHDFSSink[330] : [hdfs-sink] Persisting data at OrionHDFSSink. HDFS file (deleteme3/rooms/other_rooms/other_rooms.txt), Data ({"recvTimeTs":"1442832298","recvTime":"2015-09-21T10:44:58.280Z","entityId":"Room1","entityType":"Room","attrName":"temperature","attrType":"centigrade","attrValue":"26.5","attrMd":[]})
time=2015-09-21T12:44:59.148CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=provisionHiveTable | comp=Cygnus | msg=com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl[185] : Creating Hive external table=frb_deleteme3_rooms_other_rooms_row
time=2015-09-21T12:44:59.304CEST | lvl=ERROR | trans=1442832280-746-0000000001 | function=doCreateTable | comp=Cygnus | msg=com.telefonica.iot.cygnus.backends.hive.HiveBackend[77] : Runtime error (The Hive table cannot be created. Hive query='create external table frb_deleteme3_rooms_other_rooms_row (recvTimeTs bigint, recvTime string, entityId string, entityType string, attrName string, attrType string, attrValue string, attrMd array<string>) row format serde 'org.openx.data.jsonserde.JsonSerDe' location '/user/frb/deleteme3/rooms/other_rooms''. Details=Could not establish connection to fake.cosmos.lab.fiware.org:10000/default?user=frb&password=llBl3dQsMhX2sEPtPuf3izUGS92RZo: java.net.UnknownHostException: fake.cosmos.lab.fiware.org)
time=2015-09-21T12:44:59.305CEST | lvl=WARN | trans=1442832280-746-0000000001 | function=provisionHiveTable | comp=Cygnus | msg=com.telefonica.iot.cygnus.backends.hdfs.HDFSBackendImpl[210] : The HiveQL external table could not be created, but Cygnus can continue working... Check your Hive/Shark installation
time=2015-09-21T12:44:59.305CEST | lvl=INFO | trans=1442832280-746-0000000001 | function=process | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionSink[193] : Finishing transaction (1442832280-746-0000000001)

这将允许 Cygnus 继续，尽管 Hive 表没有自动创建，这是一个小问题（无论如何，由于当前与 HiveServer2 不兼容，它们永远不会被创建）。当然，这将在 Cygnus 0.9.0 中修复（将于 2015 年 9 月末发布）。

于 2015-09-21T10:47:30.153 回答