1

I've come across Fluentd. Why would you use such a thing when its easy enough to store raw data on a db directly? I might be misunderstanding the use of the technology here. Glad to hear some feedback.

Why would anyone want to go through another layer, when its easy enough to capture and store raw data in your own a data store?

Consider this scenario. I want to store page views. Raw data is stored in an RDBMS and formatted data is stored in Mongodb This is a short description of my current setup:

When a user visits my site. My application (Rails) resolves the IP to a match a country. After the IP is resolved, I store the raw data into an RDBMS. I have a worker/cron running to process all the raw data into a Mongo document every hour. Why would I need fluentd there? What are the benefits of having a logging framework in this instance?

4

1 回答 1

2
  1. 您无需创建/维护自己的工作人员即可在您的第一个 RDBMS 和 Mongo 之间移动内容。
  2. 您可以轻松实现将数据移动到 Mongo 的过程的并行化和冗余。您可以将其构建到您的工作人员/cron 工作中，但您为什么要重新发明轮子？
  3. 你问为什么有人会想要另一层。您的 worker/cron 工作是另一层，但比 Fluentd 测试的要少。
  4. 你会得到一堆免费的插件，所以如果你想开始将数据添加到 Mongo 之外的其他地方（即 Storm、S3、HDFS 等），你可以通过编辑配置文件而不是编写来轻松地做到这一点一堆代码自己。
  5. 您有一堆免费的内置选项，例如刷新数据的频率/刷新数据的大小。
  6. 最重要的是：您将整个日志记录/数据输入工作流程从您的应用程序框中卸载，因此如果您的应用程序框上的数据插入过程出现任何问题，问题将出现并在您的 Fluent 日志聚合器框上处理，而不是在您的应用程序框中。
于 2013-05-13T17:51:06.560 回答