16

我想将 Loggly 设置为在 AWS Elastic Beanstalk 上运行，但找不到有关如何执行此操作的任何信息。是否有任何指南，或有关如何开始的一般指南？

4

8 回答 8

6

对于 papertrailapp.com（我更喜欢它而不是 loggly），我就是这样做的。在您创建的/ebextensions文件夹（查看更多信息）logs.config中，指定：

container_commands:
  01-set-correct-hostname:
    command: hostname www.example.com
  02-forward-rsyslog-to-papertrail:
    # https://papertrailapp.com/systems/setup
    command: echo "*.* @logs.papertrailapp.com:55555" >> /etc/rsyslog.conf
  03-enable-remote-logging:
    command: echo -e "\$ModLoad imudp\n\$UDPServerRun 514\n\$ModLoad imtcp\n\$InputTCPServerRun 514\n\$EscapeControlCharactersOnReceive off" >> /etc/rsyslog.conf
  04-restart-syslog:
    command: service rsyslog restart

55555应替换为 papertrailapp.com 提供的 UDP 端口号。每次新实例引导后，都会应用此配置。然后，在你的log4j.properties：

log4j.rootLogger=WARN, SYSLOG
log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.facility=local1
log4j.appender.SYSLOG.header=true
log4j.appender.SYSLOG.syslogHost=localhost
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.ConversionPattern=[%p] %t %c: %m%n

我不确定这是否是最佳解决方案。在jcabi-beanstalk-maven-plugin中阅读有关此机制的更多信息

于 2012-11-08T09:09:02.030 回答
6

您还可以使用 loggly 本身的安装脚本。下面的设置遵循https://www.loggly.com/docs/configure-syslog-script/上的旧版设置说明，并进行了细微更改（没有确认提示，sudo 命令被替换，因为没有 tty 可用）

（编辑：更新的链接，现在在 loggly 文档中似乎是一个过时的解决方案）

将以下脚本放入 .ebextensions/loggly.config

将 TOKEN 和 ACCOUNT 替换为您自己的。

#
# Install loggly.com on AWS Elastic Beanstalk
# Tested with node.js environment
# Save this file as .ebextensions/loggly.config
# Deploy per normal scripts or aws.push. To help debug the push, ssh & tail /var/log/cfn-init.log
# See Also /var/log/eb-tools.log
#

commands:
  01_loggly_dl:
    command: wget -q -O /tmp/loggly.py https://www.loggly.com/install/configure-syslog.py
  02_loggly_config:
    command: su --session-command="python /tmp/loggly.py setup --auth TOKEN --account ACCOUNT --yes"
于 2014-03-26T02:38:09.320 回答
3

以下是 loggly 支持站点的链接，用于将 syslogd 与 loggly 一起使用： http ://wiki.loggly.com/loggingconfiguration

或将 loggly api 与您自己的应用程序一起使用： http ://wiki.loggly.com/apidocumention

于 2012-07-22T15:42:05.187 回答
2

这是我刚刚开始使用的 Loggly 的 elasticbeanstalk 配置，这要归功于来自该线程的指针和日志记录 SaaS 供应商设置说明。[Loggly 配置管理，Papertrail rsyslog ]

将文件另存为 .ebextensions 目录中的 loggly.config 并确保检查 YAML 格式约定（无制表符等）。根据需要将您的 Loggly TCP 端口号、用户名、密码和域名替换到尖括号中。

请注意，对于 AWS ruby​​ 版本的 elasticbeanstalk，EC2 /etc/rsyslog 设置可能存在差异。例如，如果 /etc/rsyslog.d 已经存在，并且已经有“$IncludeConfig /etc/rsyslog.d/*.conf”指令，那么命令“01-forward-rsyslog-to-loggly:”可以是删除。

按照普通脚本或 aws.push 进行部署。为了帮助调试推送，ssh & tail /var/log/cfn-init.log

files:
  "/etc/rsyslog.d/90-loggly.conf" :
    mode: "000664"
    owner: root
    group: root
    content: |
      # ### begin forwarding rule ###
      # The statement between the begin ... end define a SINGLE forwarding
      # rule. They belong together, do NOT split them. If you create multiple
      # forwarding rules, duplicate the whole block!
      # Remote Logging (we use TCP for reliable delivery)
      #
      # An on-disk queue is created for this action. If the remote host is
      # down, messages are spooled to disk and sent when it is up again.
      $WorkDirectory /var/lib/rsyslog # where to place spool files
      $ActionQueueFileName fwdRule1 # unique name prefix for spool files
      $ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
      $ActionQueueSaveOnShutdown on # save messages to disk on shutdown
      $ActionQueueType LinkedList   # run asynchronously
      $ActionResumeRetryCount -1    # infinite retries if host is down
      *.* @@logs.loggly.com:<yourportnum>   # !!!Loggly supplied port number for each app!!!
      # ### end of the forwarding rule ###
    encoding: plain
  "/tmp/loggly.py" :
    mode: "000755"
    owner: root
    group: root
    content: |
      import json
      import sys
      import urllib2
      '''
      Auto-authenticate Syslog TCP inputs.
      Usage: python inputs.py -u user -p pass -s subdomain
      '''
      state = None
      params = {}
      for i in range(len(sys.argv)):
         arg = sys.argv[i]
         if state:
             params[state] = arg
             state = None

         if arg == '--username' or arg == '-u':
             state = 'username'

         if arg == '--password' or arg == '-p':
             state = 'password'

         if arg == '--subdomain' or arg == '-s':
             state = 'subdomain'
      url = 'https://%s.loggly.com/api/inputs' % params['subdomain']
      password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
      password_mgr.add_password(None, url, params['username'], params['password'])
      handler = urllib2.HTTPBasicAuthHandler(password_mgr)
      opener = urllib2.build_opener(handler)
      opener.open(url)
      urllib2.install_opener(opener)
      inputs = json.loads(urllib2.urlopen(url).read())
      for input in inputs:
         if input['service']['name'] == 'syslogtcp':
             url = 'https://%s.loggly.com/api/inputs/%d/adddevice' % \
                 (params['subdomain'], input['id'])
             response = urllib2.urlopen(url, {}).read()
             print response
    encoding: plain

commands:
  01-forward-rsyslog-to-loggly:
    # http://loggly.com/support/sending-data/logging-from/syslog/rsyslog/cd
    command: test "$(grep -s '90-loggly.conf' /etc/rsyslog.conf)" == "" && echo -e "\n# Include the loggly.conf file\n\$IncludeConfig /etc/rsyslog.d/90-loggly.conf" >> /etc/rsyslog.conf
  02-restart-syslog:
    command: service rsyslog restart
  03-inform_loggly:
    command: "python /tmp/loggly.py -u <Yourloginname> -p <Yourpassword> -s <Yourdomainname>"
于 2013-04-25T23:20:34.747 回答
1

通常，/etc/rsyslog.config 最后会有一个“$IncludeConfig /etc/rsyslog.d/*.conf” - 因此您可以使用 .ebextensions 文件的“files:”部分简单地引入您自己的配置文件. 无论您是否部署到新服务器，这都有效。

对于 ruby​​ production.log，您可能在 .ebextensions/01loggly.config 文件中有类似的内容。请注意，这也会将您的 beanstalk 环境名称作为 loggly 标记。

# For docs on eb configs, see http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html
# This set of commands sets up loggly forwarding
files:
   "/etc/rsyslog.d/myapp-loggly.conf" :
      mode: "000664"
      owner: root
      group: root
      content: |
         $template LogglyFormat,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [yourlogglyid@41058 tag=`{ "Ref" : "AWSEBEnvironmentName" }`] %msg%\n"
         *.* @@logs-01.loggly.com:514;LogglyFormat

         # One time config
         $ModLoad imfile
         $InputFilePollInterval 10 
         $PrivDropToGroup adm
         $WorkDirectory /var/spool/rsyslog

         # Add a tag for file events

         # For production.log
         $InputFileName /var/app/support/logs/production.log
         $InputFileTag production-log
         $InputFileStateFile stat-production-log #this must be unique for each file being polled
         $InputFileSeverity info
         $InputFilePersistStateInterval 20000
         $InputRunFileMonitor
         # Send to Loggly then discard
         if $programname == 'myapp-production-log' then @@logs-01.loggly.com:514;LogglyFormat
         if $programname == 'myapp-production-log' then ~

      encoding: plain
commands:
   00-make-work-directory:
      command: mkdir -p /var/spool/rsyslog
   01-restart-syslog:
      command: service rsyslog restart

对于 Tomcat，您可以在 .ebextesions/01logglyg.config 文件中执行以下操作：

# For docs on eb configs, see http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html
# This set of commands sets up loggly forwarding
files:
    "/etc/rsyslog.d/mytomcatapp-loggly.conf" :
        mode: "000664"
        owner: root
        group: root
        content: |
            $template LogglyFormat,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [yourlogglygidhere@41058 tag=`{ "Ref" : "AWSEBEnvironmentName" }`] %msg%\n"
            *.* @@logs-01.loggly.com:514;LogglyFormat

            # One time config
            $ModLoad imfile
            $InputFilePollInterval 10 
            $PrivDropToGroup adm
            $WorkDirectory /var/spool/rsyslog

            # catalina.log
            $InputFileName /var/log/tomcat7/catalina.log
            $InputFileTag catalina-log
            $InputFileStateFile stat-catalina-log
            $InputFileSeverity info
            $InputFilePersistStateInterval 20000
            $InputRunFileMonitor
            if $programname == 'catalina-log' then @@logs-01.loggly.com:514;LogglyFormat
            if $programname == 'catalina-log' then ~

            # catalina.out
            $InputFileName /var/log/tomcat7/catalina.out
            $InputFileTag catalina-out
            $InputFileStateFile stat-catalina-out
            $InputFileSeverity info
            $InputFilePersistStateInterval 20000
            $InputRunFileMonitor
            if $programname == 'catalina-out' then @@logs-01.loggly.com:514;LogglyFormat
            if $programname == 'catalina-out' then ~

            # host-manager.log
            $InputFileName /var/log/tomcat7/host-manager.log
            $InputFileTag host-manager
            $InputFileStateFile stat-host-manager
            $InputFileSeverity info
            $InputFilePersistStateInterval 20000
            $InputRunFileMonitor
            if $programname == 'host-manager' then @@logs-01.loggly.com:514;LogglyFormat
            if $programname == 'host-manager' then ~

            # initd.log
            $InputFileName /var/log/tomcat7/initd.log
            $InputFileTag initd
            $InputFileStateFile stat-initd
            $InputFileSeverity info
            $InputFilePersistStateInterval 20000
            $InputRunFileMonitor
            if $programname == 'initd' then @@logs-01.loggly.com:514;LogglyFormat
            if $programname == 'initd' then ~

            # localhost.log
            $InputFileName /var/log/tomcat7/localhost.log
            $InputFileTag localhost-log
            $InputFileStateFile stat-localhost-log
            $InputFileSeverity info
            $InputFilePersistStateInterval 20000
            $InputRunFileMonitor
            if $programname == 'localhost-log' then @@logs-01.loggly.com:514;LogglyFormat
            if $programname == 'localhost-log' then ~

            # manager.log
            $InputFileName /var/log/tomcat7/manager.log
            $InputFileTag manager
            $InputFileStateFile stat-manager
            $InputFileSeverity info
            $InputFilePersistStateInterval 20000
            $InputRunFileMonitor
            if $programname == 'manager' then @@logs-01.loggly.com:514;LogglyFormat
            if $programname == 'manager' then ~

        encoding: plain
commands:
    00-make-work-directory:
        command: mkdir -p /var/spool/rsyslog
    01-restart-syslog:
        command: service rsyslog restart

这个配置对我有用——虽然我还没有确定如何让多行条目进入 Loggly 中的单个条目。

于 2014-03-24T19:51:46.280 回答
1

我知道这是一个相当老的问题，但我发现答案真的没有回答这个问题，或者只是在实施时没有正常工作。我发现这可行（文件.ebextenstions/02loggly.config）：

container_commands:
  01-transform-rsyslog.conf:
    command: sed "s/NODE_ENV/$NODE_ENV/g" scripts/22-loggly.conf.temp > scripts/22-loggly.conf
  02-setup-rsyslog.conf:
    command: cp scripts/22-loggly.conf /etc/rsyslog.d/22-loggly.conf
  03-restart:
    command: /sbin/service rsyslog restart

“01-transform-rsyslog.conf”步骤是可选的；我用它在文件中通过 NODE_ENV 设置标签。“22-loggly.conf.temp”是“22-loggly.conf”文件的修改版本，当您运行 linux 源设置脚本 ( https://www .loggly.com/install/configure-syslog.py）。我刚刚将它安装在 ec2 实例上并复制了文件。

注意我必须在我的服务命令前面加上'/sbin'，因为没有它对我来说是失败的。此外，这会在每次部署时重新启动系统日志，这应该没问题。

现在您只需要确保您的应用程序记录到 syslog。对于 Java，它将是 log4j 或类似的。对于 Node.js（这是我正在使用的）， rconsole 有效（https://github.com/tblobaum/rconsole）。

于 2014-05-22T13:16:17.987 回答
0

我尝试过的所有事情似乎都不起作用，而且 loggly 文档非常混乱！我希望这会对某人有所帮助，这就是我让它工作的方式。

将以下内容粘贴到 .ebextensions/loggly.config

files:
  "/etc/rsyslog.conf" :
    mode: "000644"
    owner: root
    group: root
    content: |
      $ModLoad imfile
      $InputFilePollInterval 10
      $PrivDropToGroup adm

      # Input for FILE.LOG
      $InputFileName /var/app/current/PATH_TO_YOUR_LOG_FILE
      $InputFileTag social_php:
      $InputFileStateFile stat-social_php #this must be unique for each file being polled
      $InputFileSeverity info
      $InputRunFileMonitor

      #Add a tag for events from this file
      $template LogglyFormatsocial_php,"<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [TOKEN@41058 tag=\"php_log\"] %msg%\n"

      if $programname == 'social_php' then @@logs.loggly.com:37146;LogglyFormatsocial_php
      if $programname == 'social_php' then ~
      *.*    @@logs.loggly.com:37146


commands:
  01-restart-syslog:
    command: service rsyslog restart
  • 将所有 social_php 实例替换为对您的应用程序有意义的标记。
  • 将 /var/app/current/PATH_TO_YOUR_LOG_FILE 替换为您的日志文件位置
于 2014-02-23T16:14:32.587 回答
0

按照我在 elasticbeanstalk 中的 loggly 配置。对于 Linux + log4j

关于 .ebextensions 文件配置

container_commands:
  01_configure_sudo_access:
    command: sed -i -- 's/ requiretty/ \!requiretty/g' /etc/sudoers
  02_loggy_configure:
    command: sudo python .ebextensions/scripts/loggly_config.py
  03_restore_sudo_access:
    command: sed -i -- 's/ \!requiretty/ requiretty/g' /etc/sudoers

用于默认 AMI 的 python 中的 Loggly 脚本：

import os

rsyslog_path = '/etc/rsyslog.conf'
loggly_file_path = '/etc/rsyslog.d/22-loggly.conf'

class LogglyConfig:

    def __init__(self):
        self.__linux_log()
        self.__config_loggly_for_log4j()

    def __linux_log(self):
        #not installed on this machine
        if not os.path.exists(loggly_file_path):
            os.system('rm -f configure-linux.sh')
            os.system('wget https://www.loggly.com/install/configure-linux.sh')
            os.system('sudo bash configure-linux.sh -a DOMAIN -t TOKEN -u USER -p PASSWORD -s')


    def __config_loggly_for_log4j(self):
        f = open(rsyslog_path,'r')
        file_text = f.read()
        f.close()
        file_text = file_text.replace('#$ModLoad imudp', '$ModLoad imudp')
        file_text = file_text.replace('#$UDPServerRun 514', '$UDPServerRun 514')
        f = open(rsyslog_path,'w')
        f.write(file_text)
        f.close()

        os.system('service rsyslog restart')

LogglyConfig()

在您的 java 项目的 log4j.properties 中

log4j.rootLogger=INFO, SYSLOG

log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.SyslogHost=localhost
log4j.appender.SYSLOG.Facility=Local3
log4j.appender.SYSLOG.Header=true
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.ConversionPattern=java %d{ISO8601} %p %t %c{1}.%M - %m%n
于 2015-06-24T01:43:41.483 回答