0

我尝试对日志文件使用 python 创建过滤器，例如

 Thu Oct  4 23:14:40 2012 [pid 16901] CONNECT: Client "66.249.74.228"
 Thu Oct  4 23:14:40 2012 [pid 16900] [ftp] OK LOGIN: Client "66.249.74.228", anon     password "googlebot@google.com"
 Thu Oct  4 23:17:42 2012 [pid 16902] [ftp] FAIL DOWNLOAD: Client "66.249.74.228",   "/pub/10.5524/100001_101000/100039/Assembly-2011/Pa9a_assembly_config4.scafSeq.gz",  14811136 bytes, 79.99Kbyte/sec
 Fri Oct  5 00:04:13 2012 [pid 25809] CONNECT: Client "66.249.74.228"
 Fri Oct  5 00:04:14 2012 [pid 25808] [ftp] OK LOGIN: Client "66.249.74.228", anon password "googlebot@google.com"
 Fri Oct  5 00:07:16 2012 [pid 25810] [ftp] FAIL DOWNLOAD: Client "66.249.74.228", "/pub/10.5524/100001_101000/100027/Raw_data/PHOlcpDABDWABPE/090715_I80_FC427DJAAXX_L8_PHOlcpDABDWABPE_1.fq.gz", 14811136 bytes, 79.99Kbyte/sec
 Fri Oct  5 00:13:19 2012 [pid 27354] CONNECT: Client "1.202.186.53"
 Fri Oct  5 00:13:19 2012 [pid 27353] [ftp] OK LOGIN: Client "1.202.186.53", anon password "mozilla@example.com"
 Fri Oct  5 00:13:33 2012 [pid 27355] [ftp] FAIL DOWNLOAD: Client "1.202.186.53", "/pub", 0.00Kbyte/sec
 Fri Oct  5 00:26:04 2012 [pid 341] [ftp] OK DOWNLOAD: Client "210.72.156.68", "/pub/10.5524/100001_101000/100030/RNA-Seq/Mgo_2.fq.gz", 1985229528 bytes, 85.87Kbyte/sec
 Fri Oct  5 00:55:45 2012 [pid 2766] CONNECT: Client "157.82.250.217"
 Fri Oct  5 00:55:45 2012 [pid 2765] [ftp] OK LOGIN: Client "157.82.250.217", anon password "mozilla@example.com"
 Fri Oct  5 00:56:05 2012 [pid 2767] [ftp] FAIL DOWNLOAD: Client "157.82.250.217", "/pub/10.5524/100001_101000/100036/Gene_catalogue/Gene_catalogue.pep", 1638400 bytes, 81.81Kbyte/sec
 Fri Oct  5 00:57:27 2012 [pid 3056] CONNECT: Client "157.82.250.217"
 Fri Oct  5 00:57:27 2012 [pid 3055] [ftp] OK LOGIN: Client "157.82.250.217", anon password "-wget@"

日志文件中有一些机器人访问记录，那么如何使用python过滤器来实现真人访问记录。我已经建立了一个过滤器来获取每周记录，所以你能帮我在里面添加它吗？

import time
f= open("/opt/CLiMB/Storage1/log/vsftp.log")
def OnlyRecent(line):
    if  time.strptime(line.split("[")[0].strip(),"%a %b %d %H:%M:%S %Y")>  time.gmtime(time.time()-(60*60*24*7)): 
        return True
    return False
filename= time.strftime('%Y%m%d')+'.log'
f1= open(filename,'w')
for line in f:
    if OnlyRecent(line):
            print line
            f1.write(line)
f.close()
f1.close()
4

2 回答 2

0

您可以按某些标识符对事件进行分组。我想到了 pid，但您日志中的所有行似乎都有不同的 pid。您可以为每个组使用 IP 地址并在找到时启动新组CONNECT: Client "[IP]"，但如果来自某些 IP 地址的客户端一次有多个会话，这将失败。如果没有会话标识符，很难确定哪些线路用作一个会话（组）。

当您对事件进行分组时，对于每个组，您必须检查此事件中是否是机器人留下的“标志”，例如："anon password "googlebot@google.com"

于 2012-10-05T06:12:44.940 回答
0

如果您确定使用您的系统的客户端实际上是机器人，通过查看他的密码（googlebot@google.com看起来像一个实际的机器人），那么您可以拆分一个字符串并查看第二部分是否包含机器人 e - 邮件： 

# Add additional robot e-mails here
robot_emails = ["googlebot@google.com"]

def isRobotRecord(line):

    for email in robot_emails:
        if email in line.split("Client")[1]:
            return True
        else:
            return False
于 2012-10-05T05:47:00.710 回答