背景(有点读)
我们创建了一个缩短的 url 重定向器(使用 asp.net Url Routing 和一个全面的 404 路由),其中从设备扫描条形码,然后从移动设备重定向到目标站点。我们记录所有传入的请求以进行统计分析。自上周一晚上以来,已经有大量请求和随后的重定向到目标站点。通常我会说“WooHoo!” 但是涌入似乎过于一致,并且来自不同的 IP(但分批)和对不同 URL 的同步请求。只有一组有效的重定向 url 但涌入只击中那些指定的 url(即似乎不是暴力攻击)。这些网址是公开的,但只能从内部员工或从已出版杂志的代码中扫描获得。
来自请求的大多数用户代理基本上来自,显然,iPhone 和大多数 IP 是本地的,这增加了神秘感(将显示示例记录的请求和统计信息)。为了减轻涌入,我们首先实现了一个 ip 阻止规则(在代码级别),如果在 90 秒内有 3 次尝试,我们将 ip 添加到阻止列表中(将该规则从帽子中拉出:oP),我们开始捕获一些. 如果被阻止,我们基本上终止了会话。但这条规则可能会影响合法用户(因为它影响了内部员工......在白名单上工作但尚未完成)所以我们决定采用另一种策略,即插页式广告。
我们实现了登录页面,而不是用户需要交互并单击按钮继续的重定向。我们仍然有针对滥用行为的 ip 阻止规则。因此,如果合法用户通过,他们只需单击按钮即可继续。如果“访问者”在阻止列表中,我们会实施一个验证码例程,以查看他们继续进行的意图是否合法(诚然,这不是一个很好的用户体验)。
我还实现了 robots.txt 拒绝所有。我什至在登录页面中放置了一个隐藏链接,以查看是否是爬虫,但到目前为止,我还没有通过它跟踪任何流量。
我的技术合作伙伴曾认为这可能是实际苹果设备/应用程序/浏览器中的一个故障,导致它重新加载保存的扫描网址(条形码阅读器应用程序,如 RedLaser、i-nigma 等),但这似乎很奇怪这种情况继续发生。
因此,即使在实现了登陆页面之后,我们仍然会获得登陆页面但没有继续/点击的流量。这就是为什么我认为这是一种攻击,不是为了损害网站,而是为了扭曲网站统计数据的可能竞争对手?统计数据在业务中很重要,但重要的是合法的统计数据。我们不想弄乱我们的客户。它也可能是内容的 scapers,但由于用户代理和我的陷阱不起作用,我不知道......它实际上是否是人类在进行点击/扫描......迷路了。
有没有人经历过这样的事情?
Some stats over a period of approximately 2.5 days...
Count HTTP_USER_AGENT
2924 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
506 Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
424 Mozilla/5.0 (iPod; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
401 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
202 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B141
180 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B144
157 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
138 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A405
119 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B142
119 Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
111 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A551
91 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B143
88 Mozilla/5.0 (iPhone; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9A405
86 Mozilla/5.0 (iPod; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9A405
86 Mozilla/5.0 (iPod; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
69 Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B208
56 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A525
54 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B329
47 Mozilla/5.0 (iPod; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
46 Mozilla/5.0 (iPod; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B144
45 Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0 Safari/536.6
43 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B145
30 Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A550
27 Mozilla/5.0 (iPod; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B141
22 Mozilla/5.0 (Linux; Android 4.1.1; SGH-I747M Build/JRO03L) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.169 Mobile Safari/537.22
21 Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; MAM3)
17 Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2)
16 Mozilla/5.0 (iPhone; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176
13 Mozilla/5.0 (iPod; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B176
13 Mozilla/5.0 (BlackBerry; U; BlackBerry 9800; en) AppleWebKit/534.8+ (KHTML, like Gecko) Version/6.0.0.600 Mobile Safari/534.8+
13 Qrafter/7.0 CFNetwork/609.1.4 Darwin/13.0.0
For these stats, I took out source IPs (for courtesy)but referenced which ones were same...just a small sample out approximate 6.5K requests
Count REMOTE_ADDR HTTP_USER_AGENT
5 A.B.C.D Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
13 A.B.C.D Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
2 A.B.C.D Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
6 A.B.C.D Mozilla/5.0 (iPod; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
1 B.C.D.E Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
3 B.C.D.E Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B145
10 B.C.D.E Mozilla/5.0 (iPod; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A406
15 C.D.E.F Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
31 C.D.E.F Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A405
20 C.D.E.F Mozilla/5.0 (iPhone; CPU iPhone OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B141
2 D.E.F.G Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
6 D.E.F.G Mozilla/5.0 (iPod; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
2 D.E.F.G Mozilla/5.0 (iPod; CPU iPhone OS 6_0_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A523
18 E.F.G.H Mozilla/5.0 (iPhone; CPU iPhone OS 6_0_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A550
49 E.F.G.H Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146
15 E.F.G.H Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
13 E.F.I.J Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
11 E.F.I.J Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10A403
11 E.F.I.J Mozilla/5.0 (iPod; CPU iPhone OS 5_0_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9A405
3 E.F.I.J Mozilla/5.0 (iPod; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Mobile/9B206
Sample of typical sequntial flood
URL HTTP_USER_AGENT REMOTE_ADDR LogDate
http://SITEURL/UoetZx Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:45 AM
http://SITEURL/2NedO0 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:45 AM
http://SITEURL/33hCWl Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:44 AM
http://SITEURL/e11LzG Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:44 AM
http://SITEURL/bQx5Bu Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:43 AM
http://SITEURL/BtrZ3m Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:43 AM
http://SITEURL/cxfmr1 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:43 AM
http://SITEURL/KztehQ Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:42 AM
http://SITEURL/O19sq3 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:42 AM
http://SITEURL/e6Dlwb Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:42 AM
http://SITEURL/GGQ4ZO Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:41 AM
http://SITEURL/jjr_rM Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 A.B.C.D 3/21/13 1:40 AM
http://SITEURL/yIzVel Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:40 AM
http://SITEURL/D8M0_Y Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:39 AM
http://SITEURL/-GqaX9 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:38 AM
http://SITEURL/9o0Bv8 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:37 AM
http://SITEURL/65_ce8 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:35 AM
http://SITEURL/33hCWl Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:34 AM
http://SITEURL/2NedO0 Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:33 AM
http://SITEURL/UoetZx Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:33 AM
http://SITEURL/fknpPf Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:31 AM
http://SITEURL/tLEI3S Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:30 AM
http://SITEURL/MgOvvm Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:29 AM
http://SITEURL/MlJVua Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:28 AM
http://SITEURL/UcRIZj Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:27 AM
http://SITEURL/xZy-KP Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:26 AM
http://SITEURL/sXswln Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:25 AM
http://SITEURL/aQJrWx Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:24 AM
http://SITEURL/_sBrUw Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:23 AM
http://SITEURL/V7H9mK Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:22 AM
http://SITEURL/lchtkL Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:21 AM
http://SITEURL/WY7g1T Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:20 AM
http://SITEURL/bQx5Bu Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:19 AM
http://SITEURL/FznevZ Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_2 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Mobile/10B146 B.C.D.E 3/21/13 1:17 AM