1

我正在尝试确定参数值长度的方差，并在一组相应的参数/值组合之后打印方差值。

例如，dateindate=2007-04-14和的方差值将为 0。in 、和的date=2007-08-19值将为 2.88。id_eveid_eve=479989id_eve=47id_eve=479

从具有公共域和页面值的组值中，我们有一组 URL，这些 URL 被解析以提供一组 URL 的参数/值。

样本数据集：

www.domain.com/page?id_eve=479989&adm=no
www.domain.com/page?id_eve=47&adm=yes
www.domain.com/page?id_eve=479
domain.com/cal?view=month
domain.com/cal?view=day
ww2.domain.com/cal?date=2007-04-14
ww2.domain.com/cal?date=2007-08-19
www.domain.edu/some/folder/image.php?l=adm&y=5&id=2&page=http%3A//support.domain.com/downloads/index.asp&unique=12345
blog.news.org/news/calendar.php?view=day&date=2011-12-10
www.domain.edu/some/folder/image.php?l=adm&y=5&id=2&page=http%3A//.domain.com/downloads/index.asp&unique=12345
blog.news.org/news/calendar.php?view=month&date=2011-12-10

由以下 Python 代码解析：

from collections import defaultdict
from urllib import quote
from urlparse import parse_qsl, urlparse

urls = defaultdict(list)
with open('links.txt') as f:
    for url in f:
        parsed_url = urlparse(url.strip())
        params = parse_qsl(parsed_url.query, keep_blank_values=True)
        for key, value in params:
            urls[parsed_url.path].append("%s=%s" % (key, quote(value)))

# printing results
for url, params in urls.iteritems():
    print url
    for param in params:
        print param

提供：

ww2.domain.com/cal
date=2007-04-14
date=2007-08-19
www.domain.edu/some/folder/image.php
l=adm
y=5
id=2
page=http%3A//support.domain.com/downloads/index.asp
unique=12345
l=adm
y=5
id=2
page=http%3A//.domain.com/downloads/index.asp
unique=12345
domain.com/cal
view=month
view=day
www.domain.com/page
id_eve=479989
adm=no
id_eve=47
adm=yes
id_eve=479
blog.news.org/news/calendar.php
view=day
date=2011-12-10
view=month
date=2011-12-10

需要的附加部分是为每个参数/值组合打印参数值的长度变化，以匹配具有上述输出中定义的类似 URL 的参数（希望阅读清楚）。

  • 用于对 url 进行分组的分组参数
  • 计算参数值的长度
  • 确定长度的变化

所以所需的输出将是：

ww2.domain.com/cal
date=2007-04-14
date=2007-08-19
0
www.domain.edu/some/folder/image.php
l=adm
l=adm
0
y=5
y=5
0
id=2
id=2
0
page=http%3A//support.domain.com/downloads/index.asp
0
unique=12345
0      
page=http%3A//.domain.com/downloads/index.asp
unique=12345
0
domain.com/cal
0
view=month
view=day
1
www.domain.com/page
id_eve=479989
id_eve=47
id_eve=479
2.88
adm=no
adm=yes
0.25
blog.news.org/news/calendar.php
view=day
view=month
1
date=2011-12-10
date=2011-12-10
0
4

1 回答 1

3 
from collections import defaultdict
from urllib import quote
from urlparse import parse_qsl, urlparse

我们需要能够计算方差：

def variance(values):
    mean = sum(values) / float(len(values))
    return sum((elem - mean)**2 for elem in values) / float(len(values))

我们想按“键”进行分组，所以"%s=%s"我们将在defaultdict.

urls = defaultdict(lambda: defaultdict(list))
with open('links.txt') as f:
    for url in f:
        parsed_url = urlparse(url.strip())
        params = parse_qsl(parsed_url.query, keep_blank_values=True)
        for key, value in params:
            urls[parsed_url.path][key].append(quote(value))

然后我们就可以浏览并打印东西

for domain, keys in urls.items():
    print domain
    for key, values in keys.items():
        for value in values:
            print "%s=%s" % (key, value)

        if len(values) > 1:
            print variance(map(len, values))
于 2013-09-21T16:53:29.873 回答