1

我itertools.groupby用来解析一个短的制表符分隔的文本文件。文本文件有几列，我要做的就是将所有x在特定列中具有特定值的条目分组。下面的代码对名为 的列执行此操作name2，查找变量中的值x。我尝试使用csv.DictReaderand来做到这一点itertools.groupby。在表中，有8行符合此条件，因此应返回 8 个条目。而是groupby返回两组条目，一组带有单个条目，另一组带有 7 个条目，这似乎是错误的行为。我在下面对相同的数据手动进行匹配并得到正确的结果：

import itertools, operator, csv
col_name = "name2"
x = "ENSMUSG00000002459"
print "looking for entries with value %s in column %s" %(x, col_name)
print "groupby gets it wrong: "
data = csv.DictReader(open(f), delimiter="\t", fieldnames=fieldnames)
for name, entries in itertools.groupby(data, key=operator.itemgetter(col_name)):
    if name == "ENSMUSG00000002459":
        wrong_result = [e for e in entries]
        print "wrong result has %d entries" %(len(wrong_result))
print "manually grouping entries is correct: "
data = csv.DictReader(open(f), delimiter="\t", fieldnames=fieldnames)
correct_result = []
for row in data:
    if row[col_name] == "ENSMUSG00000002459":
        correct_result.append(row)
print "correct result has %d entries" %(len(correct_result))

我得到的输出是：

looking for entries with value ENSMUSG00000002459 in column name2
groupby gets it wrong: 
wrong result has 7 entries
wrong result has 1 entries
manually grouping entries is correct: 
correct result has 8 entries

这里发生了什么？如果groupby真的是分组，似乎我应该只得到一组条目x，而是返回两个。我无法弄清楚这一点。 编辑：啊，它应该被排序。

4

3 回答 3

3

According to the documentation, groupby() groups only consecutive occurrences of the same key.

于 2012-10-24T00:05:40.800 回答
3

您将要更改代码以强制数据按关键顺序排列...

data = csv.DictReader(open(f), delimiter="\t", fieldnames=fieldnames)
sorted_data = sorted(data, key=operator.itemgetter(col_name))
for name, entries in itertools.groupby(data, key=operator.itemgetter(col_name)):
    pass # whatever

不过，主要用途是当数据集很大并且数据已经按关键顺序排列时，所以当您无论如何都必须排序时，使用 adefaultdict更有效

from collections import defaultdict
name_entries = defaultdict(list)
for row in data:
    name_entries[row[col_name]].append(row)
于 2012-10-24T00:13:16.007 回答
2

I don't know what your data looks like but my guess is it's not sorted. groupby works on sorted data

于 2012-10-24T00:05:32.990 回答