0

我有大约 20000 个文本文件，编号为 1.txt、2.txt 等等。

现在，我正在创建一个字典 d，其中包含文件 5.txt、10.txt、15.txt 等的文件路径。

d[value]=filepath

ex:
d[5]=d:/articles/5.txt
d[45]=d:/articles/45.txt

我有一个包含 500 个单词的文本文件“temp.txt”

vs
mln
money

等等..

现在对于字典“d”中的每个文本文件，我需要记录列表中所有单词的出现频率。

所以，我创建了一个 d2[word][file]=count 形式的嵌套字典（这是正确的方法吗？）

where, d2[vs][5]=number of times "vs" occurs in 5.txt

简而言之，对于每个文件，我都会遍历单词列表并计算其出现次数。

我如何创建 d2？

我的错误代码是：

import collections, sys, os, re

sys.stdout=open('3.txt','w')
from collections import Counter
from glob import glob

folderpath='d:/individual-articles'
folderpaths='d:/individual-articles/'
counter=Counter()
filepaths = glob(os.path.join(folderpath,'*.txt'))

# returns the next word in the file
def words_generator(fileobj):
    for line in fileobj:
        for word in line.split():
            yield word

d= collections.defaultdict(list)

#to print the filenames:(creation of d)

with open('topics.txt','r') as f:
    for line in f.readlines():
        value=(line.split('~')[0])
        if int(value)%5==0:
            file=folderpaths+value+'.txt'
            d[value].append(file)

d2= collections.defaultdict(list)

for file in filepaths:
    f = open(file,"r")
    words = words_generator(f)
    for word in words:
        if  file in d[file]:
            d2[word][file]+= 1              
#i have no idea how to go further, beyond this point.

请帮忙！！

4

2 回答 2

1

像这样的东西：

import os
from collections import Counter,defaultdict
d2 = defaultdict(dict)
word_list = ['vs', 'mln', 'money']
for fil in d.values():
    with open(fil[0]) as f:
       path, name = os.path.split(fil[0])
       words_c = Counter([word for line in f for word in line.split()])
       for word in word_list:
           d2[word][name] = words_c[word]

现在访问 d2 为：

d2['vs']['5.txt']
于 2013-07-03T12:47:58.877 回答
1

如果您可以交换字典，则可以将@Ashwini Chaudhary 的答案简化一些（尽管效率可能会降低一些）：

from collections import Counter, defaultdict

with open('temp.txt') as f:
    word_list = set(f.read.split())

d2 = defaultdict(Counter)
for n in range(number_of_files):
    with open('{}.txt'.format(n)) as f:
        d2[fil] = Counter([word for word in f.read().split() if word in word_list])
于 2013-07-03T13:00:31.607 回答