我编写了一个程序来对少量数据进行一些处理。其中涉及三个步骤：
- 读取数据
- 做处理
- 将数据写入文件。
在运行代码的前两个步骤成功完成。（由于我的笔记本电脑配置数据量很大，我正在使用 linux 中的交换空间来完成这项工作）。
现在第三步：数据写入文件成功。但是我的代码卡在了这条线上feat.close()（壮举是文件指针）。当我在进程运行时打开文件时，正在写入完整的数据，但我的文件没有关闭。
代码：
#!/usr/bin/env python
from __future__ import print_function
import pickle
import sys
import numpy as np
import posTagsToTriGramFrequency as pt
import itertools
import gc
if sys.argv[1] == '-h':
print("Usage: ./featureSelection <n(authorLimit)> <k(SD weight)> <nGramSize> <folder>")
sys.exit()
n = int(sys.argv[1])
k = int(sys.argv[2])
nGramSize = int(sys.argv[3])
folder = sys.argv[4]
print('reading features')
feat = open('../../../../dataDump/'+ folder +'/features.dump','r')
features = {}
tagsnGram = [tuple(x) for x in itertools.product(pt.getTags(), repeat=nGramSize)]
gramdict = {}
for gram in tagsnGram:
gramdict[gram] = []
flag = 1
author = ''
for line in feat:
if(line == '\n'):
flag = 1
elif flag == 1:
author = line.split('/')[-1][:-1]
print(line, end='')
features[author] = gramdict.copy()
flag = 0
else:
tagsFreq = iter(line.split())
for tag in tagsnGram:
features[author][tag].append(float(tagsFreq.next()))
feat.close()
print('Calculating waht to delete')
nflag = 0
kflag = 0
toDel = []
for tagGram in tagsnGram:
nflag = 0
for author in features:
kflag = 0
for doc in features[author][tagGram]:
if doc == 0 : kflag += 1
if kflag >= k:
nflag += 1
break
if nflag >= n:
toDel.append(tagGram)
break
for item in toDel:
for author in features:
del features[author][item]
f = open('../../../../dataDump/'+ folder +'/tagsInfo.dump','w')
f.write('k:' + str(k) + ',\t n:' + str(n) + '\n')
f.write('Deleted tags:\n')
for item in toDel:
f.write(str(item) + ' ')
f.write('\n\nSelected Tags:\n')
for tagGram in features.itervalues().next():
f.write(str(tagGram) + ' ')
f.close()
print("Writing Back Features")
feat = open('../../../../dataDump/'+ folder +'/selectedFeatures.dump','w')
for author in features.keys():
feat.write(author + '\n')
print(author)
for tag in features[author]:
for doc in features[author][tag]:
feat.write(str(doc) + ' ')
feat.write('\n')
feat.write('\n')
del features[author]
#gc.collect()
print('Closing File')
feat.close()
见最后一行。在我的控制台Closing file上正在打印，但之后我的代码被卡住了。
我的控制台输出：
abhi@abhi-me~/Projects/workspace/irProject/completepythonbased/authAttrib (irProject)>>./featureSelection.py 35 125 3 3GramFreq
reading features
Ajit_Popat
Mukund_Mehta
Parajit_Patel
Priyadarshi
Kumarpad_Desai
Bhaven_Kacchi
Shantibhai_Agrawat
Pravin_Darji
Ankit_Trivedi
Sharad_Rawal
Tushar_Shukla
Chandrakant_Mehta
Jay_Vasavda
Dolat_Bhatt
Munindra
Mrugesh_Vaishnav
Kulinchandra_Yagnik
Zaverilal_Mehta
Priti_Shah
Vasant_Mistri
Vatsal_Vasani
Dinesh_Mistri
Devesh_Mehta
Dhaval_Mehta
Urvish_Kothari
Madhusudan_Parekh
Vihari_Chaya
Virendra_Kapoor
Mukul_Choksi
Joravarsinh_Jadav
Ashok_Dave
Nasir_Ismaeli
Joban_Pandit
Priyakant_Parikh
Sudarshan_Upadhyay
Gajendra_Shah
Altaf_Patel
Bhalchandra_Jani
Shashin
Hansal_Bhachech
Calculating waht to delete
Writing Back Features
Pravin_Darji
Ajit_Popat
Kulinchandra_Yagnik
Sharad_Rawal
Madhusudan_Parekh
Shantibhai_Agrawat
Gajendra_Shah
Hansal_Bhachech
Vihari_Chaya
Virendra_Kapoor
Sudarshan_Upadhyay
Priyadarshi
Tushar_Shukla
Dolat_Bhatt
Urvish_Kothari
Vasant_Mistri
Mukund_Mehta
Zaverilal_Mehta
Kumarpad_Desai
Vatsal_Vasani
Bhaven_Kacchi
Mrugesh_Vaishnav
Bhalchandra_Jani
Priyakant_Parikh
Chandrakant_Mehta
Mukul_Choksi
Joravarsinh_Jadav
Munindra
Joban_Pandit
Devesh_Mehta
Priti_Shah
Ankit_Trivedi
Dinesh_Mistri
Dhaval_Mehta
Ashok_Dave
Nasir_Ismaeli
Parajit_Patel
Jay_Vasavda
Altaf_Patel
Shashin
Closing File
^C
[1]+ Killed ./featureSelection.py 35 125 3 3GramFreq
为什么会这样？
你可以在这里找到 strace 。
编辑： 我试图在关闭后打印一些东西并且它正在打印，这意味着问题在于退出程序。它使用了大约 3 GB 的 RAM 和 3 GB 的交换空间 @Justing。填充这个内存需要 10-20 分钟，清除这个内存我等了大约 2 小时，我认为有问题。 我上传了上面的strace，请看。