我目前正在使用线程和所有这些编写基于 python 的数据报服务器。
我遇到了以下问题:我正在使用多个分配线程将传入的包分配给不同的处理线程。在处理线程中,我使用 threading.local() 来跟踪线程局部变量。
我目前正在测试我的服务器在高负载期间的反应(大约 2 秒内 2000 个数据包),并且遇到了 local()-Object 的奇怪行为。
似乎它在一段时间内工作得很好,然后,在某个时候,它抛出了一个异常:
Exception in thread 192.168.1.102: # <-- This is the Processing Thread
Traceback (most recent call last):
File "/opt/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/volume1/home/Max/Python/MyThread/pyProcThread.py", line 37, in run
print self.loc.father
AttributeError: 'thread._local' object has no attribute 'father'
# The following three lines are debug
# This is the Allocation thread that has called the Processing thread
<Thread(192.168.1.102, started 1106023568)>
# This confirms that the queue it tries to access exists
<Queue.Queue instance at 0x40662b48>
# This is the Processing thread, which has stopped executing on exception
<Thread(192.168.1.102, stopped 1106023568)>
Exception in thread pyAlloc-0: # <-- This is the Allocation thread
Traceback (most recent call last):
File "/opt/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/volume1/home/Max/Python/MyThread/pyAllocThread.py", line 60, in run
foundThread.addTask(str(data))
File "/volume1/home/Max/Python/MyThread/pyProcThread.py", line 58, in addTask
print self.loc.todo
AttributeError: 'thread._local' object has no attribute 'todo'
处理线程的一部分
# Imports
import threading
import time
import Queue
class Thread(threading.Thread):
def __init__(self,pThread):
threading.Thread.__init__(self)
self.loc = threading.local()
self.loc.todo = Queue.Queue()
self.loc.father = pThread
print "Konstruktor ausgefuehrt"
print self.loc.todo
def run(self):
self.loc.run = True;
print self.loc.father
while (self.loc.run):
try:
task = self.loc.todo.get(True, 1)
print "processing..."
if(task == self):
self.loc.run=False
else:
print task
except Queue.Empty:
pass
self.loc.father.threadTerminated(self)
print self.name, "terminating..."
def addTask(self, pTask):
print self
print self.loc.todo
self.loc.todo.put(pTask)
和分配线程:
import threading
import pyProcThread # My processing Thread
import Queue
import time
class Thread(threading.Thread):
# Lock-Objects
threadListLock = threading.Lock()
waitListLock = threading.Lock()
alive = True;
taskQueue = Queue.Queue()
# Lists
# List of all running threads
threads = []
def threadExists(self,pIP):
"""Checks if there is already a thread with the given Name"""
for x in self.threads:
if x.name == pIP:
return x
return None
def threadTerminated(self,pThread):
"""Called when a Processing Thread terminates"""
with self.threadListLock:
self.threads.remove(pThread)
print "Thread removed"
def threadRegistered(self,pThread):
"""Registers a new Thread"""
self.threads.append(pThread)
def killThread(self):
self.alive = False
def run(self):
while(self.alive):
# print "Verarbeite Nachricht ", self.Message
# print "Von ", self.IP
try:
data, addtemp = self.taskQueue.get(True, 1)
addr, _junk = addtemp
with self.threadListLock:
foundThread=self.threadExists(str(addr))
# print "Thread " + self.name + " verarbeitet " + data
if (foundThread!=None):
#print "recycling thread"
foundThread.addTask(str(data))
else:
print "running new Thread"
t = pyProcThread.Thread(self)
t.name = str(addr)
t.addTask(str(data))
t.start()
self.threadRegistered(t)
self.taskQueue.task_done()
except Queue.Empty:
pass
print self.name, "terminating..."
with self.threadListLock:
for thread in self.threads:
thread.addTask(thread)
完整输出,包括所有调试打印:
running new Thread
Konstruktor ausgefuehrt
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, initial)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
Exception in thread 192.168.1.102:
Traceback (most recent call last):
File "/opt/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/volume1/home/Max/Python/MyThread/pyProcThread.py", line 37, in run
print self.loc.father
AttributeError: 'thread._local' object has no attribute 'father'
<Thread(192.168.1.102, started 1106023568)>
<Queue.Queue instance at 0x40662b48>
<Thread(192.168.1.102, stopped 1106023568)>
Exception in thread pyAlloc-0:
Traceback (most recent call last):
File "/opt/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/volume1/home/Max/Python/MyThread/pyAllocThread.py", line 60, in run
foundThread.addTask(str(data))
File "/volume1/home/Max/Python/MyThread/pyProcThread.py", line 58, in addTask
print self.loc.todo
AttributeError: 'thread._local' object has no attribute 'todo'
Terminating main
192.168.1.102
DEINEMUDDA sent to 192.168.1.102 : 8082
pyAlloc-1 terminating...
<Thread(192.168.1.102, stopped 1106023568)>
<Queue.Queue instance at 0x40662b48>
正如您在日志中看到的那样,一切似乎都正常运行了一段时间,尽管它从未到达print "processing
处理线程的 main 函数中。
我已经在网络和 StackOverflow 上搜索过类似的问题,但找不到任何问题。在此先感谢您的帮助,并原谅我的编码风格,我现在只编程 Python 几天,并且仍在学习它的一些功能。
编辑:当然,这个服务器还有比这更多的东西。有一个主线程正在接收数据包并将它们发送到分配线程,还有一堆其他的东西在后台。此外,服务器还远未完成,但我想在开始处理其他事情之前让接收工作正常进行。