我正在使用 PyHive 0.6.1 在 Hive 中运行一个较长的插入查询,thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
运行大约 5 分钟后它失败了。在服务器端,查询一直运行直到成功完成。快速查询没有这个问题。
我无法使用相同的 python 版本在我的 Mac 上本地重现它:代码正确地等待直到查询完成。发生这种情况的环境是一个基于 python:3.6-slim 的 Docker 容器。除其他外,我正在安装 libsasl2-dev 和 libsasl2-modules 包,以及 pyhive[hive] python 包。
任何线索为什么会这样?提前致谢。
我正在使用的代码是:
import contextlib
from pyhive.hive import connect
def get_conn():
return connect(
host='my-host',
port=10000,
auth='NONE',
username='username',
database='database'
)
with contextlib.closing(get_conn()) as conn, \
contextlib.closing(conn.cursor()) as cur:
cur.execute('My long insert statement')
这是完整的追溯
Traceback (most recent call last):
File "<stdin>", line 5, in <module>
File "/usr/local/lib/python3.6/site-packages/pyhive/hive.py", line 364, in execute
response = self._connection.client.ExecuteStatement(req)
File "/usr/local/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 280, in ExecuteStatement
return self.recv_ExecuteStatement()
File "/usr/local/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 292, in recv_ExecuteStatement
(fname, mtype, rseqid) = iprot.readMessageBegin()
File "/usr/local/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 134, in readMessageBegin
sz = self.readI32()
File "/usr/local/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 217, in readI32
buff = self.trans.readAll(4)
File "/usr/local/lib/python3.6/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 166, in read
self._read_frame()
File "/usr/local/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 170, in _read_frame
header = self._trans.readAll(4)
File "/usr/local/lib/python3.6/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python3.6/site-packages/thrift/transport/TSocket.py", line 132, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 5, in <module>
File "/usr/local/lib/python3.6/contextlib.py", line 185, in __exit__
self.thing.close()
File "/usr/local/lib/python3.6/site-packages/pyhive/hive.py", line 221, in close
response = self._client.CloseSession(req)
File "/usr/local/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 218, in CloseSession
return self.recv_CloseSession()
File "/usr/local/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 230, in recv_CloseSession
(fname, mtype, rseqid) = iprot.readMessageBegin()
File "/usr/local/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 134, in readMessageBegin
sz = self.readI32()
File "/usr/local/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 217, in readI32
buff = self.trans.readAll(4)
File "/usr/local/lib/python3.6/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 166, in read
self._read_frame()
File "/usr/local/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 170, in _read_frame
header = self._trans.readAll(4)
File "/usr/local/lib/python3.6/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/usr/local/lib/python3.6/site-packages/thrift/transport/TSocket.py", line 132, in read
message='TSocket read 0 bytes')
thrift.transport.TTransport.TTransportException: TSocket read 0 bytes