3

我正在向数据库中插入一些数据，并且大多数查询都正确插入，但我不断收到至少一个随机查询错误。

我正在使用 Python 3、MySQL 5.6.17 和 MySQL python 连接器 2.1.3（在遇到与 2.0.2 相同的问题后升级）。

查询在多处理池 map_async() 中运行。

multiprocessing.pool.RemoteTraceback: bytearray index out of range
Traceback (most recent call last):
  File "./../../../my-python-script.py", line 930, in insert_into_database
    mysql_query(mysql_script, values) # <-- My mysql wrapper function
  File "./../../../my-python-script.py", line 297, in mysql_query
    for row in results:
  File "./../../../mysql/connector/cursor.py", line 450, in _execute_iter
    result = next(query_iter)
  File "./../../../mysql/connector/connection.py", line 520, in cmd_query_iter
    yield self._handle_result(self._send_cmd(ServerCmd.QUERY, statements))
  File "./../../../mysql/connector/connection.py", line 405, in _handle_result
    self._socket.recv(), self.python_charset)
  File "./../../../mysql/connector/protocol.py", line 238, in parse_column
    (packet, _) = utils.read_lc_string(packet[4:])  # catalog
  File "./../../../mysql/connector/utils.py", line 199, in read_lc_string
    if buf[0] == 251:  # \xfb
IndexError: bytearray index out of range

The above exception was the direct cause of the following exception:

IndexError: bytearray index out of range

或者有时我得到（从“结果中的行”行）

  File "./../../../mysql/connector/cursor.py", line 450, in _execute_iter
    result = next(query_iter)
  File "./../../../mysql/connector/connection.py", line 520, in cmd_query_iter
    yield self._handle_result(self._send_cmd(ServerCmd.QUERY, statements))
  File "./../../../mysql/connector/connection.py", line 384, in _handle_result
    elif packet[4] == 0:
IndexError: bytearray index out of range

The above exception was the direct cause of the following exception:

IndexError: bytearray index out of range

我的设置类似于

class InsertData:

    def __init__(self):
        with(multiprocessing.Pool(2) as Pool:
            Pool.map_async(self.insert_into_database(),set(1,2,3.....))
            Pool.close()
            Pool.join()

    def insert_into_database(self,values):
        # use the values to do some calculations then insert to database
        mysql_query(mysql_script, values)

def mysql_query(script, values):
    cursor.execute(query, values, multi = True)

和 sql 脚本

'INSERT INTO table1 ( column1 ) VALUES ( "x"  ); '
'SET @table1 = LAST_INSERT_ID(); '
'INSERT INTO table2 ( column1, column2 ) VALUES ( "y", @table1 ); '
'SET @table2 = LAST_INSERT_ID(); '
...

我目前正在查看 connector.py 和 utils 代码，试图弄清楚发生了什么。但这对我来说太先进了。

https://github.com/mysql/mysql-connector-python/blob/master/lib/mysql/connector/connection.py#L368

https://github.com/mysql/mysql-connector-python/blob/master/lib/mysql/connector/utils.py#L167

在绝望的尝试中，我尝试将缓冲设置为 True https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursorbuffered.html

我需要阅读字节数组，但我怀疑我的查询脚本导致了问题，因为当我一次运行一个连接器时我没有（我认为？）这个问题cursor.execute(query, values, multi = False)

4

1 回答 1

2

当我按照从 Python 多处理访问 MySQL 连接池中所述发送数据库连接时，问题就消失了。

就像是

mysql_conn = None

def db_conn():
  global mysql_conn
  mysql_conn = connector.connect(...)

class InsertData:

    def __init__(self):
        with(multiprocessing.Pool(2, initializer = db_conn) as Pool:
            Pool.map_async(self.insert_into_database(),set(1,2,3.....))
            Pool.close()
            Pool.join()

    def insert_into_database(self,values):
        # use the values to do some calculations then insert to database
        self.mysql_query(mysql_script, values)

    def mysql_query(script, values):
        cursor = mysql_conn.cursor()
        cursor.execute(query, values, multi = True)
于 2015-10-29T05:16:18.370 回答