我正在尝试在 mysql 中编写几个数据帧。我使用mysql.connector进行连接,使用sqlalchemy创建引擎。
大多数数据帧都正确写入数据库。不幸的是,应用程序停止并出现以下错误:
sqlalchemy.exc.DatabaseError: (mysql.connector.errors.DatabaseError) 2006 (HY000): MySQL server has gone away
...
During handling of the above exception, another exception occurred:
...
_mysql_connector.MySQLInterfaceError: MySQL server has gone away
由于断开连接的数据帧也是最大的数据帧(pickle 文件:198MB),我认为这是由于MySql-Server 设置 max_allowed_packet 或 timeout造成的。(如此处所述)
所以我通过'connect_timeout'扩展了我的SQLConnector(见下文)中的配置文件:900。不幸的是没有结果。我还读到您应该调整 MySql 服务器的 my.cnf。不幸的是,我不知道在哪里可以找到它们。经过长时间的搜索,我找到了以下顺序。C:\Program Files\MySQL\MySQL Server 8.0\etc 和文件 mysqlrouter.conf.sample。 这里我下载了一个 my.cnf 文件并将其放入文件夹中。不幸的是没有结果,因为我不知道如何设置 MySql 以便服务器使用这个文件。
有谁知道我该如何解决这个错误?或者我如何配置 MySql Server 首选项设置 max_allowed_packet 或超时?
主要的:
mysqlConnector = MySQLConnector()
dbConnection = mysqlConnector.init_engine()
for df_tuple in df_all_tuple:
df_name = df_tuple[0]
df = pd.DataFrame(df_tuple[1])
df.to_sql(con=mydb, name=df_name, if_exists='fail', chunksize=20000)
SQL连接器:
import mysql.connector
from mysql.connector import errorcode
import sqlalchemy as db
config = {
'user': 'root',
'password': '',
'host': '127.0.0.1',
'database': 'test',
'raise_on_warnings': True,
'use_pure': False,
'autocommit': True,
'connect_timeout': 900
}
class MySQLConnector:
def __init__(self):
connectTest = self.connect()
print("Connection to database: " + str(connectTest))
def init_engine(self):
try:
connect_string = 'mysql+mysqlconnector://{}:{}@{}/{}?charset=utf8mb4'.format(config.get("user"),
config.get("password"),
config.get("host"),
config.get("database"),
pool_pre_ping=True)
print("Engine: " + connect_string)
sqlengine = db.create_engine(connect_string)
except:
print("SQLConnector: Sqlalchemy error. Can't create engine....")
else:
dbConnection = sqlengine.connect()
return dbConnection
def connect(self):
try:
con = mysql.connector.connect(**config)
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
print("Something is wrong with your user name or password")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
print("Database does not exist")
if self.createDB() is True:
return self.connect()
elif err.errno == errorcode.CR_SERVER_GONE_ERROR:
print("The client couldn't send a question to the server.")
print("err: " + err)
elif err.errno == errorcode.CR_SERVER_LOST:
print(
"The client didn't get an error when writing to the server, but it didn't get a full answer (or any answer) to the question.")
print("err: " + err)
else:
print(err)
return None
else:
if con.is_connected():
db_Info = con.get_server_info()
print("Connected to MySQL Server version ", db_Info)
sqlcursor = con.cursor()
return con
def createDB(self):
try:
mydb = mysql.connector.connect(
host=config.get("host"),
user=config.get("user"),
passwd=config.get("password")
)
except mysql.connector.Error as err:
print(err)
else:
sqlcursor = mydb.cursor()
try:
sqlcursor.execute('CREATE DATABASE {}'.format(config.get("database")))
except mysql.connector.Error as err:
print(err)
return False
else:
print("Database created!")
return True