使用这个问题的解决方案:How to read a list of parquet files from S3 as a pandas dataframe using pyarrow? ,我正在导入以下内容,以便将数据从 s3 存储桶复制到雪花表:
from snowflake.connector.pandas_tools import write_pandas
当我尝试使用“write_pandas”方法时,出现错误:
TypeError: __init__() missing 1 required positional argument: 'backend'
这是我的代码:
conn = snow.connect(
user="ll",
password="zzz",
account="account",
role="role",
warehouse="WAREHOUSE",
database="db",
schema="C")
s3 = s3fs.S3FileSystem(anon=False, key = 'AKI', secret = 'zzz')
pandas_dataframe = pq.ParquetDataset('s3://bucket/folder2/File.snappy.parquet',
filesystem=s3).read_pandas().to_pandas()
write_pandas(conn, pandas_dataframe, "DATATABLE")
我需要做什么来添加这个必需的参数?还是有不同的解决方案?