我正在使用 pandas (1.3.4) 和 s3fs (2021.10.1) 从 S3 存储桶中读取 parquet 文件,如下所示:
pd.read_parquet(s3_path)
此读取是作为 API 后台进程的一部分定期完成的,因此我想处理由任何间歇性 S3 服务不可用引起的异常。
问题在于 s3fs 将 S3 ClientErrors 转换为本地文件系统错误。例如 NoSuchKey 变成 FileNotFoundError (参见https://github.com/dask/s3fs/blob/main/s3fs/errors.py)。
我想获得原始的 S3 异常,因为转换后的异常可能是非特定的或具有误导性的。
有趣的是,s3fs 源代码似乎表明这是可以通过__cause__
属性实现的:
def translate_boto_error(error, message=None, set_cause=True, *args, **kwargs):
"""Convert a ClientError exception into a Python one.
Parameters
----------
error : botocore.exceptions.ClientError
The exception returned by the boto API.
message : str
An error message to use for the returned exception. If not given, the
error message returned by the server is used instead.
set_cause : bool
Whether to set the __cause__ attribute to the previous exception if the
exception is translated.
*args, **kwargs :
Additional arguments to pass to the exception constructor, after the
error message. Useful for passing the filename arguments to ``IOError``.
Returns
-------
An instantiated exception ready to be thrown. If the error code isn't
recognized, an IOError with the original error message is returned.
"""
if not hasattr(error, "response"):
# non-http error:
return error
code = error.response["Error"].get("Code")
constructor = ERROR_CODE_TO_EXCEPTION.get(code)
if constructor:
if not message:
message = error.response["Error"].get("Message", str(error))
custom_exc = constructor(message, *args, **kwargs)
else:
# No match found, wrap this in an IOError with the appropriate message.
custom_exc = IOError(errno.EIO, message or str(error), *args)
if set_cause:
custom_exc.__cause__ = error
return custom_exc
但是,通过传递无效路径自己尝试时,__cause__
是None
:
try:
result = pd.read_parquet(path)
except Exception as e:
print(e.__cause__) # printing None
我错过了什么吗?