0

如何在两个不同的函数中引用相同的临时目录。我需要访问在 move_source_to_dest 中解压缩的文件，作为 pd.read_csv 语句中函数 df_to_csv 的输入。我尝试了一些更改，但没有任何效果。请帮忙。

def move_source_to_dest(key, src_session):

    with tempfile.TemporaryDirectory() as tempdir:

        try:
            print("downloading {}/{}".format(s3_src_bucket, key))
            src_session.client('s3').download_file(Bucket=s3_src_bucket, Key=key,
                                               Filename=os.path.join(tempdir, os.path.basename(key)))

            #Command to decompress the files
            command = "bzip2 -dk " + os.path.join(tempdir, os.path.basename(key))
            subprocess.call(command,shell = True)


        except Exception as e:
            print("exception handling {}/{}".format(s3_src_bucket, key))
            raise e
def df_to_csv(key, src_session):
    with tempfile.TemporaryDirectory() as tempdir:
        try:
            #Reading all the columns names from the file "ambs_ambi_ColumnsNames.txt"
            with open('./shakenbake_ds/ambs_ambi_ColumnsNames.txt') as f:
                clist= f.read().splitlines()

                #file = open('ambs_ambi_ColumnsNames.txt','r')
                #clist=file.readlines()

            Filename=os.path.join(tempdir, os.path.basename(key[:-4]))
            Fileout=os.path.join(tempdir, os.path.basename(key[:-4])) + "-out.csv" 

            with open('./shakenbake_ds/ambs_ambi_OutColumnsNames.txt') as o:
                outcols= o.read().splitlines()
                #file = open('ambs_ambi_OutColumnsNames.txt','r')
                #outcols=file.readlines()
                #global Filename
            c=0
            for chunk in pd.read_csv(Filename, sep="\x01", names=clist ,iterator=True, chunksize=300000):
4

2 回答 2

2

将临时目录作为参数传递给两个函数：

with tempfile.TemporaryDirectory() as tempdir:
    move_source_to_dest(key, src_session, tempdir)
    df_to_csv(key, src_session, tempdir)
于 2018-01-19T16:59:04.040 回答
0

在函数之间共享状态的一种常见方法是将状态作为参数传递给这些函数。

例如，

with tempfile.TemporaryDirectory() as tempdir:
    move_source_to_dest(tempdir, ...)
    df_to_csv(tempdir, ...)

并move_source_to_dest重新df_to_csv定义如下：

def move_source_to_dest(tempdir, key, src_session):
    # no additional tempdir created here
    # ...

def df_to_csv(key, src_session):
    # no additional tempdir created here
    # ...
于 2018-01-19T16:58:11.460 回答