0

我知道在这里抛出代码并寻求帮助以解决问题是不好的。这个问题似乎有点超出我的想象。

该代码应该遍历所有文件和子文件夹。我认为这里没有任何日志错误。问题是我遇到了再次处理相同文件并导致数据库插入在主键约束上失败的问题。

这是我的代码：

import csv
import pypyodbc
import os
import sys

extension = ".tsv"
connStr = """DSN=database_test;"""

sys.stdout = open('c:\\temp\\python.log', 'w')
print 'starting ...'

def LoadFile(path):   
    i = 0 
    for item in os.listdir(path): # loop through items in dir        
        full_path = os.path.join(path, item)
        if os.path.isfile(full_path) and full_path.endswith(extension): # check for ".tsv" extension

            if full_path.find('IM') > 0:
                table_name = 'table_a'
            else:
                table_name = 'table_b'

            if os.stat(full_path).st_size > 0:
                print "Processing file:", i, "|", full_path
                i = i + 1
                with open (full_path, 'r') as f:
                    reader = csv.reader(f, delimiter='\t')
                    columns = next(reader) 
                    query = 'insert into ' + table_name + '({0}) values ({1})'
                    crsr = cnxn.cursor()
                    for data in reader:
                        query = query.format(', '.join(columns), ', '.join('?' * len(columns)))          
                        #print(query, "with ", data)
                        if(data[1] != ''):
                            crsr.execute(query, data)
                    crsr.commit()  
                    crsr.close()                                
        elif os.path.isdir(full_path):                        
            print "Process Folder: ", full_path    
            LoadFile(full_path)                        
        else:
            print("invalid file name:", item)
    print "Process Folder total files: ", i, ":", full_path
    return         

cnxn = pypyodbc.connect(connStr)

dir_name = 'X:\\TopLevelFolder'
LoadFile(dir_name)

cnxn.close()
print("Completed")
4

1 回答 1

0

我最初的问题有点误导......由于一些无法解释的硬件问题，代码无法正常工作。在我切换到 python 2.7 并重新安装所有库之后。代码工作正常。

于 2015-10-14T21:54:27.373 回答