1

我制作了以下有效的代码，但我想改进它。我不想重新读取该文件，但如果我删除 sales_input.seek(0) 它不会迭代将每一行抛出销售。我该如何改进呢？

def computeCritics(mode, cleaned_sales_input = "data/cleaned_sales.csv"):
    if mode == 1:
        print "creating customer.critics.recommendations"
        critics_output = open("data/customer/customer.critics.recommendations", 
                              "wb")
        ID = getCustomerSet(cleaned_sales_input)
        sales_dict = pickle.load(open("data/customer/books.dict.recommendations", 
                                      "r"))
    else: 
        print "creating books.critics.recommendations"
        critics_output = open("data/books/books.critics.recommendations", 
                              "wb")
        ID = getBookSet(cleaned_sales_input)
        sales_dict = pickle.load(open("data/books/users.dict.recommendations", 
                                      "r"))
    critics = {}
    # make critics dict and pickle it 
    for i in ID:
        with open(cleaned_sales_input, 'rb') as sales_input:
            sales = csv.reader(sales_input)  # read new 
            for j in sales:
                if mode == 1:
                    if int(i) == int(j[2]):
                        sales_dict[int(j[6])] = 1
                else: 
                    if int(i) == int(j[6]):
                        sales_dict[int(j[2])] = 1
            critics[int(i)] = sales_dict
    pickle.dump(critics, critics_output)
    print "done"

clean_sales_input 看起来像

6042772,2723,3546414,9782072488887,1,9.99,314968
6042769,2723,3546414,9782072488887,1,9.99,314968
...

其中数字 6 是图书 ID，数字 0 是客户 ID

我想得到一个看起来像的字典 

critics = {
    CustomerID1: {
        BookID1: 1,
        BookID2: 0,
        ........
        BookIDX: 0
    },
    CustomerID2: {
        BookID1: 0,
        BookID2: 1,
        ...
    }
}

或者

critics = {
    BookID1: {
        CustomerID1: 1,
        CustomerID2: 0,
        ........
        CustomerIDX: 0
    },
    BookID1: {
        CustomerID1: 0,
        CustomerID2: 1,
        ...
        CustomerIDX: 0
    }
}

我希望这不是太多信息

4

2 回答 2

2

以下是一些建议：

我们先来看看这个代码模式：

for i in ID:
    for j in sales:
        if int(i) == int(j[2])

请注意，i仅与 比较j[2]。这是它在循环中的唯一目的。int(i) == int(j[2])每个 最多只能为 True 一次i。

因此，我们可以for i in ID通过将循环重写为

for j in sales:
    key = j[2]
    if key in ID:

基于函数名称getCustomerSetand getBookSet，听起来好像 ID是一个集合（而不是列表或元组）。我们希望 ID 是一个集合，因为测试集合中的成员资格是 O(1)（而不是 O(n) 的列表或元组）。

接下来，考虑这一行：

critics[int(i)] = sales_dict

这里有一个潜在的陷阱。此行分配sales_dict给 critics[int(i)]每个iin ID。每个键int(i)都被映射到相同的dict. 当我们遍历salesandID时，我们正在sales_dict像这样修改，例如：

sales_dict[int(j[6])] = 1

但这会导致in 中的所有值critics同时被修改，因为 in 中的所有键都critics指向同一个 dict, sales_dict。我怀疑那是你想要的。

为了避免这个陷阱，我们需要复制 sales_dict：

critics = {i:sales_dict.copy() for i in ID}

def computeCritics(mode, cleaned_sales_input="data/cleaned_sales.csv"):
    if mode == 1:
        filename = 'customer.critics.recommendations'
        path = os.path.join("data/customer", filename)
        ID = getCustomerSet(cleaned_sales_input)
        sales_dict = pickle.load(
            open("data/customer/books.dict.recommendations", "r"))
        key_idx, other_idx = 2, 6
    else:
        filename = 'books.critics.recommendations'
        path = os.path.join("data/books", filename)        
        ID = getBookSet(cleaned_sales_input)
        sales_dict = pickle.load(
            open("data/books/users.dict.recommendations", "r"))
        key_idx, other_idx = 6, 2

    print "creating {}".format(filename)
    ID = {int(item) for item in ID}
    critics = {i:sales_dict.copy() for i in ID}
    with open(path, "wb") as critics_output:
        # make critics dict and pickle it
        with open(cleaned_sales_input, 'rb') as sales_input:
            sales = csv.reader(sales_input)  # read new
            for j in sales:
                key = int(j[key_idx])
                if key in ID:
                    other_key = int(j[other_idx])
                    critics[key][other_key] = 1                    
                critics[key] = sales_dict
        pickle.dump(dict(critics), critics_output)
        print "done"
于 2013-09-05T12:14:05.447 回答
0

@unutbu 的答案更好，但如果你被这个结构卡住了，你可以把整个文件放在内存中：

sales = []
with open(cleaned_sales_input, 'rb') as sales_input:
     sales_reader = csv.reader(sales_input)       
     [sales.append(line) for line in sales_reader]

     for i in ID:
        for j in sales:
            #do stuff
于 2013-09-05T12:18:56.743 回答