[使用 Python3] 我正在使用 pandas 读取 csv 文件,对数据帧进行分组,将函数应用于分组数据并将这些结果添加回原始数据帧。
我的输入如下所示:
email cc timebucket total_value
john@john.com us 1 110.50
example@example.com uk 3 208.84
... ... ... ...
基本上我正在尝试分组cc
并计算该组中每个值的百分位数total_value
。其次,我想对这些结果应用一个流程语句。我需要将这些结果添加回原始/父 DataFrame。这样它看起来像这样:
email cc timebucket total_value percentrank rankbucket
john@john.com us 1 110.50 48.59 mid50
example@example.com uk 3 208.84 99.24 top25
... ... ... ... ... ...
下面的代码给了我一个AssertionError
,我不知道为什么。我对 Python 和 pandas 很陌生,所以这可能会解释一个又一个。
代码:
import pandas as pd
import numpy as np
from scipy.stats import rankdata
def percentilerank(frame, groupkey='cc', rankkey='total_value'):
from pandas.compat.scipy import percentileofscore
# Technically the below percentileofscore function should do the trick but I cannot
# get that to work, hence the alternative below. It would be great if the answer would
# include both so that I can understand why one works and the other doesnt.
# func = lambda x, score: percentileofscore(x[rankkey], score, kind='mean')
func = lambda x: (rankdata(x.total_value)-1)/(len(x.total_value)-1)*100
frame['percentrank'] = frame.groupby(groupkey).transform(func)
def calc_and_write(filename):
"""
Function reads the file (must be tab-separated) and stores in a pandas DataFrame.
Next, the percentile rank score based is calculated based on total_value and is done so within a country.
Secondly, based on the percentile rank score (prs) a row is assigned to one of three buckets:
rankbucket = 'top25' if prs > 75
rankbucket = 'mid50' if 25 > prs < 75
rankbucket = 'bottom25' if prs < 25
"""
# Define headers for pandas to read in DataFrame, stored in a list
headers = [
'email', # 0
'cc', # 1
'last_trans_date', # 3
'timebucket', # 4
'total_value', # 5
]
# Reading csv file in chunks and creating an iterator (is supposed to be much faster than reading at once)
tp = pd.read_csv(filename, delimiter='\t', names=headers, iterator=True, chunksize=50000)
# Concatenating the chunks and sorting total DataFrame by booker_cc and total_nett_spend
df = pd.concat(tp, ignore_index=True).sort(['cc', 'total_value'], ascending=False)
percentilerank(df)
编辑:根据要求,这是回溯日志:
Traceback (most recent call last):
File "C:\Users\m\Documents\Python\filter_n_split_3.py", line 85, in <module>
print(calc_and_write('tsv/test.tsv'))
File "C:\Users\m\Documents\Python\filter_n_split_3.py", line 74, in calc_and_write
percentilerank(df)
File "C:\Users\m\Documents\Python\filter_n_split_3.py", line 33, in percentilerank
frame['percentrank'] = frame.groupby(groupkey).transform(func)
File "C:\Python33\lib\site-packages\pandas\core\groupby.py", line 1844, in transform
axis=self.axis, verify_integrity=False)
File "C:\Python33\lib\site-packages\pandas\tools\merge.py", line 894, in concat
verify_integrity=verify_integrity)
File "C:\Python33\lib\site-packages\pandas\tools\merge.py", line 964, in __init__
self.new_axes = self._get_new_axes()
File "C:\Python33\lib\site-packages\pandas\tools\merge.py", line 1124, in _get_new_axes
assert(len(self.join_axes) == ndim - 1)
AssertionError