1

我有一个 df，其 DateTimeIndex 在很长一段时间（> 1 年）内的间隔为 30 分钟，因此 >17520 行。由于与夏令时相关的原因，索引中的两个索引值重复，并且缺少两个值。所以重复的值是：

In[1]: df[df.index.duplicated('first')] 
Out[2]: 
                            a           b        c 
timestamp                                                                   
2012-10-07 01:00:00           NaN        NaN      NaN     
2012-10-07 01:30:00           NaN        NaN      NaN     
2013-10-06 01:00:00           NaN        NaN      NaN      
2013-10-06 01:30:00           NaN        NaN      NaN

我想在 1 小时后将这些更改为缺失值：

In[3]: df[df.index.duplicated('first')].shift(1,freq="H")
Out[4]: 
                           a            b        c
timestamp                                                                   
2012-10-07 02:00:00           NaN        NaN      NaN     
2012-10-07 02:30:00           NaN        NaN      NaN     
2013-10-06 02:00:00           NaN        NaN      NaN        
2013-10-06 02:30:00           NaN        NaN      NaN

但这不会改变索引：

df[df.index.duplicated('first')] = df[df.index.duplicated('first')].shift(1,freq="H")

什么会？

4

1 回答 1

0

我认为您需要duplicated index使用rename以下地图dict：

print (df)
                     a   b   c
timestamp                     
2013-10-06 01:00:00  1 NaN NaN
2013-10-06 01:30:00  2 NaN NaN
2013-10-06 01:00:00  3 NaN NaN
2013-10-06 01:30:00  4 NaN NaN
2012-10-08 01:30:00  5 NaN NaN
2013-10-10 01:00:00  6 NaN NaN


df1 = df[df.index.duplicated('first')]
d = dict(zip(df1.index, df1.shift(1,freq="H").index))
print (d)
{Timestamp('2013-10-06 01:00:00'): Timestamp('2013-10-06 02:00:00'), 
 Timestamp('2013-10-06 01:30:00'): Timestamp('2013-10-06 02:30:00')}

df = df.rename(index=d)
print (df)
                     a   b   c
timestamp                     
2013-10-06 02:00:00  1 NaN NaN
2013-10-06 02:30:00  2 NaN NaN
2013-10-06 02:00:00  3 NaN NaN
2013-10-06 02:30:00  4 NaN NaN
2012-10-08 01:30:00  5 NaN NaN
2013-10-10 01:00:00  6 NaN NaN

类似的解决方案：

idx = df.index[df.index.duplicated('first')]
d = dict(zip(idx, idx.to_series().shift(freq="H").index))
print (d)
{Timestamp('2013-10-06 01:00:00'): Timestamp('2013-10-06 02:00:00'), 
 Timestamp('2013-10-06 01:30:00'): Timestamp('2013-10-06 02:30:00')}

df = df.rename(index=d)
print (df)
                     a   b   c
timestamp                     
2013-10-06 02:00:00  1 NaN NaN
2013-10-06 02:30:00  2 NaN NaN
2013-10-06 02:00:00  3 NaN NaN
2013-10-06 02:30:00  4 NaN NaN
2012-10-08 01:30:00  5 NaN NaN
2013-10-10 01:00:00  6 NaN NaN
2013-10-06 02:30:00   8 NaN NaN
2012-10-08 01:30:00   9 NaN NaN
2013-10-10 01:00:00  10 NaN NaN

idx = df.index[df.index.duplicated('first')]
s = idx.to_series().shift(freq="H")
#swap index with values in Series
d = pd.Series(s.index.values, index = s.values).to_dict()
print (d)
{Timestamp('2013-10-06 01:00:00'): Timestamp('2013-10-06 02:00:00'), 
 Timestamp('2013-10-06 01:30:00'): Timestamp('2013-10-06 02:30:00')}

df = df.rename(index=d)
print (df)
                     a   b   c
timestamp                     
2013-10-06 02:00:00  1 NaN NaN
2013-10-06 02:30:00  2 NaN NaN
2013-10-06 02:00:00  3 NaN NaN
2013-10-06 02:30:00  4 NaN NaN
2012-10-08 01:30:00  5 NaN NaN
2013-10-10 01:00:00  6 NaN NaN

编辑1：

您需要将withtimedeltas创建的添加到原始索引。cumcountto_timedelta

delta = pd.to_timedelta(df.groupby(level=0).cumcount(), unit='H')
print (delta)
timestamp
2013-10-06 01:00:00   00:00:00
2013-10-06 01:30:00   00:00:00
2013-10-06 01:00:00   01:00:00
2013-10-06 01:30:00   01:00:00
2012-10-08 01:30:00   00:00:00
2013-10-10 01:00:00   00:00:00
dtype: timedelta64[ns]

df.index = df.index + delta
print (df)
                     a   b   c
2013-10-06 01:00:00  1 NaN NaN
2013-10-06 01:30:00  2 NaN NaN
2013-10-06 02:00:00  3 NaN NaN
2013-10-06 02:30:00  4 NaN NaN
2012-10-08 01:30:00  5 NaN NaN
2013-10-10 01:00:00  6 NaN NaN
于 2017-04-07T06:12:37.380 回答