我正在使用 pandas-0.8rc2 读取输入 CSV,其中两列本地化日期时间字符串缺少 UTC 偏移信息,并且需要将数据帧系列正确转换为 UTC。
我一直在尝试解决方法来减轻时间戳列都不代表索引的事实,它们是数据。tz_localize 和 tz_convert 显然仅适用于系列/数据帧的索引,而不是列。我非常想学习一种更好的方法来做到这一点,而不是下面的代码:
# test.py
import pandas
# input.csv:
# starting,ending,measure
# 2012-06-21 00:00,2012-06-23 07:00,77
# 2012-06-23 07:00,2012-06-23 16:30,65
# 2012-06-23 16:30,2012-06-25 08:00,77
# 2012-06-25 08:00,2012-06-26 12:00,0
# 2012-06-26 12:00,2012-06-27 08:00,77
df = pandas.read_csv('input.csv', parse_dates=[0,1])
print df
ser_starting = df.starting
ser_starting.index = ser_starting.values
ser_starting = ser_starting.tz_localize('US/Eastern')
ser_starting = ser_starting.tz_convert('UTC')
ser_ending = df.ending
ser_ending.index = ser_ending.values
ser_ending = ser_ending.tz_localize('US/Eastern')
ser_ending = ser_ending.tz_convert('UTC')
df.starting = ser_starting.index
print df
df.ending = ser_ending.index
print df
其次,代码遇到了一些奇怪的行为。它将第二个赋值的时间戳数据改回数据帧,无论顺序是 df.starting 还是 df.ending:
$ python test.py
starting ending measure
0 2012-06-21 00:00:00 2012-06-23 07:00:00 77
1 2012-06-23 07:00:00 2012-06-23 16:30:00 65
2 2012-06-23 16:30:00 2012-06-25 08:00:00 77
3 2012-06-25 08:00:00 2012-06-26 12:00:00 0
4 2012-06-26 12:00:00 2012-06-27 08:00:00 77
starting ending measure
0 2012-06-21 04:00:00 2012-06-23 07:00:00 77
1 2012-06-23 11:00:00 2012-06-23 16:30:00 65
2 2012-06-23 20:30:00 2012-06-25 08:00:00 77
3 2012-06-25 12:00:00 2012-06-26 12:00:00 0
4 2012-06-26 16:00:00 2012-06-27 08:00:00 77
Traceback (most recent call last):
File "test.py", line 28, in <module>
print df
File "/path/to/lib/python2.7/site-packages/pandas/core/frame.py", line 572, in __repr__
if self._need_info_repr_():
File "/path/to/lib/python2.7/site-packages/pandas/core/frame.py", line 560, in _need_info_repr_
self.to_string(buf=buf)
File "/path/to/lib/python2.7/site-packages/pandas/core/frame.py", line 1207, in to_string
formatter.to_string(force_unicode=force_unicode)
File "/path/to/lib/python2.7/site-packages/pandas/core/format.py", line 200, in to_string
fmt_values = self._format_col(i)
File "/path/to/lib/python2.7/site-packages/pandas/core/format.py", line 242, in _format_col
space=self.col_space)
File "/path/to/lib/python2.7/site-packages/pandas/core/format.py", line 462, in format_array
return fmt_obj.get_result()
File "/path/to/lib/python2.7/site-packages/pandas/core/format.py", line 589, in get_result
fmt_values = [formatter(x) for x in self.values]
File "/path/to/lib/python2.7/site-packages/pandas/core/format.py", line 597, in _format_datetime64
base = stamp.strftime('%Y-%m-%d %H:%M:%S')
ValueError: year=1768 is before 1900; the datetime strftime() methods require year >= 1900
打印语句只是为了演示问题。如果我避免使用 repr 和其他调用 strftime 的方法,错误的值将毫无例外地继续存在。
奇怪的是,如果我继续在 repl 调用 df.{starting,ending} 分配,我通常会得到一个正确的数据帧,并带有时间戳:
In [151]: df
Out[151]:
starting ending measure
0 2012-06-21 04:00:00 2012-06-23 11:00:00 77
1 2012-06-23 11:00:00 2012-06-23 20:30:00 65
2 2012-06-23 20:30:00 2012-06-25 12:00:00 77
3 2012-06-25 12:00:00 2012-06-26 16:00:00 0
4 2012-06-26 16:00:00 2012-06-27 12:00:00 77
这是不可重复的,AFAICT,我无法描述通过上述 ValueError 的调用的确切顺序,但确实如此
如果我遇到错误,或者这是不受支持的 API 使用,我将不胜感激。
如上所述,我宁愿学习更好地使用 pandas API 来避免这样做。