python - python blaze (pandas) 无法 safley 转换用户 dtype 我想使用 python blaze从uk nga geonames 下载uk.txt

Question

我想使用 python blaze从uk nga geonames 下载uk.txt中读取文件，然后odo将其插入 Postgresql 数据库。

代码是：

import blaze as bz
from odo import odo

dataPath = 'uk.txt'
myData = bz.Data(dataPath, sep='\t')
out = odo(myData, 'postgresql://postgres:postgres@localhost:5432/blaze_test::uk_geonames')

我收到错误ValueError: cannot safely convert passed user dtype of <i8 for object dtyped data in column 0，我认为我理解为“无法转换数据类型以插入数据库”

我应该强迫dtype平等吗？我将如何解决这个问题？

该文件的示例输入是：

RC  UFI UNI LAT LONG    DMS_LAT DMS_LONG    MGRS    JOG FC  DSG PC  CC1 ADM1    POP ELEV    CC2 NT  LC  SHORT_FORM  GENERIC SORT_NAME_RO    FULL_NAME_RO    FULL_NAME_ND_RO SORT_NAME_RG    FULL_NAME_RG    FULL_NAME_ND_RG NOTE    MODIFY_DATE DISPLAY NAME_RANK   NAME_LINK   TRANSL_CD   NM_MODIFY_DATE

1   380952  475802  54.086111   -6.655556   540510  -63920  29UPV5334795644 NN29-06 H   STM     EI,UK               EI,UK   N       Clarebane       CLAREBANERIVER  Clarebane River Clarebane River CLAREBANERIVER  Clarebane River Clarebane River     2014-06-27  1,2,3   2

Answer 1

由于某种原因，标头没有被正确推断。您可以像这样传递infer_header关键字参数：

In [12]: from blaze import Data

In [13]: from odo import CSV, odo

In [14]: d = Data(CSV('uk.txt', sep='\t', has_header=True))

In [15]: d.head(5)
Out[15]:
   RC     UFI     UNI        LAT      LONG  DMS_LAT  DMS_LONG  \
0   1  380952  475802  54.086111 -6.655556   540510    -63920
1   1  380952  475801  54.086111 -6.655556   540510    -63920
2   1  380954  475805  54.104722 -6.648889   540617    -63856
3   1  380955  475806  54.098056 -6.644167   540553    -63839
4   1  380958  475810  54.040556 -6.614444   540226    -63652

              MGRS      JOG FC      ...          SORT_NAME_RG  \
0  29UPV5334795644  NN29-06  H      ...        CLAREBANERIVER
1  29UPV5334795644  NN29-06  H      ...             CLAREBANE
2  29UPV5371497729  NN29-06  H      ...           ALINA LOUGH
3  29UPV5404796997  NN29-06  H      ...          CORLISSLOUGH
4  29UPV5620690667  NN29-06  H      ...          DRUMBOYLOUGH

      FULL_NAME_RG  FULL_NAME_ND_RG NOTE MODIFY_DATE DISPLAY NAME_RANK  \
0  Clarebane River  Clarebane River  NaN  2014-06-27   1,2,3         2
1        Clarebane        Clarebane  NaN  2014-06-27   1,2,3         1
2     Alina, Lough     Alina, Lough  NaN  2014-06-27   1,2,3         1
3    Corliss Lough    Corliss Lough  NaN  2014-06-27   1,2,3         1
4    Drumboy Lough    Drumboy Lough  NaN  2014-06-27   1,2,3         1

  NAME_LINK TRANSL_CD NM_MODIFY_DATE
0       NaN       NaN     2014-06-27
1       NaN       NaN     2014-06-27
2       NaN       NaN     2014-06-27
3       NaN       NaN     2014-06-27
4       NaN       NaN     2014-06-27

[5 rows x 34 columns]

之后，只需odo将其放入所需的表中：

In [16]: t = odo(d, 'postgresql://localhost::uk')

In [17]: uk = Data(t)

In [19]: uk.head(5)
Out[19]:
   RC     UFI     UNI        LAT      LONG  DMS_LAT  DMS_LONG  \
0   1  380952  475802  54.086111 -6.655556   540510    -63920
1   1  380952  475801  54.086111 -6.655556   540510    -63920
2   1  380954  475805  54.104722 -6.648889   540617    -63856
3   1  380955  475806  54.098056 -6.644167   540553    -63839
4   1  380958  475810  54.040556 -6.614444   540226    -63652

              MGRS      JOG FC      ...          SORT_NAME_RG  \
0  29UPV5334795644  NN29-06  H      ...        CLAREBANERIVER
1  29UPV5334795644  NN29-06  H      ...             CLAREBANE
2  29UPV5371497729  NN29-06  H      ...           ALINA LOUGH
3  29UPV5404796997  NN29-06  H      ...          CORLISSLOUGH
4  29UPV5620690667  NN29-06  H      ...          DRUMBOYLOUGH

      FULL_NAME_RG  FULL_NAME_ND_RG NOTE MODIFY_DATE DISPLAY NAME_RANK  \
0  Clarebane River  Clarebane River  NaN  2014-06-27   1,2,3         2
1        Clarebane        Clarebane  NaN  2014-06-27   1,2,3         1
2     Alina, Lough     Alina, Lough  NaN  2014-06-27   1,2,3         1
3    Corliss Lough    Corliss Lough  NaN  2014-06-27   1,2,3         1
4    Drumboy Lough    Drumboy Lough  NaN  2014-06-27   1,2,3         1

  NAME_LINK TRANSL_CD NM_MODIFY_DATE
0       NaN       NaN     2014-06-27
1       NaN       NaN     2014-06-27
2       NaN       NaN     2014-06-27
3       NaN       NaN     2014-06-27
4       NaN       NaN     2014-06-27

[5 rows x 34 columns]