python - 是在 PyTables 中添加列以创建新表并复制的唯一方法吗？

Question

我正在寻找一种可以处理存储在磁盘上的异构数据的持久数据存储解决方案。PyTables 似乎是一个显而易见的选择，但我能找到的关于如何附加新列的唯一信息是一个教程示例。本教程让用户创建一个添加列的新表，将旧表复制到新表中，最后删除旧表。这似乎是一个巨大的痛苦。这是必须要做的吗？

如果是这样，将混合数据存储在磁盘上可以相对轻松地容纳新列的更好选择是什么？我也看过 sqlite3 并且列选项在那里似乎也相当有限。

Answer 1

是的，您必须创建一个新表并复制原始数据。这是因为表格是一种密集格式。这给它带来了巨大的性能优势，但成本之一是添加新列有点昂贵。

Answer 2

感谢Anthony Scopatz的回答。

我搜索网站并在 github 中，我发现有人展示了如何在 PyTables 中添加列。显示如何在 PyTables 中添加列的示例

原始版本，示例显示如何在 PyTables 中添加列，但迁移有一些困难。

修改后的版本，隔离了复制逻辑，部分术语被弃用，新增列有一些小错误。

根据他们的贡献，我更新了在 PyTables 中添加新列的代码。（Python 3.6，窗口）

# -*- coding: utf-8 -*-
"""
PyTables, append a column
    """
import tables as tb
pth='d:/download/'

# Describe a water class
class Water(tb.IsDescription):
    waterbody_name   = tb.StringCol(16, pos=1)   # 16-character String
    lati             = tb.Int32Col(pos=2)        # integer
    longi            = tb.Int32Col(pos=3)        # integer
    airpressure      = tb.Float32Col(pos=4)      # float  (single-precision)
    temperature      = tb.Float64Col(pos=5)      # double (double-precision)

# Open a file in "w"rite mode
# if don't include pth, then it will be in the same path as the code.
fileh = tb.open_file(pth+"myadd-column.h5", mode = "w")

# Create a table in the root directory and append data...
tableroot = fileh.create_table(fileh.root, 'root_table', Water,
                               "A table at root", tb.Filters(1))
tableroot.append([("Mediterranean", 10, 0, 10*10, 10**2),
              ("Mediterranean", 11, -1, 11*11, 11**2),
              ("Adriatic", 12, -2, 12*12, 12**2)])
print ("\nContents of the table in root:\n",
       fileh.root.root_table[:])

# Create a new table in newgroup group and append several rows
group = fileh.create_group(fileh.root, "newgroup")
table = fileh.create_table(group, 'orginal_table', Water, "A table", tb.Filters(1))
table.append([("Atlantic", 10, 0, 10*10, 10**2),
              ("Pacific", 11, -1, 11*11, 11**2),
              ("Atlantic", 12, -2, 12*12, 12**2)])
print ("\nContents of the original table in newgroup:\n",
       fileh.root.newgroup.orginal_table[:])
# close the file
fileh.close()

#%% Open it again in append mode
fileh = tb.open_file(pth+"myadd-column.h5", "a")
group = fileh.root.newgroup
table = group.orginal_table

# Isolated the copying logic
def append_column(table, group, name, column):
    """Returns a copy of `table` with an empty `column` appended named `name`."""
    description = table.description._v_colObjects.copy()
    description[name] = column
    copy = tb.Table(group, table.name+"_copy", description)

    # Copy the user attributes
    table.attrs._f_copy(copy)

    # Fill the rows of new table with default values
    for i in range(table.nrows):
        copy.row.append()
    # Flush the rows to disk
    copy.flush()

    # Copy the columns of source table to destination
    for col in descr:
        getattr(copy.cols, col)[:] = getattr(table.cols, col)[:]

    # choose wether remove the original table
#    table.remove()

    return copy

# Get a description of table in dictionary format
descr = table.description._v_colObjects
descr2 = descr.copy()

# Add a column to description
descr2["hot"] = tb.BoolCol(dflt=False)

# append orginal and added data to table2 
table2 = append_column(table, group, "hot", tb.BoolCol(dflt=False))
# Fill the new column
table2.cols.hot[:] = [row["temperature"] > 11**2 for row in table ]
# Move table2 to table, you can use the same name as original one.
table2.move('/newgroup','new_table')

# Print the new table
print ("\nContents of the table with column added:\n",
       fileh.root.newgroup.new_table[:])
# Finally, close the file
fileh.close()