我正在填充两个表,它们具有一对多的关系。
因此,我在 中插入一行outer
,获取该行的(自动增量主键)id
,然后将 100 行插入inner
(全部带有指向 的外键outer.id
)。
然后我重复 50 次。对于中的每个条目,outer
我必须插入、读取id
,然后插入到内部。
这很慢。大部分时间都花在将 100 行加载到inner
. 我怀疑如果我可以在一个批处理操作中插入所有 50*100 行会快得多。inner
但是我看不到该怎么做-如何使外键起作用?
其他人如何提高效率?
我正在使用 Java / Spring。100 行以JdbcTemplate.batchUpdate()
.
public final void insert(final JdbcTemplate db,
final Iterable<DataBlock> data) {
String insertSql = getInsertSql();
String idQuery = getIdQuery();
ItemRowMapper.IdRowMapper mapper = new ItemRowMapper.IdRowMapper();
for (DataBlock block: data) {
Object[] outer = block.getOuter();
LOG.trace("Loading outer");
db.update(insertSql, outer);
LOG.trace("Getting index");
// currently retrieve index based on natural key, but could use last index
int id = db.query(idQuery, mapper, uniqueData(outer)).get(0);
LOG.trace("Getting inner");
List<Object[]> inner = block.getInner(id);
// most time spent here
LOG.trace(format("Loading inner (%d)", inner.size()));
innerTable.insert(db, inner);
}
}
和伪SQL:
create table outer (
integer id primary key autoincrement,
...
);
create table inner (
integer outer references outer(id),
...
);
更新- 以下似乎适用于 Spring 3.1.1 和 Postgres 9.2-1003.jdbc4。
/**
* An alternative implementation that should be faster, since it inserts
* in just two batches (one for inner and one fo router).
*
* @param db A connection to the database.
* @param data The data to insert.
*/
public final void insertBatchier(final JdbcTemplate db,
final AllDataBlocks data) {
final List<Object[]> outers = data.getOuter();
List<Integer> ids = db.execute(
new PreparedStatementCreator() {
@Override
public PreparedStatement createPreparedStatement(
final Connection con) throws SQLException {
return con.prepareStatement(getInsertSql(),
Statement.RETURN_GENERATED_KEYS);
}
},
new PreparedStatementCallback<List<Integer>>() {
@Override
public List<Integer> doInPreparedStatement(final PreparedStatement ps)
throws SQLException {
for (Object[] outer: outers) {
for (int i = 0; i < outer.length; ++i) {
setParameterValue(ps, i + 1,
SqlTypeValue.TYPE_UNKNOWN, outer[i]);
}
ps.addBatch();
}
ps.executeBatch();
RowMapperResultSetExtractor<Integer> ids =
new RowMapperResultSetExtractor<Integer>(
new ItemRowMapper.IdRowMapper());
try (ResultSet keys = ps.getGeneratedKeys()) {
return ids.extractData(keys);
}
}
});
innerTable.insert(db, data.getInner(ids));
}