我很难将我在 sql (postgres) 中创建的查询翻译成 sqlalchemy。特别是,我在 sqlalchemy 中尝试的映射导致了荒谬的递归结果,其运行速度比我最初编写的要慢得多。
给定以下类型的表结构:
metadata
------------------------------
primary_id - integer
secondary_count - integer
property - string (many to each primary_id)
data
-----------------------------
primary_id - integer
secondary_id - integer (many to each primary_id)
primary_json - json bytes
secondary_json - json bytes
我正在尝试使用以下方式检索成对的主要和辅助数据:
- 我们匹配给定的属性
- 我们只返回“一些”原始数据(比如 1000)
- 我们返回“最好的”原始数据,即具有最多次要数据的原始数据。
- 每个主要条目我们只能得到“一些”(比如 10 个)辅助数据
第一个很容易通过两个表之间的连接来完成,但是第二个更复杂。我在原始 SQL 中使用的解决方案(请参阅此处了解导致我使用此解决方案的解释)是:
SELECT primary_id, primary_json, secondary_json, secondary_count
FROM
(
SELECT primary_id, secondary_count
FROM metadata
WHERE property='whatever I want'
-- Get the "best" 1000 results
ORDER BY secondary_count DESC
LIMIT 1000
) my_primary_ids
LEFT OUTER JOIN LATERAL
(
SELECT primary_json, seondary_json
FROM data
WHERE primary_id = my_primary_ids.primary_id
-- Only return 10 pieces of secondary json per primary json
LIMIT 10
) json_content ON true;
我已尽最大努力将其转换为 sqlalchemy,但是我一直遇到问题,即生成的查询在FROM
横向连接查询的子句中重写了子查询。
例如,下面的 sqlalchemy 代码(假设表对象定义与上述匹配)是部分解决方案。我想我可以添加缺少的列(正如您将在生成的 sql 中看到的那样):
from sqlalchemy import true
my_prim_ids_al = (
query(Metadata.primary_id.label('primary_id'),
Metadata.secondary_count.label('secondary_count'))
.filter_by(property='whatever I want')
.order_by(Metadata.secondary_count)
.limit(1000)
.from_self()
.subquery('my_primary_ids')
)
json_content_al = (
query(Data.primary_json.label('primary_json'),
Data.secondary_json.label('secondary_json'))
.filter_by(primary_id=my_primary_ids_al.c.primary_id)
.limit(10)
.from_self()
.subquery('json_content')
.lateral()
)
joined_query = (
my_primary_ids_al
.outerjoin(json_content_al, true())
.subquery('joined_query')
)
长形式的连接查询如下,具有上述荒谬的嵌套结构:
SELECT anon_1.primary_id, anon_1.secondary_count
FROM
(
SELECT metadata.primary_id AS primary_id,
metadata.secondary_count AS secondary_count
FROM metadata
WHERE metadata.property = 'whatever I want'
ORDER BY metadata.secondary_count DESC
LIMIT :param_1
) AS anon_1
LEFT OUTER JOIN LATERAL
(
SELECT anon_4.anon_3_secondary_json AS anon_3_secondary_json,
anon_4.anon_3_primary_json AS anon_3_primary_json,
FROM
(
SELECT anon_3.secondary_json AS anon_3_secondary_json,
anon_3.primary_json AS anon_3_primary_json,
FROM
(
SELECT data.secondary_json AS secondary_json,
data.primary_json AS primary_json,
FROM data
JOIN
(
SELECT anon_1.primary_id AS primary_id,
anon_1.secondary_count AS secondary_count
FROM
(
SELECT metadata.primary_id AS primary_id,
metadata.secondary_count AS secondary_count
FROM metadata
WHERE metadata.property = 'whatever I want'
ORDER BY metadata.secondary_count DESC
LIMIT :param_1
) AS anon_1
) AS primary_ids ON data.primary_id = primary_ides.primary_id
) AS anon_3
LIMIT :param_2) AS anon_4) AS anon_2 ON true
再一次,我意识到这是一个不完整的尝试,因为并不是所有的列都在一开始就被 SELECTed,但关键问题是sqlalchemy 在横向连接子查询中创建了大量的嵌套查询。这是我一直无法解决的核心问题,除非解决了,否则完成剩下的查询毫无意义。