我正在尝试在 spark jdbc 中执行按日期划分的查询 sql。我已经看到了许多带有单个表的示例,但是如何在带有过滤器的子选择的查询中做到这一点?
查询示例:
select
col1, col2
from table
inner join
(
select col1, col2 from table2 where {partitionColumn} > ? and {partitionColumn} < ?
) as table2ToBeFiltered
代码java示例:
this.sparkSession.read()
.format("jdbc")
// what could I put here?
.option("partitionColumn", "name of col in subselect")
.option("lowerBound", "2021-01-01")
.option("upperBound", "2021-02-04")
.option("numPartitions", 4)
.option("oracle.jdbc.mapDateToTimestamp", "false")
.option("sessionInitStatement", "ALTER SESSION SET NLS_DATE_FORMAT = 'YYYY-MM-DD'")
// what could I put here?
.option("dbtable", "how to use select with subselect and joins and partition by specific col")
.load();
有没有办法做到这一点?