1

我正在对一些遗留的 SQL 算法进行逆向工程以迁移到 apache spark。

我遇到了一个跨应用程序，我理解它是特定于 TSQL 的，并且在 ANSII 或 Spark SQL 中没有直接的等价物。

净化后的算法是：

SELECT
        Id_P ,
        Monthindex     ,
        (
                SELECT
                        100 * (STDEV(ResEligible.num_valid) / AVG(ResEligible.num_valid)) AS Pre_Coef_Var
                FROM
                        tbl_p a CROSS APPLY
                        (
                                SELECT
                                        e.Monthindex ,
                                        e.num AS num_valid
                                FROM
                                        dbo.tbl_p e
                                WHERE
                                        e.Monthindex     = a.MonthIndex
                                AND     e.Id_P = a.Id_P

                                UNION ALL

                                SELECT DISTINCT
                                        B1.[MonthIndex       ] ,
                                        Tr.num AS num_valid
                                FROM
                                        #tbl_pr B1
                                INNER JOIN
                                        #tbl_pr B2
                                ON
                                        B1.[Id_P] = B2.[Id_P]
                                AND     B2.Rang - B1.Rang BETWEEN 0 AND     2
                                INNER JOIN
                                        dbo.tbl_p Tr
                                ON
                                        Tr.Id_P = B1.Id_P
                                AND     Tr.Monthindex     = B1.Monthindex
                                WHERE
                                        a.Id_P = B1.[Id_P]
                                AND     B2.[MonthIndex]  =
                                        (
                                                SELECT
                                                        MAX([MonthIndex])
                                                FROM
                                                        #tbl_pr
                                                WHERE
                                                        [MonthIndex]     < a.MonthIndex
                                                AND     [Id_P] = a.Id_P) ) AS ResEligible
                WHERE
                        a.Id_P = result.Id_P
                AND     a.MonthIndex     = result.MonthIndex) AS Coeff
FROM
        tbl_p AS result
WHERE
        1          = 1
AND     MonthIndex = @CurrentMonth
GROUP BY
        Id_P ,
        Monthindex) AS CC

所以对于别名 b 中的每一行，我们交叉应用到内部查询。

是否可以根据连接操作（或其他方式）重写交叉应用，以便我可以在 spark sql 中重新实现？

干杯

特里

4

1 回答 1

0

似乎您可以将查询重写如下：

SELECT T1.col1,
       T1.col2,
       sq.col3Sum
FROM tbl1 T1
     CROSS JOIN (SELECT SUM(T1sq.Col3) AS col3Sum
                 FROM tbl1 T1sq
                     JOIN tbl2 T2 ON T1sq.Col1 = T2.Col2
                     JOIN tbl3 T3 ON T2.col1 = T3.Col1) sq;

然而，这似乎很奇怪，JOIN在 2 次引用之间没有标准tbl1。

于 2019-06-12T15:06:28.480 回答