5

我正在使用MySQL 5.6。我创建了一个包含 366 个分区的表来按天保存数据，这意味着一年中我们最多有 366 天，所以我在该表上创建了 366 个分区。哈希分区由一个整数列管理，该列为每条记录存储 1 到 366。

Report_Summary表：

CREATE TABLE `Report_Summary` (
  `PartitionsID` int(4) unsigned NOT NULL,
  `ReportTime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `Amount` int(10) NOT NULL,
  UNIQUE KEY `UNIQUE` (`PartitionsID`,`ReportTime`),
  KEY `PartitionsID` (`PartitionsID`),
  KEY `ReportTime` (`ReportTime`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED
/*!50100 PARTITION BY HASH (PartitionsID)
PARTITIONS 366 */

我目前的查询：

SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
FROM Report_Summary RS
WHERE RS.ReportTime >= '2014-12-26 00:00:00' AND RS.ReportTime <= '2014-12-30 23:59:59' AND 
      RS.PartitionsID BETWEEN DAYOFYEAR('2014-12-26 00:00:00') AND DAYOFYEAR('2014-12-30 23:59:59')
GROUP BY ReportDate;

上面的查询运行良好，并且使用分区p360到p364来获取数据。现在的问题是当我将fromDate 传递给 '2014-12-26' 并将 toDate 传递给 '2015-01-01'然后上面的查询将不起作用。因为“2015-01-01”的年份是1，所以我的条件失败了。

现在我尝试在IN运算符中传递值，然后它在查询下面的数据库检查中完美运行：

SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
FROM Report_Summary RS
WHERE RS.ReportTime >= '2014-12-26 00:00:00' AND RS.ReportTime <= '2015-01-01 23:59:59' AND 
      RS.PartitionsID IN (360,361,362,363,364,365,1)
GROUP BY ReportDate;

为了生成上述场景，我创建了一个函数并传递了两个日期并生成了一个逗号分隔的 ID 字符串

SELECT GenerateRange('2014-12-26 00:00:00', '2015-01-01 23:59:59');

这将我的数据返回为：

'360,361,362,363,364,365,366,1'

我尝试在我的查询中使用该函数，所以我将查询更改如下：

SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
FROM Report_Summary RS
WHERE RS.ReportTime >= '2014-12-26 00:00:00' AND RS.ReportTime <= '2015-01-01 23:59:59' AND 
      FIND_IN_SET(RS.PartitionsID, GenerateRange('2014-12-26 00:00:00', '2015-01-01 00:00:00'))
GROUP BY ReportDate;

然后我使用EXPLAIN PARTITION SELECT...检查了上述查询的执行计划。我发现我的条件行不通。它使用所有分区来获取数据。我只想使用这些日期的特定分区。必须只检查这360,361,362,363,364,365,366,1分区意味着p360到p366和p1。

为什么我的查询不起作用？这不是实现这个的正确方法然后我想要解决方案我该如何实现呢？

我从编码中知道我可以实现它，但我必须编写一个查询来实现它。

谢谢...

4

4 回答 4

1

我能想到几个选项。

  1. 创建case涵盖多年搜索条件的语句。
  2. 创建一个表并使用它来为您的子句CalendarDays获取不同的列表。DayOfYearin
  3. 选项 1 的变体，但使用 aunion分别搜索每个范围

选项 1：使用case语句。它不漂亮，但似乎有效。如果查询跨越非闰年，则此选项可能会搜索一个额外的分区 366。此外，我不确定优化器是否会喜欢过滤器OR中的RS.ParitionsID，但您可以尝试一下。

SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
FROM Report_Summary RS
WHERE RS.ReportTime >= @startDate AND RS.ReportTime <= @endDate
    AND 
    (
    RS.PartitionsID BETWEEN 
        CASE 
            WHEN
                --more than one year, search all days 
                year(@endDate) - year(@startDate) > 1
                --one full year difference 
                OR year(@endDate) - year(@startDate) = 1 
                    AND DAYOFYEAR(@startDate) <= DAYOFYEAR(@endDate)
            THEN 1
            ELSE DAYOFYEAR(@startDate)
        END
        and 
        CASE
            WHEN 
                --query spans the end of a year
                year(@endDate) - year(@startDate) >= 1
            THEN 366
            ELSE DAYOFYEAR(@endDate)
        END
    --Additional query to search less than portion of next year
    OR RS.PartitionsID <=
        CASE
            WHEN year(@endDate) - year(@startDate) > 1
                OR DAYOFYEAR(@startDate) > DAYOFYEAR(@endDate)
            THEN DAYOFYEAR(@endDate)
            ELSE NULL
        END
    )
GROUP BY ReportDate;

选项 2：使用CalendarDays表格。这个选项更干净。CalendarDays缺点是如果您没有新表，则需要创建一个新表。

SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
FROM Report_Summary RS
WHERE RS.ReportTime >= @startDate AND RS.ReportTime <= @endDate
    AND RS.PartitionsID IN
    (
        SELECT DISTINCT DAYOFYEAR(c.calDate) 
        FROM dbo.calendarDays c
        WHERE c.calDate >= @startDate and c.calDate <= @endDate
    )

编辑：选项 3：选项 1 的变体，但Union All用于分别搜索每个范围。这里的想法是，由于OR语句中没有 ，优化器将能够应用分区修剪。注意：我通常不在MySQL.

DECLARE @startDate datetime, @endDate datetime;
DECLARE @rangeOneStart datetime, @rangeOneEnd datetime, @rangeTwoStart datetime, @rangeTwoEnd datetime;

SELECT @rangeOneStart := 
        CASE 
            WHEN
                --more than one year, search all days 
                year(@endDate) - year(@startDate) > 1
                --one full year difference 
                OR year(@endDate) - year(@startDate) = 1 
                    AND DAYOFYEAR(@startDate) <= DAYOFYEAR(@endDate)
            THEN 1
            ELSE DAYOFYEAR(@startDate)
        END
    , @rangeOneEnd := 
        CASE
            WHEN 
                --query spans the end of a year
                year(@endDate) - year(@startDate) >= 1
            THEN 366
            ELSE DAYOFYEAR(@endDate)
        END 
    , @rangeTwoStart := 1
    , @rangeTwoEnd := 
        CASE
            WHEN year(@endDate) - year(@startDate) > 1
                OR DAYOFYEAR(@startDate) > DAYOFYEAR(@endDate)
            THEN DAYOFYEAR(@endDate)
            ELSE NULL
        END
;

SELECT t.ReportDate, sum(t.Amount) as Total
FROM 
(
    SELECT DATE(RS.ReportTime) AS ReportDate, RS.Amount
    FROM Report_Summary RS
    WHERE RS.PartitionsID BETWEEN @rangeOneStart AND @rangeOneEnd
        AND RS.ReportTime >= @startDate AND RS.ReportTime <= @endDate

    UNION ALL

    SELECT DATE(RS.ReportTime) AS ReportDate, RS.Amount
    FROM Report_Summary RS
    WHERE RS.PartitionsID BETWEEN @rangeTwoStart AND @rangeTwoEnd
        AND @rangeTwoEnd IS NOT NULL
        AND RS.ReportTime >= @startDate AND RS.ReportTime <= @endDate
) t
GROUP BY ReportDate;
于 2015-01-05T23:31:15.880 回答
0

我得到了解决方案，我改变了在我的表中存储PartitionsId列的逻辑。最初我将DayOfYear(reportTime)列存储在PartitionsId列中。现在我通过存储TO_DAYS(reportTime)并存储到PartitionsId列中更改了该逻辑。

所以我的表结构如下：

CREATE TABLE `Report_Summary` (
  `PartitionsID` int(10) unsigned NOT NULL,
  `ReportTime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `Amount` int(10) NOT NULL,
  UNIQUE KEY `UNIQUE` (`PartitionsID`,`ReportTime`),
  KEY `PartitionsID` (`PartitionsID`),
  KEY `ReportTime` (`ReportTime`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=COMPRESSED
/*!50100 PARTITION BY HASH (PartitionsID)
PARTITIONS 366 */

INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735928','2014-12-26 11:46:12','100');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735929','2014-12-27 11:46:23','50');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735930','2014-12-28 11:46:37','44');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735931','2014-12-29 11:46:49','15');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735932','2014-12-30 11:46:59','56');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735933','2014-12-31 11:47:22','68');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735934','2015-01-01 11:47:35','76');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735935','2015-01-02 11:47:43','88');
INSERT INTO `Report_Summary` (`PartitionsID`, `ReportTime`, `Amount`) VALUES('735936','2015-01-03 11:47:59','77');

检查SQL FIDDLE 演示：

我的查询是：

EXPLAIN PARTITIONS 
SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
FROM Report_Summary RS
WHERE RS.ReportTime >= '2014-12-26 00:00:00' AND RS.ReportTime <= '2015-01-01 23:59:59' AND 
      RS.PartitionsID BETWEEN TO_DAYS('2014-12-26 00:00:00') AND TO_DAYS('2015-01-01 23:59:59')
GROUP BY ReportDate;

上面的查询扫描了我需要的特定分区，它还使用了正确的索引。因此，在更改PartitionsId列的逻辑后，我找到了正确的解决方案。

感谢所有的回复，非常感谢大家的时间......

于 2015-01-11T05:20:54.857 回答
0

要开始解决这个问题，您需要一个子查询，给定一个日期范围，返回一个包含DAYOFYEAR()该范围内所有值的结果集。

让我们解决这个问题。对于初学者，我们需要一个可以返回从 0 到至少 366 的所有整数的序列的查询。这是那个查询。它返回一列seq值 0-624。

SELECT A.N + 5*(B.N + 5*(C.N + 5*(D.N))) AS seq
  FROM (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2 
                      UNION SELECT 3 UNION SELECT 4) AS A
  JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                      UNION SELECT 3 UNION SELECT 4) AS B
  JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                      UNION SELECT 3 UNION SELECT 4) AS C
  JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                      UNION SELECT 3 UNION SELECT 4) AS D

（这是生成 5**4 数字的所有组合的简单交叉连接技巧。）

接下来，我们需要使用它来生成 DAYOFYEAR() 值的列表。为了示例，让我们使用您的开始日期和结束日期。此查询生成一个结果集，其中包含一堆整数，显示该日期范围内一年中的哪几天。 

SELECT DISTINCT DAYOFYEAR(first_day + INTERVAL seq DAY) doy
  FROM (SELECT DATE('2014-12-26 00:00:00') AS first_day,
               DATE('2015-01-01 23:59:59') AS last_day
       ) params
  JOIN (
         SELECT A.N + 5*(B.N + 5*(C.N + 5*(D.N))) AS seq
           FROM (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2 
                               UNION SELECT 3 UNION SELECT 4) AS A
           JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                               UNION SELECT 3 UNION SELECT 4) AS B
           JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                               UNION SELECT 3 UNION SELECT 4) AS C
           JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                               UNION SELECT 3 UNION SELECT 4) AS D
       ) seq ON seq.seq <= TIMESTAMPDIFF(DAY,first_day,last_day)
 ORDER BY 1

我认为您可以说服自己，这个粗糙的小查询在大约一年半（625 天）或更短的任何合理天数范围内都能正常工作。如果你使用更长的时间跨度，你可能会搞砸闰年。

最后，您可以在PartitionsID IN ()子句中使用此查询。看起来像这样。

SELECT DATE(RS.ReportTime) AS ReportDate, SUM(RS.Amount) AS Total
  FROM Report_Summary RS
 WHERE RS.ReportTime >= '2014-12-26 00:00:00'
   AND RS.ReportTime <= '2015-01-01 23:59:59'
   AND RS.PartitionsID 
     IN (
         SELECT DISTINCT DAYOFYEAR(first_day + INTERVAL seq DAY) doy
           FROM (SELECT DATE('2014-12-26 00:00:00') AS first_day,
                        DATE('2015-01-01 23:59:59') AS last_day
                ) params
           JOIN (
                  SELECT A.N + 5*(B.N + 5*(C.N + 5*(D.N))) AS seq
                    FROM (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2 
                                        UNION SELECT 3 UNION SELECT 4) AS A
                    JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                                        UNION SELECT 3 UNION SELECT 4) AS B
                    JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                                        UNION SELECT 3 UNION SELECT 4) AS C
                    JOIN (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2
                                        UNION SELECT 3 UNION SELECT 4) AS D
                ) seq ON seq.seq <= TIMESTAMPDIFF(DAY,first_day,last_day)
          ORDER BY 1
         ) 
GROUP BY ReportDate;

那应该为你做。

如果您使用的是 MariaDB 10+，则有内置的序列表，名为seq_0_to_624.

这里有一篇关于这个主题的文章：

http://www.plumislandmedia.net/mysql/filling-missing-data-sequences-cardinal-integers/

于 2015-01-08T15:45:31.973 回答
0

根据您的选择，您真正需要的是一种称为“汇总表”的数据仓库技术。这样，您每天（或每小时或其他任何时间）汇总数据并将小计存储在一个小得多的表中。然后“报告”查看该表并汇总小计。这通常比原始数据的强力扫描快 10 倍。更多细节：http: //mysql.rjweb.org/doc.php/datawarehouse。

这样做消除了在原始数据（“事实表”）或汇总表中进行分区的需要。

但是，如果您需要清除旧数据，那么 PARTITIONing 可以派上用场，因为 DROP PARTITION。为此，您将使用 BY RANGE(TO_DAYS(...))，而不是 BY HASH。

于 2015-03-05T05:12:27.260 回答