6

我有一个系统可以跟踪用户查看的文档。每个文档都有它的 ID 和它所属的集群。我的系统会跟踪会话 ID 和查看次数。我现在想构建一个 SQL 查询,它会给我两列——会话 ID 和分类集群。分类算法很简单:

1. select all sessions
2. for each session S
   I. prepare an accumulator ACC for clusters
   II. select the clusters of viewed documents for this session
   III. for each cluster C accumulate the cluster count ( ACC[C]++ )
   IV. find the maximum in the ACC. That is the cluster that the session was classified to

表结构如下,我使用的是 MySQL 5.5.16:

会议

+-------+-----------+--------------------+
| ID    | sessionID | classified_cluster |
+-------+-----------+--------------------+

会话文档

+-------+-----------+------------+
| ID    | sessionID | documentID |
+-------+-----------+------------+

+-------+-------+
| ID    | label |
+-------+-------+

集群文档

+-------+-----------+------------+
| ID    | clusterID | documentID |
+-------+-----------+------------+

所以基本上,我想为每个会话选择集群,计算查看文档的每个集群的出现次数并找到最大出现次数。然后出现最多的集群的 ID 是会话的结果,因此最终结果集包含会话 ID 和出现次数最多的集群:

结果

+-----------+-----------------------+
| sessionID | classifiedIntoCluster |
+-----------+-----------------------+

我设法使用以下查询获取每个会话的已查看文档集群(步骤 2/II。):

SELECT SD.session_id, CD.cluster_id 
FROM cluster_document AS CD 
INNER JOIN session_document AS SD 
ON CD.document_id = SD.document_id
WHERE session_id IN (SELECT session_id FROM session) 

我很难弄清楚其余的。这甚至可以使用嵌套的 SELECT 查询吗?我应该使用光标,如果是的话,有人可以用光标显示一个例子吗?任何帮助都感激不尽。

编辑 #1:添加了 C# 实现、MySQL 转储和预期结果

C# 实现

    private void ClassifyUsers() {
        int nClusters = Database.SelectClusterCount(); //get number of clusters
        DataSet sessions = Database.SelectSessions(); //get all sessions
        foreach (DataRow session in sessions.Tables[0].Rows) { //foreach session
           int[] acc = new int[nClusters]; //prepare an accumulator for each known cluster
           string s_id = session["session_id"].ToString();
           DataSet sessionClusters = Database.SelectSessionClusters(s_id); //get clusters for this session

           foreach (DataRow cluster in sessionClusters.Tables[0].Rows) { //for each cluster
               int c = Convert.ToInt32(cluster["cluster_id"].ToString()) - 1;
               acc[c]++; //accumulate the cluster count
           }

           //find the maximum in the accumulator -> that is the most relevant cluster
           int max = 0;
           for (int j = 0; j < acc.Length; j++) {
               if (acc[j] >= acc[max]) max = j;
           }
           max++;
           Database.UpdateSessionCluster(s_id, max); //update the session with its new assigned cluster
       }
    }

表结构、测试数据和预期结果

表结构和测试数据

预期结果

编辑#2:添加了一个较小的数据集和进一步的算法演练

这是一个较小的数据集:

会议

session id    |  cluster
abc                 0
def                 0
ghi                 0
jkl                 0       
mno                 0

cluster_id  | label
1               A
2               B
3               C
4               D
5               E

SESSION_DOCUMENT

id      | session_id    |   document_id
1           abc             1
2           def             5
3           jkl             3
4           ghi             4
5           mno             2
6           def             2
7           abc             5
8           ghi             3

CLUSTER_DOCUMENT

id      | cluster_id    |   document_id
1           1                  2
2           1                  3
3           2                  5
4           3                  5
5           3                  1
6           4                  3
7           5                  2
8           5                  4

算法详解

第 1 步:获取会话查看的文档的集群

session_id  |  cluster_id   | label     | document_id   
abc             3               C           1
abc             2               B           5
abc             3               C           5
-----
def             2               B           5
def             3               C           5   
def             1               A           2
def             5               E           2   
----
ghi             5               E           4   
ghi             1               A           3   
ghi             4               D           3   
----
jkl             1               A           3   
jkl             4               D           3   
----
mno             1               A           2
mno             5               E           2

第 2 步:计算集群的出现次数

session_id |    cluster_id  | label |   occurrence
abc             3               C           2   <--- MAX
abc             2               B           1
----
def             2               B           1
def             3               C           1   
def             1               A           1
def             5               E           1   <--- MAX
----
ghi             5               E           1   
ghi             1               A           1   
ghi             4               D           1   <--- MAX
----
jkl             1               A           1   
jkl             4               D           1   <--- MAX
----
mno             1               A           1   
mno             5               E           1   <--- MAX

第 3 步(最终结果):找到每个会话的最大发生集群(见上文)并构建最终结果集(session_id,cluster_id):

session_id |    cluster_id  
abc                 3           
def                 5
ghi                 4
jkl                 4
mno                 5

编辑#3:接受的答案说明

两个给出的答案都是正确的。它们都为问题提供了解决方案。我给了 Mosty Mostacho 接受的答案,因为他首先提供了解决方案,并提供了另一个版本的解决方案,带有VIEW. mankuTimma 的解决方案与 Mosty Mostacho 的解决方案质量相同。因此,我们有两个同样好的解决方案,我选择了莫斯蒂·莫斯塔乔,因为他是第一个。

感谢他们俩的贡献。.

4

2 回答 2

2

如果我正确理解您的问题,那么对于每个会话,您都希望集群具有最多的文档视图。所以,你去下面的查询返回每个会话 id 的特定集群 id 的最大计数或出现次数。

SELECT SESSION_ID,MAX(CNT) MAX_CNT
FROM (SELECT SD.SESSION_ID, CD.CLUSTER_ID,COUNT(*) AS CNT
FROM CLUSTER_DOCUMENT AS CD 
INNER JOIN SESSION_DOCUMENT AS SD 
ON CD.DOCUMENT_ID = SD.DOCUMENT_ID
GROUP BY SD.SESSION_ID,CD.CLUSTER_ID) CNT1
GROUP BY SESSION_ID

然后将上述结果与子查询(我正在计算计数)再次连接以获得最大出现的集群 id。如果有两个出现次数相同的集群 ID,我将使用具有最大值的集群 ID。我已经对您的数据进行了测试,并且可以正常工作。此外,现在此查询应该适用于所有数据库。

SELECT B.SESSION_ID, MAX(CNT2.CLUSTER_ID) FROM 
(SELECT SESSION_ID,MAX(CNT) MAX_CNT
FROM (SELECT SD.SESSION_ID, CD.CLUSTER_ID,COUNT(*) AS CNT
FROM CLUSTER_DOCUMENT AS CD 
INNER JOIN SESSION_DOCUMENT AS SD 
ON CD.DOCUMENT_ID = SD.DOCUMENT_ID
GROUP BY SD.SESSION_ID,CD.CLUSTER_ID) CNT1
GROUP BY SESSION_ID) B
JOIN (SELECT SD.SESSION_ID, CD.CLUSTER_ID,COUNT(*) AS CNT
FROM CLUSTER_DOCUMENT AS CD 
INNER JOIN SESSION_DOCUMENT AS SD 
ON CD.DOCUMENT_ID = SD.DOCUMENT_ID
GROUP BY SD.SESSION_ID,CD.CLUSTER_ID) CNT2
ON B.SESSION_ID = CNT2.SESSION_ID
AND B.MAX_CNT = CNT2.CNT
GROUP BY B.SESSION_ID
于 2012-02-16T10:31:00.013 回答
2

好吧,当有很多相等时,我对如何选择一个事件有一些疑问,但是看看 C# 代码,似乎这种选择是不确定的。

现在,给定样本数据,步骤 2 的实际结果是:

+------------+------------+-------+------------+
| SESSION_ID | CLUSTER_ID | LABEL | OCCURRENCE |
+------------+------------+-------+------------+
| abc        |          3 | C     |          2 |
| def        |          1 | A     |          1 |
| def        |          2 | B     |          1 |
| def        |          3 | C     |          1 |
| def        |          5 | E     |          1 |
| ghi        |          1 | A     |          1 |
| ghi        |          4 | D     |          1 |
| ghi        |          5 | E     |          1 |
| jkl        |          1 | A     |          1 |
| jkl        |          4 | D     |          1 |
| mno        |          1 | A     |          1 |
| mno        |          5 | E     |          1 |
+------------+------------+-------+------------+

因此,继续处理这些数据,我得到了该会话 id 的 session_id 和 max(cluster_id),结果是:

+------------+------------+
| SESSION_ID | CLUSTER_ID |
+------------+------------+
| abc        |          3 |
| def        |          5 |
| ghi        |          5 |
| jkl        |          4 |
| mno        |          5 |
+------------+------------+

max(cluster_id) 只是用来执行非确定性选择。这是查询:

select s1.session_id, max(s1.cluster_id) as cluster_id from (
  select sd.session_id, cd.cluster_id, count(*) as Occurrence
  from session_document sd
  join cluster_document cd
  on sd.document_id = cd.document_id
  join cluster c
  on c.cluster_id = cd.cluster_id
  group by sd.session_id, cd.cluster_id, c.label
) as s1
left join (
  select sd.session_id, count(*) as Occurrence
  from session_document sd
  join cluster_document cd
  on sd.document_id = cd.document_id
  join cluster c
  on c.cluster_id = cd.cluster_id
  group by sd.session_id, cd.cluster_id, c.label
) as s2
on s1.session_id = s2.session_id and s1.occurrence < s2.occurrence
where s2.occurrence is null
group by s1.session_id

也许添加视图会提高性能(替换上述查询):

create view MaxOccurrences as (
  select sd.session_id, cd.cluster_id, count(*) as Occurrence
  from session_document sd
  join cluster_document cd
  on sd.document_id = cd.document_id
  join cluster c
  on c.cluster_id = cd.cluster_id
  group by sd.session_id, cd.cluster_id, c.label
);

select s1.session_id, max(s1.cluster_id) as cluster_id
from MaxOccurrences as s1
left join MaxOccurrences as s2
on s1.session_id = s2.session_id and s1.occurrence < s2.occurrence
where s2.occurrence is null
group by s1.session_id

让我知道它是否有效。

于 2012-02-16T18:46:32.753 回答