6

一些背景可以帮助首先解释这个难题:

我有一个数据库,通过将用户提交的值与全球平均值进行比较来估计用户的可靠性。值介于 0 和 1 之间。因此,其中:

  • 此特定用户的可靠性 =r
  • 此特定用户提交值的平均值 =a
  • 全球,“商定”平均值 =g

可靠性:

r = 1 - ABS(g - a)

这就是计算每个用户的可靠性的方式。现在,“商定”的全局平均值g是使用加权平均值计算的,其中权重为r,值为a。如果总共有 3 个用户:

  g = ((r1 * a1) + (r2 * a2) + (r3 * a3)) / (r1 + r2 + r3)

问题是,一旦用户有了高可靠性,他们就完全垄断了,任何新的价值观都无法改变这一点。举个例子:

g was initially 0.5
user1 r was initially 0.5
user2 r was initially 0.5
user3 r was initially 0.5

现在,他们将一一提交值,并观察会发生什么:

user1 a is submitted, 1.0
user1 reliability goes slightly down because it differs from g (0.5)
user2 a is submitted, 1.0
user1 and user2 reliability go up to 100%, g is now 1.0.
user3 a is submitted, 0.0
user3 reliability goes down to 0%. g is still 1.0.

由于 user3 的可靠性非常低,因此加权对 没有任何影响g。User3 的可靠性下降,因为提交的值与全局平均值完全不同。怎么做才能让 user3 的提交对最终值产生一些影响?也许我需要添加一些常数,以便可靠性永远不会完全为零(但接近)?

现在,对于 SQL 代码。我添加了一个演示问题的 SQL 小提琴: http ://sqlfiddle.com/#!3/d3fd1/21 我已经抽象了代码以使其保持简短,但它仍然很长。

表创建、存储过程和触发器:

-- Stores user info
CREATE TABLE dbo.Users(
    [UserID] [int] NOT NULL,
    [Reliability] [float] NOT NULL
  )

-- Contains global averages from all users who submitted data
CREATE TABLE dbo.GlobalSubmission(
    GlobalSubmissionID  [int] NOT NULL,
    Name [varchar](50) NULL,
    GlobalAverage [float] NOT NULL,
)

CREATE TABLE dbo.UserSubmission(
    SubValue float NOT NULL,
    GlobalSubmissionID int NOT NULL,
    UserID int NOT NULL,
)


GO

--Calculate the "ideal value", used for GlobalSubmission.
CREATE FUNCTION dbo.IdealValueCalc(@globalSubmissionID INT)
RETURNS int
AS
BEGIN

DECLARE @tmpReliability TABLE (SubValue float, Reliability float)


INSERT INTO @tmpReliability
    SELECT AVG(us.SubValue) as SubValue, usr.Reliability Reliability FROM UserSubmission us
    JOIN Users usr 
    ON us.UserID = usr.UserID
    WHERE GlobalSubmissionID = @GlobalSubmissionID
    GROUP BY us.UserID, usr.Reliability

--Perform weighted mean calculations.
Return (SELECT SUM(SubValue * Reliability) / SUM(Reliability) FROM @tmpReliability)
END
go


--Calculate the reliability of one user.
CREATE FUNCTION dbo.GetReliabilityForUser
(@userID int)
Returns Float
AS BEGIN
Return (SELECT 1 - AVG(ABS(db.userAvg - db.GlobalAverage))
    FROM (
      SELECT pmd.UserID,
            gs.GlobalAverage, 
            AVG(pmd.SubValue) as userAvg
      FROM UserSubmission pmd
      -- Joins average value for each user with "ideal" value from GlobalSubmission
      JOIN GlobalSubmission gs 
        ON gs.GlobalSubmissionID = pmd.GlobalSubmissionID
        WHERE pmd.UserID = 1
      GROUP BY pmd.UserID, gs.GlobalSubmissionID, gs.GlobalAverage
     ) db
     GROUP BY db.UserID)
End
go



CREATE TRIGGER trg_SubmissionComputation
ON UserSubmission 
AFTER INSERT, UPDATE
AS BEGIN
    --Calculate this uer's reliability
    DECLARE @userID int = (SELECT TOP(1) UserID FROM inserted)
    DECLARE @userReliability float = dbo.GetReliabilityForUser(@userID)

    UPDATE Users
    SET Reliability=@userReliability
    WHERE UserID = @userID

    --Recalculate globalSubmission values:
    DECLARE @globalSubmissionID int = (SELECT TOP(1) GlobalSubmissionID FROM inserted)
    DECLARE @globalAverage float = dbo.IdealValueCalc(@globalSubmissionID)
        --The global average for this set of submissions has been recalculated. Now inserting:

    UPDATE GlobalSubmission
    SET GlobalAverage = @globalAverage 
    WHERE GlobalSubmissionID = @globalSubmissionID
END
GO

测试它:

--Creating 3 new users
INSERT INTO Users
(UserID, Reliability)
values 
(1, 0.5),
(2, 0.5),
(3, 0.5)
GO

--Creating a new GlobalSubmission
INSERT INTO GlobalSubmission
(GlobalSubmissionID, NAME, GlobalAverage)
values (1, 'BOILER2B' , 0.5)
GO

--First, we will submit values of 1 for two users:
INSERT INTO UserSubmission values (1.0, 1, 1); -- Value: 1.0, User 1, Submission 1
GO
INSERT INTO UserSubmission values (1.0, 1, 2); -- Value: 1.0, User 2, Submission 1
GO
INSERT INTO UserSubmission values (1.0, 1, 1); -- Value: 1.0, User 1, Submission 1
GO
INSERT INTO UserSubmission values (1.0, 1, 2); -- Value: 1.0, User 2, Submission 1
GO


--Now, we will submit values of 0 for the third user:
INSERT INTO UserSubmission values (0.0, 1, 3); -- Value: 0.0, User 3, Submission 1
GO
INSERT INTO UserSubmission values (0.0, 1, 3); -- Value: 0.0, User 3, Submission 1
GO

SELECT * FROM Users -- This results in 0% reliability for the last user.

--If we create new users and add them, the reliability won't budge:
INSERT INTO Users
(UserID, Reliability)
values 
(4, 0.5),
(5, 0.5),
(6, 0.5),
(7, 0.5),
(8, 0.5)
GO


INSERT INTO UserSubmission values (0, 1, 4); -- Value: 0, User 4, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 5); -- Value: 0, User 5, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 6); -- Value: 0, User 6, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 7); -- Value: 0, User 7, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 8); -- Value: 0, User 8, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 4); -- Value: 0, User 4, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 5); -- Value: 0, User 5, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 6); -- Value: 0, User 6, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 7); -- Value: 0, User 7, Submission 1
GO
INSERT INTO UserSubmission values (0, 1, 8); -- Value: 0, User 8, Submission 1
GO


SELECT * FROM Users -- Even though we've added loads of new users suggesting 0 as value, the final value
-- is remaining 1.0, because when a new value (0) is submitted, it varies too much from the global average
--(1), causing the reliability of that user to go down, and that user ends up making no influence on the
-- global average!
4

3 回答 3

1

这是一个替代估计,它仍然有点临时,但不会产生权重 0。

1)为每个用户产生一个指数衰减的平方误差估计。从一个可调的任意估计 K 开始。然后每次用户产生一个值 a 并且组均值为 g 产生一个平方误差 E = (a - g) * (a - g) 并将平方误差的估计从之前更改为after = before * x + E * (1 - x) 其中 x 是另一个介于 0 和 1 之间的可调常数,它调整旧估计值的衰减速度。这个估计值永远不会完全降到零,但由于下一步,最好阻止它降低到某个可调值以下。

2) 要获得新的全局估计值,请像以前一样使用加权平均值,但将权重设为该用户当前平方误差估计值的倒数。

如果所有用户都是无偏的,那么指数衰减的估计最终可能会成为每个用户的平均平方误差的不错估计,然后权重将是估计的线性组合,从而最小化全局估计的预期平方误差。检查:如果不同的用户我提交了来自同一来源的 Ni 估计的平均值,那么每个用户估计的均方误差将为 1/Ni,因此乘以这个的倒数会将他们的平均值变成每个用户产生的估计的原始总和用户和加权估计最终只会合并估计。

于 2013-04-11T19:35:05.527 回答
0

第一个建议,摆脱绝对差异。他们使数学变得比应有的困难。使用平方差来保持简单。

跟踪每个用户提交值的总和和计数。

g= (sum1*r1+sum2*r2+sum3*r3)/(count1*r1+count2*r2+count3*r3)

将 sum 初始化为 0.5,将 count 初始化为 1,将 r 初始化为 1.0。

每当您收到评分时,更新该用户的总和和计数、总体 g 和每个用户的可靠性,使用:

r = 1- (g - 总和/计数)^2。

从本质上讲,您正在跟踪一些“先前”的评级。如果将 count 初始化为较大的数字,则该算法可以抵抗错误的提交值,但需要更多时间来收敛。如果您减少初始计数(极端情况为 0),情况正好相反。

于 2013-04-11T19:34:52.720 回答
0

在计算新的全局平均值之前不要更新用户的可靠性。

g was initially "don't care"
user1 r was initially 0.5
user2 r was initially 0.5
user3 r was initially 0.5

然后

user1 a is submitted, 1.0
g = (0.5)*(1.0)/(0.5) = 1.0
user1 reliability = 1 - ABS (1 -1) = 1
user2 a is submitted, 1.0
g = ((1.0)*(1.0) + (0.5)*(1.0))/(1 + 0.5) = 1.0
user1 reliability = 1 - ABS (1 -1) = 1
user2 reliability = 1 - ABS (1 -1) = 1
user3 a is submitted, 0.0
g = ((1.0)*(1.0) + (1.0)*(1.0) + (0.5)*(0))/(1 + 1 + 0.5) = 0.8
user1 reliability = 1 - ABS (0.8 -1) = 0.8
user2 reliability = 1 - ABS (0.8 -1) = 0.8
user3 reliability = 1 - ABS (0.8 - 0.0) = 0.2
于 2013-04-11T20:49:04.237 回答