6

替代文字

这是场景:

我有一个名为 MarketDataCurrent (MDC) 的表,它可以实时更新股票价格。

我有一个名为“LiveFeed”的进程,它从网络中读取价格流,对插入进行排队,并使用“批量上传到临时表然后插入/更新到 MDC 表”。(批量更新)

我有另一个进程读取这些数据,计算其他数据,然后使用类似的 BulkUpsert 存储过程将结果保存回同一个表中。

第三,有大量用户运行 C# Gui 轮询 MDC 表并从中读取更新。

现在,在数据快速变化的那一天,事情运行得相当顺利,但是,在市场交易时间之后,我们最近开始看到数据库中出现越来越多的死锁异常,现在我们每天看到 10-20 个. 这里要注意的重要一点是,这些发生在值不变的情况下。

以下是所有相关信息:

表定义:

CREATE TABLE [dbo].[MarketDataCurrent](
 [MDID] [int] NOT NULL,
 [LastUpdate] [datetime] NOT NULL,
 [Value] [float] NOT NULL,
 [Source] [varchar](20) NULL, 
CONSTRAINT [PK_MarketDataCurrent] PRIMARY KEY CLUSTERED 
(
 [MDID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, 
ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

-

替代文字

我有一个正在运行的 Sql Profiler Trace,正在捕获死锁,这就是所有图表的样子。

替代文字

进程 258 被称为以下“BulkUpsert”存储过程,重复,而 73 正在调用下一个:

ALTER proc [dbo].[MarketDataCurrent_BulkUpload]
 @updateTime datetime,
 @source varchar(10)
as

begin transaction

update c with (rowlock) set LastUpdate = getdate(), Value = t.Value, Source = @source 
from MarketDataCurrent c INNER JOIN #MDTUP t ON c.MDID = t.mdid
where c.lastUpdate < @updateTime
and   c.mdid not in (select mdid from MarketData where LiveFeedTicker is not null     and     PriceSource like 'LiveFeed.%')
and   c.value <> t.value

insert  into MarketDataCurrent
with (rowlock)
select  MDID, getdate(), Value, @source from #MDTUP 
where mdid not in (select mdid from MarketDataCurrent with (nolock))
and  mdid not in (select mdid from MarketData where LiveFeedTicker is not null     and PriceSource like 'LiveFeed.%')

commit

还有一个:

ALTER PROCEDURE [dbo].[MarketDataCurrent_LiveFeedUpload] 
AS
begin transaction

 -- Update existing mdid
 UPDATE c WITH (ROWLOCK) SET LastUpdate = t.LastUpdate, Value = t.Value, Source = t.Source 
 FROM MarketDataCurrent c INNER JOIN #TEMPTABLE2 t ON c.MDID = t.mdid;

 -- Insert new MDID
 INSERT INTO MarketDataCurrent with (ROWLOCK) SELECT * FROM #TEMPTABLE2 
 WHERE MDID NOT IN (SELECT MDID FROM MarketDataCurrent with (NOLOCK))

 -- Clean up the temp table
 DELETE #TEMPTABLE2

commit

澄清一下,这些临时表是由同一连接上的 C# 代码创建的,并使用 C# SqlBulkCopy 类填充。

对我来说,它看起来像是在桌子的 PK 上死锁,所以我尝试删除那个 PK 并改用唯一约束,但这使死锁的数量增加了 10 倍。

我完全不知道如何处理这种情况,并且对任何建议都持开放态度。

帮助!!


响应对 XDL 的请求,这里是:

<deadlock-list>
 <deadlock victim="processc19978">
  <process-list>
   <process id="processaf0b68" taskpriority="0" logused="0" waitresource="KEY: 6:72057594090487808 (d900ed5a6cc6)" waittime="718" ownerId="1102128174" transactionname="user_transaction" lasttranstarted="2010-06-11T16:30:44.750" XDES="0xffffffff817f9a40" lockMode="U" schedulerid="3" kpid="8228" status="suspended" spid="73" sbid="0" ecid="0" priority="0" transcount="2" lastbatchstarted="2010-06-11T16:30:44.750" lastbatchcompleted="2010-06-11T16:30:44.750" clientapp=".Net SqlClient Data Provider" hostname="RISKAPPS_VM" hostpid="3836" loginname="RiskOpt" isolationlevel="read committed (2)" xactid="1102128174" currentdb="6" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
    <executionStack>
     <frame procname="MKP_RISKDB.dbo.MarketDataCurrent_BulkUpload" line="28" stmtstart="1062" stmtend="1720" sqlhandle="0x03000600a28e5e4ef4fd8e00849d00000100000000000000">
UPDATE c WITH (ROWLOCK) SET LastUpdate = getdate(), Value = t.Value, Source = @source 
FROM MarketDataCurrent c INNER JOIN #MDTUP t ON c.MDID = t.mdid
WHERE c.lastUpdate &lt; @updateTime
and   c.mdid not in (select mdid from MarketData where BloombergTicker is not null and PriceSource like &apos;Blbg.%&apos;)
and   c.value &lt;&gt; t.value     </frame>
     <frame procname="adhoc" line="1" stmtstart="88" sqlhandle="0x01000600c1653d0598706ca7000000000000000000000000">
exec MarketDataCurrent_BulkUpload @clearBefore, @source     </frame>
     <frame procname="unknown" line="1" sqlhandle="0x000000000000000000000000000000000000000000000000">
unknown     </frame>
    </executionStack>
    <inputbuf>
(@clearBefore datetime,@source nvarchar(10))exec MarketDataCurrent_BulkUpload @clearBefore, @source    </inputbuf>
   </process>
   <process id="processc19978" taskpriority="0" logused="0" waitresource="KEY: 6:72057594090487808 (74008e31572b)" waittime="718" ownerId="1102128228" transactionname="user_transaction" lasttranstarted="2010-06-11T16:30:44.780" XDES="0x380be9d8" lockMode="U" schedulerid="5" kpid="8464" status="suspended" spid="248" sbid="0" ecid="0" priority="0" transcount="2" lastbatchstarted="2010-06-11T16:30:44.780" lastbatchcompleted="2010-06-11T16:30:44.780" clientapp=".Net SqlClient Data Provider" hostname="RISKBBG_VM" hostpid="4480" loginname="RiskOpt" isolationlevel="read committed (2)" xactid="1102128228" currentdb="6" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
    <executionStack>
     <frame procname="MKP_RISKDB.dbo.MarketDataCurrentBlbgRtUpload" line="14" stmtstart="840" stmtend="1220" sqlhandle="0x03000600005f9d24c8878f00849d00000100000000000000">
UPDATE c WITH (ROWLOCK) SET LastUpdate = t.LastUpdate, Value = t.Value, Source = t.Source 
        FROM MarketDataCurrent c INNER JOIN #TEMPTABLE2 t ON c.MDID = t.mdid;

        -- Insert new MDID     </frame>
     <frame procname="adhoc" line="1" sqlhandle="0x010006004a58132228bf8d73000000000000000000000000">
MarketDataCurrentBlbgRtUpload     </frame>
    </executionStack>
    <inputbuf>
MarketDataCurrentBlbgRtUpload    </inputbuf>
   </process>
  </process-list>
  <resource-list>
   <keylock hobtid="72057594090487808" dbid="6" objectname="MKP_RISKDB.dbo.MarketDataCurrent" indexname="PK_MarketDataCurrent" id="lock5ba77b00" mode="U" associatedObjectId="72057594090487808">
    <owner-list>
     <owner id="processc19978" mode="U"/>
    </owner-list>
    <waiter-list>
     <waiter id="processaf0b68" mode="U" requestType="wait"/>
    </waiter-list>
   </keylock>
   <keylock hobtid="72057594090487808" dbid="6" objectname="MKP_RISKDB.dbo.MarketDataCurrent" indexname="PK_MarketDataCurrent" id="lock65dca340" mode="U" associatedObjectId="72057594090487808">
    <owner-list>
     <owner id="processaf0b68" mode="U"/>
    </owner-list>
    <waiter-list>
     <waiter id="processc19978" mode="U" requestType="wait"/>
    </waiter-list>
   </keylock>
  </resource-list>
 </deadlock>
</deadlock-list>
4

4 回答 4

2

死锁似乎是密钥访问顺序上的直接死锁。一个简单的解释是两个批量更新操作之间更新的密钥重叠。

不过,一个不太简单的解释是,在 SQL Server(以及其他服务器)中,锁定的键是hashed,并且存在(非常重要的)散列冲突概率。这可以解释为什么您最近看到的死锁比以前更多:只是您的数据量增加了,因此冲突概率增加了。如果这看起来很深奥且不太可能,请继续阅读%%lockres%% 碰撞概率魔法标记:16,777,215以及从该链接链接的文章。概率高得惊人,因为完美仅插入约 1600 万次后,您的密钥分配就有 50% 的冲突概率。对于正常的、现实世界的密钥分布,您仅在几千次插入时就有显着的冲突概率。不幸的是,没有解决方法。如果这确实是问题,您唯一的解决方案是减少批次的大小(#temp 表的大小),以降低冲突概率。或者处理死锁并重试......无论如何你都必须这样做,但至少你可以处理更少的死锁。

于 2010-06-11T22:29:59.793 回答
1

它发生在主要工作时间之后,数据没有变化,而且最近才开始。服务器最近有什么变化吗?我怀疑一些新的数据库维护工作可能会干扰。

顺便说一句,如果您知道市场关闭并且数据没有变化,为什么您的流程仍在运行?

于 2010-06-11T13:27:20.427 回答
1

我想回答我在评论中提出的一个问题,即,

“你如何识别锁定的行?”。

在下面的死锁 XDL 中,在锁定的两个“进程”节点上,有一个waitresource属性。在这种情况下:

waitresource="KEY: 6:72057594090487808 (d4005c04b35f)

waitresource="KEY: 6:72057594090487808 (b00072ea4ffd)

使用%%lockres%%Remus 所指的关键字,

select %%lockres%%, * from MarketDataCurrent 
   where %%lockres%% in ('(d4005c04b35f)', '(b00072ea4ffd)')

这产生了冲突的两行。它们确实是唯一的 id,并且没有冲突。我仍然不知道为什么我在这里陷入僵局,但我越来越接近了。

我会注意到这两个 id 应该只来自 LiveFeed 程序,但同样,更新中有一个子句应该从另一端的实际更新中过滤掉这一行。

<deadlock-list>
 <deadlock victim="processffffffff8f5872e8">
  <process-list>
   <process id="process8dcb68" taskpriority="0" logused="1256" waitresource="KEY: 6:72057594090487808 (d4005c04b35f)" waittime="1906" ownerId="1349627324" transactionname="user_transaction" lasttranstarted="2010-06-16T16:50:04.727" XDES="0x424e6258" lockMode="U" schedulerid="2" kpid="1004" status="suspended" spid="683" sbid="0" ecid="0" priority="0" transcount="2" lastbatchstarted="2010-06-16T16:50:04.727" lastbatchcompleted="2010-06-16T16:50:04.727" clientapp=".Net SqlClient Data Provider" hostname="RISKAPPS_VM" hostpid="2600" loginname="RiskOpt" isolationlevel="read committed (2)" xactid="1349627324" currentdb="6" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
    <executionStack>
     <frame procname="MKP_RISKDB.dbo.MarketDataCurrent_BulkUpload" line="28" stmtstart="1062" stmtend="1720" sqlhandle="0x03000600a28e5e4ef4fd8e00849d00000100000000000000">
        UPDATE c WITH (ROWLOCK) SET LastUpdate = getdate(), Value = t.Value, Source = @source 
        FROM MarketDataCurrent c INNER JOIN #MDTUP t ON c.MDID = t.mdid
        WHERE c.lastUpdate &lt; @updateTime
        and   c.mdid not in (select mdid from MarketData where BloombergTicker is not null and PriceSource like &apos;Blbg.%&apos;)
        and   c.value &lt;&gt; t.value     </frame>
             <frame procname="adhoc" line="1" stmtstart="88" sqlhandle="0x01000600c1653d0598706ca7000000000000000000000000">
        exec MarketDataCurrent_BulkUpload @clearBefore, @source     </frame>
     <frame procname="unknown" line="1" sqlhandle="0x000000000000000000000000000000000000000000000000">unknown</frame>
    </executionStack>
    <inputbuf>(@clearBefore datetime,@source nvarchar(10))exec MarketDataCurrent_BulkUpload @clearBefore, @source</inputbuf>
   </process>
   <process id="processffffffff8f5872e8" taskpriority="0" logused="0" waitresource="KEY: 6:72057594090487808 (b00072ea4ffd)" waittime="1921" ownerId="1349627388" transactionname="user_transaction" lasttranstarted="2010-06-16T16:50:04.757" XDES="0x289ea040" lockMode="U" schedulerid="5" kpid="11192" status="suspended" spid="382" sbid="0" ecid="0" priority="0" transcount="2" lastbatchstarted="2010-06-16T16:50:04.757" lastbatchcompleted="2010-06-16T16:50:04.757" clientapp=".Net SqlClient Data Provider" hostname="RISKBBG_VM" hostpid="2452" loginname="RiskOpt" isolationlevel="read committed (2)" xactid="1349627388" currentdb="6" lockTimeout="4294967295" clientoption1="671088672" clientoption2="128056">
    <executionStack>
     <frame procname="MKP_RISKDB.dbo.MarketDataCurrentBlbgRtUpload" line="14" stmtstart="840" stmtend="1220" sqlhandle="0x03000600005f9d24c8878f00849d00000100000000000000">
        UPDATE c WITH (ROWLOCK) SET LastUpdate = t.LastUpdate, Value = t.Value, Source = t.Source 
        FROM MarketDataCurrent c INNER JOIN #TEMPTABLE2 t ON c.MDID = t.mdid;
    </frame>
     <frame procname="adhoc" line="1" sqlhandle="0x010006004a58132228bf8d73000000000000000000000000">
        MarketDataCurrentBlbgRtUpload     </frame>
    </executionStack>
    <inputbuf>
        MarketDataCurrentBlbgRtUpload    </inputbuf>
   </process>
  </process-list>
  <resource-list>
   <keylock hobtid="72057594090487808" dbid="6" objectname="MKP_RISKDB.dbo.MarketDataCurrent" indexname="PK_MarketDataCurrent" id="lock409d32c0" mode="U" associatedObjectId="72057594090487808">
    <owner-list>
     <owner id="processffffffff8f5872e8" mode="U"/>
    </owner-list>
    <waiter-list>
     <waiter id="process8dcb68" mode="U" requestType="wait"/>
    </waiter-list>
   </keylock>
   <keylock hobtid="72057594090487808" dbid="6" objectname="MKP_RISKDB.dbo.MarketDataCurrent" indexname="PK_MarketDataCurrent" id="lock706647c0" mode="U" associatedObjectId="72057594090487808">
    <owner-list>
     <owner id="process8dcb68" mode="U"/>
    </owner-list>
    <waiter-list>
     <waiter id="processffffffff8f5872e8" mode="U" requestType="wait"/>
    </waiter-list>
   </keylock>
  </resource-list>
 </deadlock>
</deadlock-list>
于 2010-06-16T21:13:43.210 回答
0

经过近两年烦人的死锁警告电子邮件后,我终于解决了这个问题。

我通过在我的竞争插入上使用 FULL TABLE LOCKING 解决了这个问题。我曾尝试将锁定减少到行级别,但锁定正在升级到表级别。最后,我认为该表足够小,即使有很多用户每秒都在对其进行读写,但为了数据一致性,我愿意为全锁带来很小的性能损失。

此外,使用 MERGE 将插入/更新组合到一个原子语句允许我这样做。

这是解决的生产代码(它有效!):

declare @date datetime;
set @date = getdate();

merge marketdatacurrent with (tablockx) as mdc

using #MDTUP as upload
    on mdc.MDID = upload.MDID

when matched then
    update
    set mdc.lastupdate = @date,
        mdc.value = upload.value,
        mdc.source = @source

when not matched then
    insert ( mdid, lastupdate, value, source )
    values ( upload.mdid, @date, upload.value, @source);
于 2012-02-29T15:26:07.910 回答