16

Background:

We have a large (15+ million) table of items which is updated frequently throughout the day (average 300K daily changes). Pending changes are stored in a staging table, and a job runs throughout the day to read changes from this table, make the updates, and mark the changes as processed.

Many other applications use the data in the items table to perform various tasks. Often, these tasks are scheduled and intensive, as they involve comparing live data with old snapshots, and updating other systems accordingly. For example, we list items on eBay, and compare live item data with our existing listings to see whether we need to insert any new eBay listings, remove items we've sold, update quantities, etc. Because the data is so large, most of these applications run infrequently, leaving things out of date much of the time.

My Question:

We are considering implementing a publisher/subscriber pattern using the Service Broker. The goal would be to publish a message when an item changes, which various other systems (like our eBay application) can subscribe to. This would enable us to make more granular updates closer to real-time, rather than large and infrequent updates that involve querying all data, not just what has changed. However, after using the Google it doesn't seem like this is a common database pattern, and that raises red flags. Is this not a valid use of the Service Broker (though I found a small section in Pro Sql Server 2008 Service Broker on doing Pub/Sub)? How is this problem normally solved? It seems like a common enough problem.

TL;DR:

Goal: Update various systems in a dynamic, loosely coupled way when single items change.

Question: Is a pub/sub style solution implemented with Service Broker a viable solution in a high volume setting?

4

3 回答 3

3

这是一种非常常见的集成模式。

我不熟悉 SQL Server Service Broker,但如果您查看任何集成中间件堆栈 - TIBCO、Oracle、webMethods、Mule、Informatica 等 - 它们都提供了一个“数据库适配器”来执行您描述的任务。

一般模式是数据库中的更新触发消息发布。这可以通过数据库表中的触发器或通过适配器“轮询”表以获取新更新来完成。任何一种方法都有优点和缺点。

这种模式的主要好处是(正如您所怀疑的)更频繁和及时地更新其他系统 - 一种更“实时”的业务方式。此外,如果您执行到规范消息格式的转换,那么系统之间的耦合会更松散,因此在需要更改/更新系统时会减少痛苦。

于 2012-07-01T08:14:56.213 回答
0

通常这类问题最终会变得比以数据库为中心的 pub/sub 实现所能提供的更大(范围)。如果我可以提出建议,请考虑使用全面的服务总线:

nServiceBus(免费到一定大小,过去相当便宜)

您将获得一个可靠的发布/订阅框架,但易于使用,并且拥有庞大的客户、文章和示例用户群。

于 2012-07-09T23:45:24.273 回答
0

这不是一种常见的数据库模式,因为大多数关系数据库都非常缺乏开箱即用的能力。如果没有 Service Broker,SQL Server 也会如此。

Service Broker 的一位开发人员谈到了 Service Broker 如何应对大多数公司转向 NoSQL 解决方案的挑战

据我了解,Service Broker 旨在完全满足您的需求。当数据发生变化时,它甚至可以External Activation用来运行自定义应用程序

于 2012-06-30T08:46:25.143 回答