orleans - How to handle split-brain?

Question

I have read in Orleans FAQ when split-brain could happen but I don't understand what bad can happen and how to handle it properly.

FAQ says something vague like:

You just need to consider the rare possibility of having two instances of an actor while writing your application.

But how actually should I consider this and what can happen if I won't?

Orleans Paper (http://research.microsoft.com/pubs/210931/Orleans-MSR-TR-2014-41.pdf) says this:

application can rely on external persistent storage to provide stronger data consistency

But I don't understand what this means.

Suppose split brain happened. Now I have two instances of one grain. When I'll send a few messages they could be received by these two (or there can be even more?) different instances. Suppose each instance prior to receiving these messages had same state. Now, after processing these messages they have different states.

How they should persist their states? There could be a conflict.

When another instances will be destroyed and only one will remain what will happen to the states of destroyed instances? It'll be like messages processed by them has never been processed? Then client state and server state could be desyncronized IIUC.

I see this (split-brain) as a big problem and I don't understand why there is so little attention to it.

Answer 1

Orleans 利用了存储提供者的一致性保证。当你this.WriteStateAsync()从一个grain调用时，存储提供者确保grain已经看到了所有之前的写入。如果没有，则抛出异常。您可以捕获该异常并调用DeactivateOnIdle()并重新抛出异常或调用并重ReadStateAsync()试。因此，如果您在脑裂情况下有 2 个颗粒，其中一个WriteStateAsync()首先调用会阻止另一个颗粒在没有首先读取最新状态的情况下写入状态。

更新：从Orleans v1.5.0开始，当当前执行的调用完成时，允许将 anInconsistentStateException扔回调用者的颗粒将自动停用。一个grain可以catch处理异常以避免自动停用。