1

Technology Versions:

Hibernate 3.6.5

Hibernate Search 3.4.0

Lucene (lucene-core-*.jar) 3.1.0

Spring 3.1.0.M2

Hibernate (Search) Configuration:

<prop key="hibernate.search.default.indexBase">/some/dir</prop>
<prop key="hibernate.search.default.directory_provider">org.hibernate.search.store.FSDirectoryProvider</prop>
<prop key="hibernate.search.default.locking_strategy">native</prop>
<prop key="hibernate.search.default.exclusive_index_use">false</prop>

The issue:
 Our application is deployed in Amazon (AWS cloud), but we've faced this issue in our local clusters as well:

The application design is such that there is a Thread that is spawned from within the main (Web) application wherefrom we need to update an Indexed entity. Basically it's like a status monitor thread which reads a .status file and updates the database every 30 seconds or so. This keeps happening for about 10mins to 1/2 an hour on an average. The issue we see is that: every few days we need to regenerate the indexes because Hibernate Search stops returning anything for the entity in question (the one discussed above).

I went through few forums and seems it is suggested that only a single thread should be updating the Lucene indexes. But it is also given that index writing is thread-safe. So even if multiple threads are writing to the same index, I still expect that it should not cause the issue (of nothing being returned in the search). That is to say, I may get stale status of the entity in question, but still, something should be returned.

We're using the default IndexReader/Writer implementation of Hibernate Search.

Any help would be highly appreciated. Thanks.

4

2 回答 2

2

这里有一些想法。

我浏览了几个论坛，似乎建议只有一个线程应该更新 Lucene 索引。

这通常不是真的。Lucene 和 Hibernate Search 允许多个索引编写器，但是对索引的访问必须通过 Lucene 的org.apache.lucene.store.LockFactory正确同步。锁工厂是可配置的，您可以通过属性 *hibernate.search.default.locking_strategy* 使用本地锁工厂。问题可能在于此策略是基于文件的。我对亚马逊的分布式文件系统内部工作了解不多，但我想文件锁在这种情况下不起作用。您可能需要实施自定义锁定策略。

但也考虑到索引写入是线程安全的。因此，即使多个线程正在写入同一个索引，我仍然希望它不会导致问题

正确，只要锁定有效。

如果您可以保证只有您的更新线程将写入索引，则另一种方法是不使用锁（将 *hibernate.search.default.locking_strategy* 设置为none ）。您是否启用了更新线程自动索引？如果是这样，请尝试将其关闭（允许您的用例允许您这样做）。

于 2012-05-14T12:13:43.487 回答
0

好的，对于那些来到这里寻找解决方案的人——这就是我们最终所做的，这似乎已经解决了这个问题（我们已经在 AWS 上对此修复进行了负载测试，到目前为止还没有遇到任何问题）：

我们基于 Lucene 编写了锁工厂的实现NativeFSLockFactory。我们修改了obtain方法，以便在放弃之前重试几次获取锁。我们在重试之间添加了一个短暂的延迟 ( sleep) 以处理NFS延迟。

我们用 Lucene 测试了这个修复，LockVerifyServer并观察到在高负载下，即使一些锁获取请求必须等待获取锁——但最终每个锁获取请求都会得到解决。这实时解释为索引文件的成功更新。

谢谢哈代向我们展示了道路。:)

更新：昨天，我们不得不将重试计数提高到更高的数字 ~ 30，因为我们遇到了先前值为 3 的索引更新滑点。此后一切似乎都很好。

于 2012-06-07T06:27:09.397 回答