5

So, my google-fu is weak... I could not find another instance of my errors.

I've been having problems with my teamcity nuget repository since a day. NuGet downloads from the repo fail with unexpected EOF's or corrupt package warnings.

As far as I can tell this is not a hardware failure, the vm, and vm-host do not report disk errors.

To add insult to injury the teamcity logfile 'teamcity-javaLogging-2013-07-17.log' grows unbounded (+3GB in the time to type this, where 10 MB a day is normal) with the stacktraces like the ones below.

My Teamcity version is 7.1.5 (build 24400)

Anyone know how to recover from this failure?

I've not yet summoned up enough courage to just clear all the caches I can find on the teamcity admin page ( Administration > Diagnostics > Caches) because there is a warning in scary yellow on that page not to do that.

Below is a sample of the stacktraces I'm getting.

17-jul-2013 3:00:02 net.sf.ehcache.store.DiskStore get
SEVERE: provider-nugetCache: Could not read disk store element for key 2731. Error was unexpected EOF in middle of data block
java.io.StreamCorruptedException: unexpected EOF in middle of data block
        at java.io.ObjectInputStream$BlockDataInputStream.refill(Unknown Source)
        at java.io.ObjectInputStream$BlockDataInputStream.read(Unknown Source)
        at java.io.DataInputStream.readInt(Unknown Source)
        at java.io.ObjectInputStream$BlockDataInputStream.readInt(Unknown Source)
        at java.io.ObjectInputStream.readInt(Unknown Source)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.SerializableEntry.readSplitted(SerializableEntry.java:5)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.EntryImpl.readObjectInternal(EntryImpl.java:34)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.SerializableEntry.readExternal(SerializableEntry.java:16)
        at java.io.ObjectInputStream.readExternalData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at net.sf.ehcache.store.DiskStore.loadElementFromDiskElement(DiskStore.java:313)
        at net.sf.ehcache.store.DiskStore.get(DiskStore.java:268)
        at net.sf.ehcache.Cache.searchInDiskStore(Cache.java:1290)
        at net.sf.ehcache.Cache.get(Cache.java:904)
        at net.sf.ehcache.Cache.get(Cache.java:879)
        at jetbrains.buildServer.serverSide.metadata.impl.cache.TypedCacheImpl.getValue(TypedCacheImpl.java:3)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.MetadataStorageImpl.getReportedKeys(MetadataStorageImpl.java:7)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.MetadataStorageImpl.removeBuild(MetadataStorageImpl.java:45)
        at jetbrains.buildServer.serverSide.metadata.impl.indexer.BuildIndexCleaner.performCleanup(BuildIndexCleaner.java:16)
        at jetbrains.buildServer.serverSide.impl.cleanup.HistoryEntryCleaner.cleanupExtensionsData(HistoryEntryCleaner.java:38)
        at jetbrains.buildServer.serverSide.impl.cleanup.HistoryEntryCleaner.performCleanup(HistoryEntryCleaner.java:138)
        at jetbrains.buildServer.serverSide.impl.cleanup.HistoryEntryCleaner.performCleanup(HistoryEntryCleaner.java:132)
        at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl$3.performCleanup(ServerCleanupManagerImpl.java)
        at jetbrains.buildServer.serverSide.db.DBFacade$1$1.doInConnection(DBFacade.java:178)
        at jetbrains.buildServer.serverSide.db.DBFacade$6.doInConnection(DBFacade.java:415)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:341)
        at jetbrains.buildServer.serverSide.db.DBFacade._runSql(DBFacade.java:411)
        at jetbrains.buildServer.serverSide.db.DBFacade.access$000(DBFacade.java:33)
        at jetbrains.buildServer.serverSide.db.DBFacade$1.doInTransaction(DBFacade.java:174)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
        at jetbrains.buildServer.serverSide.db.DBFacade.compact(DBFacade.java:171)
        at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl.startCleanup(ServerCleanupManagerImpl.java:74)
        at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl$2.run(ServerCleanupManagerImpl.java:0)
        at java.util.TimerThread.mainLoop(Unknown Source)
        at java.util.TimerThread.run(Unknown Source)
17-jul-2013 3:00:02 net.sf.ehcache.store.DiskStore remove
SEVERE: provider-nugetCache: Could not remove disk store entry for key 2731. Error was unexpected EOF in middle of data block
java.io.StreamCorruptedException: unexpected EOF in middle of data block
        at java.io.ObjectInputStream$BlockDataInputStream.refill(Unknown Source)
        at java.io.ObjectInputStream$BlockDataInputStream.read(Unknown Source)
        at java.io.DataInputStream.readInt(Unknown Source)
        at java.io.ObjectInputStream$BlockDataInputStream.readInt(Unknown Source)
        at java.io.ObjectInputStream.readInt(Unknown Source)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.SerializableEntry.readSplitted(SerializableEntry.java:5)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.EntryImpl.readObjectInternal(EntryImpl.java:34)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.SerializableEntry.readExternal(SerializableEntry.java:16)
        at java.io.ObjectInputStream.readExternalData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
        at java.io.ObjectInputStream.readSerialData(Unknown Source)
        at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
        at java.io.ObjectInputStream.readObject0(Unknown Source)
        at java.io.ObjectInputStream.readObject(Unknown Source)
        at net.sf.ehcache.store.DiskStore.loadElementFromDiskElement(DiskStore.java:313)
        at net.sf.ehcache.store.DiskStore.remove(DiskStore.java:483)
        at net.sf.ehcache.Cache.remove(Cache.java:1465)
        at net.sf.ehcache.Cache.remove(Cache.java:1392)
        at net.sf.ehcache.Cache.remove(Cache.java:1350)
        at net.sf.ehcache.Cache.remove(Cache.java:1328)
        at jetbrains.buildServer.serverSide.metadata.impl.cache.TypedCacheImpl.remove(TypedCacheImpl.java:16)
        at jetbrains.buildServer.serverSide.metadata.impl.metadata.MetadataStorageImpl.removeBuild(MetadataStorageImpl.java:30)
        at jetbrains.buildServer.serverSide.metadata.impl.indexer.BuildIndexCleaner.performCleanup(BuildIndexCleaner.java:16)
        at jetbrains.buildServer.serverSide.impl.cleanup.HistoryEntryCleaner.cleanupExtensionsData(HistoryEntryCleaner.java:38)
        at jetbrains.buildServer.serverSide.impl.cleanup.HistoryEntryCleaner.performCleanup(HistoryEntryCleaner.java:138)
        at jetbrains.buildServer.serverSide.impl.cleanup.HistoryEntryCleaner.performCleanup(HistoryEntryCleaner.java:132)
        at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl$3.performCleanup(ServerCleanupManagerImpl.java)
        at jetbrains.buildServer.serverSide.db.DBFacade$1$1.doInConnection(DBFacade.java:178)
        at jetbrains.buildServer.serverSide.db.DBFacade$6.doInConnection(DBFacade.java:415)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:341)
        at jetbrains.buildServer.serverSide.db.DBFacade._runSql(DBFacade.java:411)
        at jetbrains.buildServer.serverSide.db.DBFacade.access$000(DBFacade.java:33)
        at jetbrains.buildServer.serverSide.db.DBFacade$1.doInTransaction(DBFacade.java:174)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
        at jetbrains.buildServer.serverSide.db.DBFacade.compact(DBFacade.java:171)
        at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl.startCleanup(ServerCleanupManagerImpl.java:74)
        at jetbrains.buildServer.serverSide.impl.cleanup.ServerCleanupManagerImpl$2.run(ServerCleanupManagerImpl.java:0)
        at java.util.TimerThread.mainLoop(Unknown Source)
        at java.util.TimerThread.run(Unknown Source)
4

3 回答 3

4

据我所知，TeamCity NuGet Server 受制于“管理 | 项目相关设置 | 构建历史清理”中定义的工件清理策略，因此请确保您的包仍然存在！

我个人的建议是始终设置专用的 nuget 服务器。您可以免费设置一个，只需克隆GitHub 上的官方 NugetGallery 项目：这与 nuget.org 使用的代码库相同，因此您将拥有熟悉的 UI 和更高的性能（NugetGallery 利用 Lucene.NET 索引功能）。

于 2013-07-28T15:58:18.717 回答
3

正如 Remco 所说，您可以通过转到 http://{teamcity}/admin/admin.html?item=diagnostics&tab=cache 并单击“buildsMetadata”旁边的“reset”来清除包缓存。这将从您的提要中删除所有 NuGet 包，直到您重新编制索引。

您可以通过调用 TeamCity 的 REST API 重新索引由单个构建生成的 NuGet 包。要重新索引所有构建，您必须编写一个脚本来循环所有构建并重新索引每个构建。

错误报告：http: //youtrack.jetbrains.com/issue/TW-25384
http://youtrack.jetbrains.com/issue/TW-23576

重新索引脚本示例：http:
//youtrack.jetbrains.com/issue/TW-19411#comment=27-408230

于 2013-12-20T00:55:19.050 回答
1

似乎 TeamCity NuGet 服务器的索引以某种方式损坏，包很好，因此清除缓存是解决方案。

但是，我没有弄清楚如何重新填充索引。

请注意，清除 nuget 包缓存会从 TeamCity NuGet 服务器中删除所有包，因此您可以从头开始...

于 2013-08-29T06:33:42.193 回答