嗨,我在服务器上运行 BAM 进行测试,该服务器具有以下文件系统分布:
root@serv:/# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VG-LV_root_ 3.8G 2.3G 1.4G 62% /
udev 3.0G 4.0K 3.0G 1% /dev
tmpfs 1.2G 224K 1.2G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.0G 0 3.0G 0% /run/shm
/dev/mapper/VG-LV_var 7.3G 4.5G 2.5G 65% /var
我将 BAM 放入具有 7.3GB 空间的 var 中。如您所见,我有足够的空间来运行 WSO2 产品,但 BAM :-D
运行 BAm 3 天后,我面临空间问题:
在 tmp 我有 1GB 的 hadoop
root@serv:/# du -sh /tmp/*
1023M /tmp/hadoop
4.0K /tmp/hadoop-root
4.0K /tmp/hsperfdata_root
4.0K /tmp/mc-root
117M /tmp/root
在 BAM 内部,它在 tmp 中使用了 2.9GB。
root@serv:/# du -sh /var/BAM/wso2bam-2.0.1/*
160K /var/BAM/wso2bam-2.0.1/bin
236K /var/BAM/wso2bam-2.0.1/dbscripts
8.0K /var/BAM/wso2bam-2.0.1/INSTALL.txt
5.0M /var/BAM/wso2bam-2.0.1/lib
52K /var/BAM/wso2bam-2.0.1/LICENSE.txt
12K /var/BAM/wso2bam-2.0.1/README.txt
8.0K /var/BAM/wso2bam-2.0.1/release-notes.html
540M /var/BAM/wso2bam-2.0.1/repository
80K /var/BAM/wso2bam-2.0.1/resources
14M /var/BAM/wso2bam-2.0.1/samples
2.9G /var/BAM/wso2bam-2.0.1/tmp
88K /var/BAM/wso2bam-2.0.1/webapp-mode
4.0K /var/BAM/wso2bam-2.0.1/wso2carbon.pid
我的问题与以下事实有关,即我仍然有磁盘空间,并且在短短 3 天内 BAM 使用了 4GB,所以:长时间部署的最佳 HDD 大小是多少,或者我可以做些什么来避免这个错误(在生产环境中我可以使用更多的磁盘空间,但这是我的客户关心的问题):注意到在这 3 天里,我只监控 2 台服务器,AS 和 ESB,负载最小。
TID: [0] [BAM] [2012-11-06 12:00:00,026] INFO {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask} - Running script executor task for script esb_stats_0. [Tue Nov 06 12:00:00 CST 2012] {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask}
TID: [0] [BAM] [2012-11-06 12:00:00,026] INFO {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask} - Running script executor task for script service_stats_848. [Tue Nov 06 12:00:00 CST 2012] {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask}
TID: [0] [BAM] [2012-11-06 12:00:01,213] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} - Exception: /var/BAM/wso2bam-2.0.1/tmp/root/hive_2012-11-06_12-00-00_487_706674934402805629/-local-10000/plan.xml (No space left on device) {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2012-11-06 12:00:01,213] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} - Exception: /var/BAM/wso2bam-2.0.1/tmp/root/hive_2012-11-06_12-00-00_487_706674934402805629/-local-10000/plan.xml (No space left on device) {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2012-11-06 12:00:01,216] ERROR {org.apache.hadoop.hive.ql.Driver} - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.apache.hadoop.hive.ql.Driver}
TID: [0] [BAM] [2012-11-06 12:00:01,216] ERROR {org.apache.hadoop.hive.ql.Driver} - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.apache.hadoop.hive.ql.Driver}
TID: [0] [BAM] [2012-11-06 12:00:01,219] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} - Exception: /var/BAM/wso2bam-2.0.1/tmp/root/hive_2012-11-06_12-00-00_500_8714455975079609718/-local-10000/plan.xml (No space left on device) {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2012-11-06 12:00:01,219] ERROR {org.apache.hadoop.hive.ql.exec.ExecDriver} - Exception: /var/BAM/wso2bam-2.0.1/tmp/root/hive_2012-11-06_12-00-00_500_8714455975079609718/-local-10000/plan.xml (No space left on device) {org.apache.hadoop.hive.ql.exec.ExecDriver}
TID: [0] [BAM] [2012-11-06 12:00:01,221] ERROR {org.apache.hadoop.hive.ql.Driver} - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.apache.hadoop.hive.ql.Driver}
TID: [0] [BAM] [2012-11-06 12:00:01,221] ERROR {org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl} - Error while executing Hive script.
Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl}
java.sql.SQLException: Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:189)
at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:325)
at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:225)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
TID: [0] [BAM] [2012-11-06 12:00:01,221] ERROR {org.apache.hadoop.hive.ql.Driver} - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.apache.hadoop.hive.ql.Driver}
TID: [0] [BAM] [2012-11-06 12:00:01,223] ERROR {org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl} - Error while executing Hive script.
Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask {org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl}
java.sql.SQLException: Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:189)
at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:325)
at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl$ScriptCallable.call(HiveExecutorServiceImpl.java:225)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
TID: [0] [BAM] [2012-11-06 12:00:01,231] ERROR {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask} - Error while executing script : esb_stats_0 {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask}
org.wso2.carbon.analytics.hive.exception.HiveExecutionException: Error while executing Hive script.Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl.execute(HiveExecutorServiceImpl.java:110)
at org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask.execute(HiveScriptExecutorTask.java:60)
at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:56)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
TID: [0] [BAM] [2012-11-06 12:00:01,232] ERROR {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask} - Error while executing script : service_stats_848 {org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask}
org.wso2.carbon.analytics.hive.exception.HiveExecutionException: Error while executing Hive script.Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask
at org.wso2.carbon.analytics.hive.impl.HiveExecutorServiceImpl.execute(HiveExecutorServiceImpl.java:110)
at org.wso2.carbon.analytics.hive.task.HiveScriptExecutorTask.execute(HiveScriptExecutorTask.java:60)
at org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:56)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
TID: [0] [BAM] [2012-11-06 12:00:01,743] INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} - 'admin@carbon.super [-1234]' logged in at [2012-11-06 12:00:01,742-0500] {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil}
TID: [0] [BAM] [2012-11-06 12:00:01,750] INFO {org.wso2.carbon.dashboard.gadgetrepopopulator.GadgetRepoPopulator} - Couldn't find a Dashboard at '/var/BAM/wso2bam-2.0.1/repository/resources/gadget-repo/gadget-repo.xml'. Giving up. {org.wso2.carbon.dashboard.gadgetrepopopulator.GadgetRepoPopulator}
TID: [0] [BAM] [2012-11-06 12:00:04,213] ERROR {org.apache.tiles.jsp.context.JspTilesRequestContext} - JSPException while including path '/hive-explorer/listscripts.jsp'. {org.apache.tiles.jsp.context.JspTilesRequestContext}
javax.servlet.jsp.JspException: ServletException while including page.
at org.apache.tiles.jsp.context.JspUtil.doInclude(JspUtil.java:102)
at org.apache.tiles.jsp.context.JspTilesRequestContext.include(JspTilesRequestContext.java:88)
at org.apache.tiles.jsp.context.JspTilesRequestContext.dispatch(JspTilesRequestContext.java:82)
at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:465)
at org.apache.tiles.jsp.taglib.InsertAttributeTag.render(InsertAttributeTag.java:140)
at org.apache.tiles.jsp.taglib.InsertAttributeTag.render(InsertAttributeTag.java:117)
at org.apache.tiles.jsp.taglib.RenderTagSupport.execute(RenderTagSupport.java:171)
at org.apache.tiles.jsp.taglib.RoleSecurityTagSupport.doEndTag(RoleSecurityTagSupport.java:75)
at org.apache.tiles.jsp.taglib.ContainerTagSupport.doEndTag(ContainerTagSupport.java:80)
at org.apache.jsp.admin.layout.template_jsp._jspx_meth_tiles_insertAttribute_7(template_jsp.java:539)
at org.apache.jsp.admin.layout.template_jsp._jspService(template_jsp.java:290)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:332)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:314)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:264)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.wso2.carbon.ui.JspServlet.service(JspServlet.java:161)
at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:80)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:36)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:67)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:690)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:477)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:402)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:329)
at org.eclipse.equinox.http.servlet.internal.RequestDispatcherAdaptor.forward(RequestDispatcherAdaptor.java:30)
at org.eclipse.equinox.http.helper.ContextPathServletAdaptor$RequestDispatcherAdaptor.forward(ContextPathServletAdaptor.java:258)
at org.apache.tiles.servlet.context.ServletTilesRequestContext.forward(ServletTilesRequestContext.java:198)
at org.apache.tiles.servlet.context.ServletTilesRequestContext.dispatch(ServletTilesRequestContext.java:185)
at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:419)
at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:370)
at org.wso2.carbon.ui.action.ActionHelper.render(ActionHelper.java:52)
at org.wso2.carbon.ui.TilesJspServlet.service(TilesJspServlet.java:101)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.eclipse.equinox.http.helper.ContextPathServletAdaptor.service(ContextPathServletAdaptor.java:36)
at org.eclipse.equinox.http.servlet.internal.ServletRegistration.handleRequest(ServletRegistration.java:90)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:111)
at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:67)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
at org.wso2.carbon.tomcat.ext.servlet.DelegationServlet.service(DelegationServlet.java:68)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.wso2.carbon.tomcat.ext.filter.CharacterSetFilter.doFilter(CharacterSetFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:172)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)