Demos: disk full

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Demos: disk full

Jacques Le Roux
Administrator
Hi,

This night I tried to create a log from the cron I run everyday at 3 AM to see if demos need update (based on svn)
But for a reason I got a disk full (the log was 47GB on 51GB available on the VM). I got rid of the file but I can't recover the VM

I will ask infra, in the meantime the demos will be unavailable

Jacques

Reply | Threaded
Open this post in threaded view
|

Re: Demos: disk full

Jacques Le Roux
Administrator
Trunk demo is back, the others will recover soon.
We have an issue (maybe 2) with stable ecommerce demodata loading, I will look at it...

Jacques

Le 03/07/2014 08:57, Jacques Le Roux a écrit :

> Hi,
>
> This night I tried to create a log from the cron I run everyday at 3 AM to see if demos need update (based on svn)
> But for a reason I got a disk full (the log was 47GB on 51GB available on the VM). I got rid of the file but I can't recover the VM
>
> I will ask infra, in the meantime the demos will be unavailable
>
> Jacques
>
>

--
Reply | Threaded
Open this post in threaded view
|

Re: Demos: disk full

Jacques Le Roux
Administrator
We got this error on R12.04 which saturated the disk again

2014-07-03 10:22:59,589 (org.ofbiz.service.job.JobPoller@2078b6ef) [   GenericDelegator.java:1296:ERROR]
---- exception report ----------------------------------------------------------
Failure in storeByCondition operation for entity [JobSandbox]: org.ofbiz.entity.GenericDataSourceException: Generic Entity Exception occured in
updateByCondition (SQL Exception while executing the following:UPDATE OFBIZ.JOB_SANDBOX SET RUN_BY_INSTANCE_ID = ?, STATUS_ID = ? WHERE ((RUN_TIME <=
? AND START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL) AND (POOL_ID IS NULL OR POOL_ID = ?)) (Container was opened
in read-only mode.)). Rolling back transaction.
Exception: org.ofbiz.entity.GenericDataSourceException
Message: Generic Entity Exception occured in updateByCondition (SQL Exception while executing the following:UPDATE OFBIZ.JOB_SANDBOX SET
RUN_BY_INSTANCE_ID = ?, STATUS_ID = ? WHERE ((RUN_TIME <= ? AND START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL)
AND (POOL_ID IS NULL OR POOL_ID = ?)) (Container was opened in read-only mode.))
---- cause ---------------------------------------------------------------------
Exception: org.ofbiz.entity.GenericDataSourceException
Message: SQL Exception while executing the following:UPDATE OFBIZ.JOB_SANDBOX SET RUN_BY_INSTANCE_ID = ?, STATUS_ID = ? WHERE ((RUN_TIME <= ? AND
START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL) AND (POOL_ID IS NULL OR POOL_ID = ?)) (Container was opened in
read-only mode.)
---- cause ---------------------------------------------------------------------
Exception: java.sql.SQLTransactionRollbackException
Message: Container was opened in read-only mode.
---- cause ---------------------------------------------------------------------
Exception: org.apache.derby.impl.jdbc.EmbedSQLException
Message: Container was opened in read-only mode.
---- cause ---------------------------------------------------------------------
Exception: org.apache.derby.iapi.error.StandardException
Message: Container was opened in read-only mode.
---- stack trace ---------------------------------------------------------------
ERROR 40XD1: Container was opened in read-only mode.

It normally relates to https://issues.apache.org/jira/browse/DERBY-541 but we use derby-10.8.2.2.jar in R12.04

Let's see if this happens again... All is restarting...

Jacques

Le 03/07/2014 11:05, Jacques Le Roux a écrit :

> Trunk demo is back, the others will recover soon.
> We have an issue (maybe 2) with stable ecommerce demodata loading, I will look at it...
>
> Jacques
>
> Le 03/07/2014 08:57, Jacques Le Roux a écrit :
>> Hi,
>>
>> This night I tried to create a log from the cron I run everyday at 3 AM to see if demos need update (based on svn)
>> But for a reason I got a disk full (the log was 47GB on 51GB available on the VM). I got rid of the file but I can't recover the VM
>>
>> I will ask infra, in the meantime the demos will be unavailable
>>
>> Jacques
>>
>>
>

--
Reply | Threaded
Open this post in threaded view
|

Re: Demos: disk full

Jacques Le Roux
Administrator
Seems more a permission issue, I will check those...

Jacques

Le 03/07/2014 17:50, Jacques Le Roux a écrit :

> We got this error on R12.04 which saturated the disk again
>
> 2014-07-03 10:22:59,589 (org.ofbiz.service.job.JobPoller@2078b6ef) [   GenericDelegator.java:1296:ERROR]
> ---- exception report ----------------------------------------------------------
> Failure in storeByCondition operation for entity [JobSandbox]: org.ofbiz.entity.GenericDataSourceException: Generic Entity Exception occured in
> updateByCondition (SQL Exception while executing the following:UPDATE OFBIZ.JOB_SANDBOX SET RUN_BY_INSTANCE_ID = ?, STATUS_ID = ? WHERE ((RUN_TIME
> <= ? AND START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL) AND (POOL_ID IS NULL OR POOL_ID = ?)) (Container was
> opened in read-only mode.)). Rolling back transaction.
> Exception: org.ofbiz.entity.GenericDataSourceException
> Message: Generic Entity Exception occured in updateByCondition (SQL Exception while executing the following:UPDATE OFBIZ.JOB_SANDBOX SET
> RUN_BY_INSTANCE_ID = ?, STATUS_ID = ? WHERE ((RUN_TIME <= ? AND START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL)
> AND (POOL_ID IS NULL OR POOL_ID = ?)) (Container was opened in read-only mode.))
> ---- cause ---------------------------------------------------------------------
> Exception: org.ofbiz.entity.GenericDataSourceException
> Message: SQL Exception while executing the following:UPDATE OFBIZ.JOB_SANDBOX SET RUN_BY_INSTANCE_ID = ?, STATUS_ID = ? WHERE ((RUN_TIME <= ? AND
> START_DATE_TIME IS NULL AND CANCEL_DATE_TIME IS NULL AND RUN_BY_INSTANCE_ID IS NULL) AND (POOL_ID IS NULL OR POOL_ID = ?)) (Container was opened in
> read-only mode.)
> ---- cause ---------------------------------------------------------------------
> Exception: java.sql.SQLTransactionRollbackException
> Message: Container was opened in read-only mode.
> ---- cause ---------------------------------------------------------------------
> Exception: org.apache.derby.impl.jdbc.EmbedSQLException
> Message: Container was opened in read-only mode.
> ---- cause ---------------------------------------------------------------------
> Exception: org.apache.derby.iapi.error.StandardException
> Message: Container was opened in read-only mode.
> ---- stack trace ---------------------------------------------------------------
> ERROR 40XD1: Container was opened in read-only mode.
>
> It normally relates to https://issues.apache.org/jira/browse/DERBY-541 but we use derby-10.8.2.2.jar in R12.04
>
> Let's see if this happens again... All is restarting...
>
> Jacques
>
> Le 03/07/2014 11:05, Jacques Le Roux a écrit :
>> Trunk demo is back, the others will recover soon.
>> We have an issue (maybe 2) with stable ecommerce demodata loading, I will look at it...
>>
>> Jacques
>>
>> Le 03/07/2014 08:57, Jacques Le Roux a écrit :
>>> Hi,
>>>
>>> This night I tried to create a log from the cron I run everyday at 3 AM to see if demos need update (based on svn)
>>> But for a reason I got a disk full (the log was 47GB on 51GB available on the VM). I got rid of the file but I can't recover the VM
>>>
>>> I will ask infra, in the meantime the demos will be unavailable
>>>
>>> Jacques
>>>
>>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Demos: disk full

Pierre Smits
Jacques,

It seems to me that 51 GB would be sufficient as a capacity for log storage
and data for many days before reaching limits.

But, is extensive logging necessary for the demo environment(s)? Shouldn't
clean-up be executed with each daily restart?

Regards,

Pierre Smits

*ORRTIZ.COM <http://www.orrtiz.com>*
Services & Solutions for Cloud-
Based Manufacturing, Professional
Services and Retail & Trade
http://www.orrtiz.com
Reply | Threaded
Open this post in threaded view
|

Re: Demos: disk full

Jacques Le Roux
Administrator
In reply to this post by Jacques Le Roux

Le 03/07/2014 11:05, Jacques Le Roux a écrit :
> We have an issue (maybe 2) with stable ecommerce demodata loading, I will look at it...
>

It was an incompatibility in Lucene component, I did not load it in stable (R13.07) for now...

[java] 2014-07-02 20:24:39,766 (main) [ ServiceDispatcher.java:512:INFO ] [[Sync service failed...- total:0.0,since last(Begin):0.0]] -
'entity-default / indexProduct'
      [java] 2014-07-02 20:24:39,768 (main) [ TransactionUtil.java:378:WARN ]
      [java] ---- exception report ----------------------------------------------------------
      [java] [TransactionUtil.setRollbackOnly] Calling transaction setRollbackOnly; this stack trace shows where this is happening:
      [java] Exception: java.lang.Exception
      [java] Message: Service [indexProduct] threw an unexpected exception/error
      [java] ---- stack trace ---------------------------------------------------------------
      [java] java.lang.Exception: Service [indexProduct] threw an unexpected exception/error
      [java] org.ofbiz.entity.transaction.TransactionUtil.setRollbackOnly(TransactionUtil.java:378)
      [java] org.ofbiz.entity.transaction.TransactionUtil.rollback(TransactionUtil.java:319)
      [java] org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:517)
      [java] org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:232)
      [java] org.ofbiz.service.GenericDispatcherFactory$GenericDispatcher.runSync(GenericDispatcherFactory.java:83)
      [java] org.ofbiz.entityext.eca.EntityEcaAction.runAction(EntityEcaAction.java:92)
      [java] org.ofbiz.entityext.eca.EntityEcaRule.eval(EntityEcaRule.java:142)
      [java] org.ofbiz.entityext.eca.DelegatorEcaHandler.evalRules(DelegatorEcaHandler.java:83)
      [java] org.ofbiz.entity.GenericDelegator$EntityEcaRuleRunner.evalRules(GenericDelegator.java:2276)
      [java] org.ofbiz.entity.GenericDelegator.create(GenericDelegator.java:871)
      [java] org.ofbiz.entity.GenericDelegator.storeAll(GenericDelegator.java:1355)
      [java] org.ofbiz.entity.util.EntitySaxReader.writeValues(EntitySaxReader.java:326)
      [java] org.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:296)
      [java] org.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:245)
      [java] org.ofbiz.entity.util.EntityDataLoader.loadData(EntityDataLoader.java:257)
      [java] org.ofbiz.entityext.data.EntityDataLoadContainer.loadContainer(EntityDataLoadContainer.java:492)
      [java] org.ofbiz.entityext.data.EntityDataLoadContainer.start(EntityDataLoadContainer.java:238)
      [java] org.ofbiz.base.container.ContainerLoader.start(ContainerLoader.java:238)
      [java] org.ofbiz.base.start.Start.startStartLoaders(Start.java:340)
      [java] org.ofbiz.base.start.Start.start(Start.java:382)
      [java] org.ofbiz.base.start.Start.main(Start.java:122)
      [java] --------------------------------------------------------------------------------
      [java]
      [java] 2014-07-02 20:24:39,769 (main) [ EntityEcaAction.java:114:ERROR]
      [java] ---- exception report ----------------------------------------------------------
      [java] Error running Entity ECA action service
      [java] Exception: org.ofbiz.service.GenericServiceException
      [java] Message: Service [indexProduct] target threw an unexpected exception (org/apache/lucene/index/CorruptIndexException : Unsupported
major.minor version 51.0)

Jacques
Reply | Threaded
Open this post in threaded view
|

Re: Demos: disk full

Jacques Le Roux
Administrator
OK this is resolved, all demos are now running under Java 7, so Lucene works also with R13.07 (with svn-external from trunk)

A demo is possibly reset at 3AM GTM if a commit has been done since last check. So trunk will be reset more often than releases branches.

Jacques


Le 03/07/2014 20:02, Jacques Le Roux a écrit :

>
> Le 03/07/2014 11:05, Jacques Le Roux a écrit :
>> We have an issue (maybe 2) with stable ecommerce demodata loading, I will look at it...
>>
>
> It was an incompatibility in Lucene component, I did not load it in stable (R13.07) for now...
>
> [java] 2014-07-02 20:24:39,766 (main) [ ServiceDispatcher.java:512:INFO ] [[Sync service failed...- total:0.0,since last(Begin):0.0]] -
> 'entity-default / indexProduct'
>      [java] 2014-07-02 20:24:39,768 (main) [ TransactionUtil.java:378:WARN ]
>      [java] ---- exception report ----------------------------------------------------------
>      [java] [TransactionUtil.setRollbackOnly] Calling transaction setRollbackOnly; this stack trace shows where this is happening:
>      [java] Exception: java.lang.Exception
>      [java] Message: Service [indexProduct] threw an unexpected exception/error
>      [java] ---- stack trace ---------------------------------------------------------------
>      [java] java.lang.Exception: Service [indexProduct] threw an unexpected exception/error
>      [java] org.ofbiz.entity.transaction.TransactionUtil.setRollbackOnly(TransactionUtil.java:378)
>      [java] org.ofbiz.entity.transaction.TransactionUtil.rollback(TransactionUtil.java:319)
>      [java] org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:517)
>      [java] org.ofbiz.service.ServiceDispatcher.runSync(ServiceDispatcher.java:232)
>      [java] org.ofbiz.service.GenericDispatcherFactory$GenericDispatcher.runSync(GenericDispatcherFactory.java:83)
>      [java] org.ofbiz.entityext.eca.EntityEcaAction.runAction(EntityEcaAction.java:92)
>      [java] org.ofbiz.entityext.eca.EntityEcaRule.eval(EntityEcaRule.java:142)
>      [java] org.ofbiz.entityext.eca.DelegatorEcaHandler.evalRules(DelegatorEcaHandler.java:83)
>      [java] org.ofbiz.entity.GenericDelegator$EntityEcaRuleRunner.evalRules(GenericDelegator.java:2276)
>      [java] org.ofbiz.entity.GenericDelegator.create(GenericDelegator.java:871)
>      [java] org.ofbiz.entity.GenericDelegator.storeAll(GenericDelegator.java:1355)
>      [java] org.ofbiz.entity.util.EntitySaxReader.writeValues(EntitySaxReader.java:326)
>      [java] org.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:296)
>      [java] org.ofbiz.entity.util.EntitySaxReader.parse(EntitySaxReader.java:245)
>      [java] org.ofbiz.entity.util.EntityDataLoader.loadData(EntityDataLoader.java:257)
>      [java] org.ofbiz.entityext.data.EntityDataLoadContainer.loadContainer(EntityDataLoadContainer.java:492)
>      [java] org.ofbiz.entityext.data.EntityDataLoadContainer.start(EntityDataLoadContainer.java:238)
>      [java] org.ofbiz.base.container.ContainerLoader.start(ContainerLoader.java:238)
>      [java] org.ofbiz.base.start.Start.startStartLoaders(Start.java:340)
>      [java] org.ofbiz.base.start.Start.start(Start.java:382)
>      [java] org.ofbiz.base.start.Start.main(Start.java:122)
>      [java] --------------------------------------------------------------------------------
>      [java]
>      [java] 2014-07-02 20:24:39,769 (main) [ EntityEcaAction.java:114:ERROR]
>      [java] ---- exception report ----------------------------------------------------------
>      [java] Error running Entity ECA action service
>      [java] Exception: org.ofbiz.service.GenericServiceException
>      [java] Message: Service [indexProduct] target threw an unexpected exception (org/apache/lucene/index/CorruptIndexException : Unsupported
> major.minor version 51.0)
>
> Jacques
>

--