Users - XML data import fails when importing more than ~10000 products

[Nick Olson-2]

I am trying to import 40k products into the database (I'm running
postgres), but when I try to import, I get an error message that says:
ERROR parsing Entity Xml file: Commit transaction failed.

This appears in the log file:

78565324 (http-0.0.0.0-8443-Processor2) [
EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
to 16000
78570220 (http-0.0.0.0-8443-Processor2) [
EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
to 17000
78575552 (http-0.0.0.0-8443-Processor2) [
EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
to 18000
78580442 (http-0.0.0.0-8443-Processor2) [
EntitySaxReader.java:261:INFO ] Finished 18999 values from
file:/home/nick/ofbiz-7818/import/demo_products.xml
78580444 (http-0.0.0.0-8443-Processor2)
[InheritableTransactionContext.java:311:ERROR] Unable to roll back
transaction
java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
        at org.apache.geronimo.transaction.manager.TransactionImpl.rollback(TransactionImpl.java:438)
        at org.apache.geronimo.transaction.context.InheritableTransactionContext.rollbackAndThrow(InheritableTransactionContext.java:308)
        at org.apache.geronimo.transaction.context.InheritableTransactionContext.complete(InheritableTransactionContext.java:199)
        at org.apache.geronimo.transaction.context.InheritableTransactionContext.commit(InheritableTransactionContext.java:146)


I get the exact same error when I try to use a derby database instead
of the postgres one. This import was working when I was using an
earlier build of ofbiz (1-2 months ago), but it is failing now.

What could be causing this error?

-Nick
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[davidnwelton]

On 6/20/06, Nick Olson <[hidden email]> wrote:
> I am trying to import 40k products into the database (I'm running
> postgres), but when I try to import, I get an error message that says:
> ERROR parsing Entity Xml file: Commit transaction failed.

How long is it taking?  No idea if this is the cause, but there is a
transaction timeout.

--
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Walter Vaughan]

Nick Olson wrote:

> I am trying to import 40k products into the database (I'm running
> postgres), but when I try to import, I get an error message that says:
> ERROR parsing Entity Xml file: Commit transaction failed.
>
> This appears in the log file:
>
> 78565324 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
> to 16000
> 78570220 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
> to 17000
> 78575552 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
> to 18000
> 78580442 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:261:INFO ] Finished 18999 values from
> file:/home/nick/ofbiz-7818/import/demo_products.xml
> 78580444 (http-0.0.0.0-8443-Processor2)
> [InheritableTransactionContext.java:311:ERROR] Unable to roll back
> transaction
> java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
> at org.apache.geronimo.transaction.manager.TransactionImpl.rollback(TransactionImpl.java:438)
> at org.apache.geronimo.transaction.context.InheritableTransactionContext.rollbackAndThrow(InheritableTransactionContext.java:308)
> at org.apache.geronimo.transaction.context.InheritableTransactionContext.complete(InheritableTransactionContext.java:199)
> at org.apache.geronimo.transaction.context.InheritableTransactionContext.commit(InheritableTransactionContext.java:146)
>
>
> I get the exact same error when I try to use a derby database instead
> of the postgres one. This import was working when I was using an
> earlier build of ofbiz (1-2 months ago), but it is failing now.
>
> What could be causing this error?

We've discovered that using "xmllint" is an excellent tool to make sure the xml
is actually clean enough to use. Does xmllint pass the file cleanly?

http://xmlsoft.org/xmllint.html

You may already have the program on your server.

It installs in freebsd as
cd /usr/ports/textproc/libxml2/ && make install clean


 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[BJ Freeman]

Possibly your running in to memory limits.
try increasing the memory in the startofbiz.bat or sh
if you upgraded there a new things that take up memory.


Nick Olson sent the following on 6/20/06 1:11 PM:

> I am trying to import 40k products into the database (I'm running
> postgres), but when I try to import, I get an error message that says:
> ERROR parsing Entity Xml file: Commit transaction failed.
>
> This appears in the log file:
>
> 78565324 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
> to 16000
> 78570220 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
> to 17000
> 78575552 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
> to 18000
> 78580442 (http-0.0.0.0-8443-Processor2) [
> EntitySaxReader.java:261:INFO ] Finished 18999 values from
> file:/home/nick/ofbiz-7818/import/demo_products.xml
> 78580444 (http-0.0.0.0-8443-Processor2)
> [InheritableTransactionContext.java:311:ERROR] Unable to roll back
> transaction
> java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
> at org.apache.geronimo.transaction.manager.TransactionImpl.rollback(TransactionImpl.java:438)
> at org.apache.geronimo.transaction.context.InheritableTransactionContext.rollbackAndThrow(InheritableTransactionContext.java:308)
> at org.apache.geronimo.transaction.context.InheritableTransactionContext.complete(InheritableTransactionContext.java:199)
> at org.apache.geronimo.transaction.context.InheritableTransactionContext.commit(InheritableTransactionContext.java:146)
>
>
> I get the exact same error when I try to use a derby database instead
> of the postgres one. This import was working when I was using an
> earlier build of ofbiz (1-2 months ago), but it is failing now.
>
> What could be causing this error?
>
> -Nick
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Nick Olson]

It takes about 2 minutes before it comes back with this error. I've
tried increasing the timeout in the services.xml file, but it doesn't
seem to make a difference.

David Welton wrote:
> On 6/20/06, Nick Olson <[hidden email]> wrote:
>> I am trying to import 40k products into the database (I'm running
>> postgres), but when I try to import, I get an error message that says:
>> ERROR parsing Entity Xml file: Commit transaction failed.
>
> How long is it taking?  No idea if this is the cause, but there is a
> transaction timeout.
>
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Nick Olson]

I thought the xml might be invalid, so I generated a test xml file with
some dummy information, like this:

    <Product productId='1'
    productTypeId='GOOD'
    productName='1'
    internalName='1'
    longDescription='1'
    taxable='Y'
    weight='1'
    largeImageUrl='1'
    chargeShipping='Y'
    autoCreateKeywords='N'
    isVirtual='N'
    isVariant='N'
    createdDate='2006-06-20 12:43:25'
    createdByUserLogin='admin'
    lastModifiedDate='2006-06-20 12:43:25'
    lastModifiedByUserLogin='admin'/>


    <Product productId='2'
    productTypeId='GOOD'
    productName='2'
    internalName='2'
    longDescription='2'
    taxable='Y'
    weight='2'
    largeImageUrl='2'
    chargeShipping='Y'
    autoCreateKeywords='N'
    isVirtual='N'
    isVariant='N'
    createdDate='2006-06-20 12:43:25'
    createdByUserLogin='admin'
    lastModifiedDate='2006-06-20 12:43:25'
    lastModifiedByUserLogin='admin'/>
...


Still, I get the same errors.

Walter Vaughan wrote:

> Nick Olson wrote:
>
>> I am trying to import 40k products into the database (I'm running
>> postgres), but when I try to import, I get an error message that says:
>> ERROR parsing Entity Xml file: Commit transaction failed.
>>
>> This appears in the log file:
>>
>> 78565324 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>> to 16000
>> 78570220 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>> to 17000
>> 78575552 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>> to 18000
>> 78580442 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:261:INFO ] Finished 18999 values from
>> file:/home/nick/ofbiz-7818/import/demo_products.xml
>> 78580444 (http-0.0.0.0-8443-Processor2)
>> [InheritableTransactionContext.java:311:ERROR] Unable to roll back
>> transaction
>> java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
>> at org.apache.geronimo.transaction.manager.TransactionImpl.rollback(TransactionImpl.java:438)
>> at org.apache.geronimo.transaction.context.InheritableTransactionContext.rollbackAndThrow(InheritableTransactionContext.java:308)
>> at org.apache.geronimo.transaction.context.InheritableTransactionContext.complete(InheritableTransactionContext.java:199)
>> at org.apache.geronimo.transaction.context.InheritableTransactionContext.commit(InheritableTransactionContext.java:146)
>>
>>
>> I get the exact same error when I try to use a derby database instead
>> of the postgres one. This import was working when I was using an
>> earlier build of ofbiz (1-2 months ago), but it is failing now.
>>
>> What could be causing this error?
>
> We've discovered that using "xmllint" is an excellent tool to make sure the xml
> is actually clean enough to use. Does xmllint pass the file cleanly?
>
> http://xmlsoft.org/xmllint.html
>
> You may already have the program on your server.
>
> It installs in freebsd as
> cd /usr/ports/textproc/libxml2/ && make install clean
>
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[davidnwelton]

> It takes about 2 minutes before it comes back with this error. I've
> tried increasing the timeout in the services.xml file, but it doesn't
> seem to make a difference.

If I may enquire...what sort of hardware is that on?  We have a lot of
products, which, at 10000 a minute, would put us at about 100
minutes:-/  Don't really have other ideas about what might be going
wrong, sorry.

--
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Nick Olson]

This is on a 2.8Ghz Xeon machine, 2 gigs RAM, running Gentoo Linux.

David Welton wrote:
>> It takes about 2 minutes before it comes back with this error. I've
>> tried increasing the timeout in the services.xml file, but it doesn't
>> seem to make a difference.
>
> If I may enquire...what sort of hardware is that on?  We have a lot of
> products, which, at 10000 a minute, would put us at about 100
> minutes:-/  Don't really have other ideas about what might be going
> wrong, sorry.
>
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Nick Olson]

Tried bumping the memory up to 1 gig minimum, but I'm getting the same
errors.

BJ Freeman wrote:

> Possibly your running in to memory limits.
> try increasing the memory in the startofbiz.bat or sh
> if you upgraded there a new things that take up memory.
>
>
> Nick Olson sent the following on 6/20/06 1:11 PM:
>> I am trying to import 40k products into the database (I'm running
>> postgres), but when I try to import, I get an error message that says:
>> ERROR parsing Entity Xml file: Commit transaction failed.
>>
>> This appears in the log file:
>>
>> 78565324 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>> to 16000
>> 78570220 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>> to 17000
>> 78575552 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>> to 18000
>> 78580442 (http-0.0.0.0-8443-Processor2) [
>> EntitySaxReader.java:261:INFO ] Finished 18999 values from
>> file:/home/nick/ofbiz-7818/import/demo_products.xml
>> 78580444 (http-0.0.0.0-8443-Processor2)
>> [InheritableTransactionContext.java:311:ERROR] Unable to roll back
>> transaction
>> java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
>> at org.apache.geronimo.transaction.manager.TransactionImpl.rollback(TransactionImpl.java:438)
>> at org.apache.geronimo.transaction.context.InheritableTransactionContext.rollbackAndThrow(InheritableTransactionContext.java:308)
>> at org.apache.geronimo.transaction.context.InheritableTransactionContext.complete(InheritableTransactionContext.java:199)
>> at org.apache.geronimo.transaction.context.InheritableTransactionContext.commit(InheritableTransactionContext.java:146)
>>
>>
>> I get the exact same error when I try to use a derby database instead
>> of the postgres one. This import was working when I was using an
>> earlier build of ofbiz (1-2 months ago), but it is failing now.
>>
>> What could be causing this error?
>>
>> -Nick
>>  
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Walter Vaughan]

Nick Olson wrote:
> I thought the xml might be invalid, so I generated a test xml file with
> some dummy information, like this:
[snip]
> Still, I get the same errors.

I can import the following no problem with this past Friday's SVN.

<entity-engine-xml>
<Product productId='1'
     productTypeId='GOOD'
     productName='1'
     internalName='1'
     longDescription='1'
     taxable='Y'
     weight='1'
     largeImageUrl='1'
     chargeShipping='Y'
     autoCreateKeywords='N'
     isVirtual='N'
     isVariant='N'
     createdDate='2006-06-20 12:43:25'
     createdByUserLogin='admin'
     lastModifiedDate='2006-06-20 12:43:25'
     lastModifiedByUserLogin='admin'/>
</entity-engine-xml>

I have discovered that I have to blow out the database, recreate it, do an "ant
run-install-seed", and then re-load stored entity-engine-xml files whenever I
have any problem with the product file.
YMMV.

--
Walter
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Si Chen-2]

Are you saying this worked with an earlier version of OFBIZ?  How  
long ago?  This may have to do with the new Geronimo transactions  
managers.

Maybe you should open a JIRA issue and attach the entire log (yes,  
all of it -- don't be shy... Sometimes the clues are not obvious.)

Si


On Jun 20, 2006, at 2:12 PM, Nick Olson wrote:

> Tried bumping the memory up to 1 gig minimum, but I'm getting the same
> errors.
>
> BJ Freeman wrote:
>> Possibly your running in to memory limits.
>> try increasing the memory in the startofbiz.bat or sh
>> if you upgraded there a new things that take up memory.
>>
>>
>> Nick Olson sent the following on 6/20/06 1:11 PM:
>>> I am trying to import 40k products into the database (I'm running
>>> postgres), but when I try to import, I get an error message that  
>>> says:
>>> ERROR parsing Entity Xml file: Commit transaction failed.
>>>
>>> This appears in the log file:
>>>
>>> 78565324 (http-0.0.0.0-8443-Processor2) [
>>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>>> to 16000
>>> 78570220 (http-0.0.0.0-8443-Processor2) [
>>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>>> to 17000
>>> 78575552 (http-0.0.0.0-8443-Processor2) [
>>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>>> to 18000
>>> 78580442 (http-0.0.0.0-8443-Processor2) [
>>> EntitySaxReader.java:261:INFO ] Finished 18999 values from
>>> file:/home/nick/ofbiz-7818/import/demo_products.xml
>>> 78580444 (http-0.0.0.0-8443-Processor2)
>>> [InheritableTransactionContext.java:311:ERROR] Unable to roll back
>>> transaction
>>> java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
>>> at  
>>> org.apache.geronimo.transaction.manager.TransactionImpl.rollback
>>> (TransactionImpl.java:438)
>>> at  
>>> org.apache.geronimo.transaction.context.InheritableTransactionContex
>>> t.rollbackAndThrow(InheritableTransactionContext.java:308)
>>> at  
>>> org.apache.geronimo.transaction.context.InheritableTransactionContex
>>> t.complete(InheritableTransactionContext.java:199)
>>> at  
>>> org.apache.geronimo.transaction.context.InheritableTransactionContex
>>> t.commit(InheritableTransactionContext.java:146)
>>>
>>>
>>> I get the exact same error when I try to use a derby database  
>>> instead
>>> of the postgres one. This import was working when I was using an
>>> earlier build of ofbiz (1-2 months ago), but it is failing now.
>>>
>>> What could be causing this error?
>>>
>>> -Nick
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Nick Olson]

Ok, I created an issue in JIRA. The full log is attached.

https://issues.apache.org/jira/browse/OFBIZ-37


Si Chen wrote:

> Are you saying this worked with an earlier version of OFBIZ?  How  
> long ago?  This may have to do with the new Geronimo transactions  
> managers.
>
> Maybe you should open a JIRA issue and attach the entire log (yes,  
> all of it -- don't be shy... Sometimes the clues are not obvious.)
>
> Si
>
>
> On Jun 20, 2006, at 2:12 PM, Nick Olson wrote:
>
>> Tried bumping the memory up to 1 gig minimum, but I'm getting the same
>> errors.
>>
>> BJ Freeman wrote:
>>> Possibly your running in to memory limits.
>>> try increasing the memory in the startofbiz.bat or sh
>>> if you upgraded there a new things that take up memory.
>>>
>>>
>>> Nick Olson sent the following on 6/20/06 1:11 PM:
>>>> I am trying to import 40k products into the database (I'm running
>>>> postgres), but when I try to import, I get an error message that  
>>>> says:
>>>> ERROR parsing Entity Xml file: Commit transaction failed.
>>>>
>>>> This appears in the log file:
>>>>
>>>> 78565324 (http-0.0.0.0-8443-Processor2) [
>>>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>>>> to 16000
>>>> 78570220 (http-0.0.0.0-8443-Processor2) [
>>>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>>>> to 17000
>>>> 78575552 (http-0.0.0.0-8443-Processor2) [
>>>> EntitySaxReader.java:403:INFO ] Another 1000 values imported: now up
>>>> to 18000
>>>> 78580442 (http-0.0.0.0-8443-Processor2) [
>>>> EntitySaxReader.java:261:INFO ] Finished 18999 values from
>>>> file:/home/nick/ofbiz-7818/import/demo_products.xml
>>>> 78580444 (http-0.0.0.0-8443-Processor2)
>>>> [InheritableTransactionContext.java:311:ERROR] Unable to roll back
>>>> transaction
>>>> java.lang.IllegalStateException: Status is STATUS_NO_TRANSACTION
>>>> at  
>>>> org.apache.geronimo.transaction.manager.TransactionImpl.rollback
>>>> (TransactionImpl.java:438)
>>>> at  
>>>> org.apache.geronimo.transaction.context.InheritableTransactionContex
>>>> t.rollbackAndThrow(InheritableTransactionContext.java:308)
>>>> at  
>>>> org.apache.geronimo.transaction.context.InheritableTransactionContex
>>>> t.complete(InheritableTransactionContext.java:199)
>>>> at  
>>>> org.apache.geronimo.transaction.context.InheritableTransactionContex
>>>> t.commit(InheritableTransactionContext.java:146)
>>>>
>>>>
>>>> I get the exact same error when I try to use a derby database  
>>>> instead
>>>> of the postgres one. This import was working when I was using an
>>>> earlier build of ofbiz (1-2 months ago), but it is failing now.
>>>>
>>>> What could be causing this error?
>>>>
>>>> -Nick
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> [hidden email]
>>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>>
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Jacopo Cappellato]

Nick, all,

I probably know the cause of this issue (but maybe not how to resolve
this): this issue has been introduced when that import page has been
converted to from jsp to screen widget; in fact 2 minutes is the default
transaction timeout set for screens.
In the "EntityImport" screen definition there is the following entry:

<set field="parameters.TRANSACTION_TIMEOUT" value="7200"/>

that should set the transaction timeout to 2 hours; if this doesn't work
it is probably too late to set the timeout there (or the timeout is
overriden by one of the screen's decorator)... any hints?

Jacopo

PS: Nick, you can always import your products using the command line:

http://ofbizwiki.go-integral.com/Wiki.jsp?page=SeedData



Nick Olson wrote:

> It takes about 2 minutes before it comes back with this error. I've
> tried increasing the timeout in the services.xml file, but it doesn't
> seem to make a difference.
>
> David Welton wrote:
>> On 6/20/06, Nick Olson <[hidden email]> wrote:
>>> I am trying to import 40k products into the database (I'm running
>>> postgres), but when I try to import, I get an error message that says:
>>> ERROR parsing Entity Xml file: Commit transaction failed.
>> How long is it taking?  No idea if this is the cause, but there is a
>> transaction timeout.
>>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[BJ Freeman]

I know command line would not work for me.
I import from emails (CSV, Tab, XML) websites (Txt using | delimiter,
XmL) FTP sites(zip, CSV, XML).
Some file are 8-100mb
These are done daily sometimes every four hours.

thanks for the tip where to look will see what I can do, if someone does
not pick up on it before then.


Not sure on second thought that widgets would effect a service, but will
look anyway.


Jacopo Cappellato sent the following on 6/20/06 9:44 PM:

> Nick, all,
>
> I probably know the cause of this issue (but maybe not how to resolve
> this): this issue has been introduced when that import page has been
> converted to from jsp to screen widget; in fact 2 minutes is the default
> transaction timeout set for screens.
> In the "EntityImport" screen definition there is the following entry:
>
> <set field="parameters.TRANSACTION_TIMEOUT" value="7200"/>
>
> that should set the transaction timeout to 2 hours; if this doesn't work
> it is probably too late to set the timeout there (or the timeout is
> overriden by one of the screen's decorator)... any hints?
>
> Jacopo
>
> PS: Nick, you can always import your products using the command line:
>
> http://ofbizwiki.go-integral.com/Wiki.jsp?page=SeedData
>
>
>
> Nick Olson wrote:
>
>>It takes about 2 minutes before it comes back with this error. I've
>>tried increasing the timeout in the services.xml file, but it doesn't
>>seem to make a difference.
>>
>>David Welton wrote:
>>
>>>On 6/20/06, Nick Olson <[hidden email]> wrote:
>>>
>>>>I am trying to import 40k products into the database (I'm running
>>>>postgres), but when I try to import, I get an error message that says:
>>>>ERROR parsing Entity Xml file: Commit transaction failed.
>>>
>>>How long is it taking?  No idea if this is the cause, but there is a
>>>transaction timeout.
>>>
>>
>>
>>_______________________________________________
>>Users mailing list
>>[hidden email]
>>http://lists.ofbiz.org/mailman/listinfo/users
>>
>
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Nick Olson]

Ok, I figured out how to get it to work. The Geronimo transaction was
timing out, and that TRANSACTION_TIMEOUT parameter doesn't seem to
effect that. I went into the GeronimoTransactionFactory.java file and
changed the default transaction timeout to 30 min and it imported the
data without any problems. There is probably a more elegant way to fix
this, but it seems to be working for now.

-Nick

BJ Freeman wrote:

> I know command line would not work for me.
> I import from emails (CSV, Tab, XML) websites (Txt using | delimiter,
> XmL) FTP sites(zip, CSV, XML).
> Some file are 8-100mb
> These are done daily sometimes every four hours.
>
> thanks for the tip where to look will see what I can do, if someone does
> not pick up on it before then.
>
>
> Not sure on second thought that widgets would effect a service, but will
> look anyway.
>
>
> Jacopo Cappellato sent the following on 6/20/06 9:44 PM:
>> Nick, all,
>>
>> I probably know the cause of this issue (but maybe not how to resolve
>> this): this issue has been introduced when that import page has been
>> converted to from jsp to screen widget; in fact 2 minutes is the default
>> transaction timeout set for screens.
>> In the "EntityImport" screen definition there is the following entry:
>>
>> <set field="parameters.TRANSACTION_TIMEOUT" value="7200"/>
>>
>> that should set the transaction timeout to 2 hours; if this doesn't work
>> it is probably too late to set the timeout there (or the timeout is
>> overriden by one of the screen's decorator)... any hints?
>>
>> Jacopo
>>
>> PS: Nick, you can always import your products using the command line:
>>
>> http://ofbizwiki.go-integral.com/Wiki.jsp?page=SeedData
>>
>>
>>
>> Nick Olson wrote:
>>
>>> It takes about 2 minutes before it comes back with this error. I've
>>> tried increasing the timeout in the services.xml file, but it doesn't
>>> seem to make a difference.
>>>
>>> David Welton wrote:
>>>
>>>> On 6/20/06, Nick Olson <[hidden email]> wrote:
>>>>
>>>>> I am trying to import 40k products into the database (I'm running
>>>>> postgres), but when I try to import, I get an error message that says:
>>>>> ERROR parsing Entity Xml file: Commit transaction failed.
>>>> How long is it taking?  No idea if this is the cause, but there is a
>>>> transaction timeout.
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>
>>  
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>
>

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[davidnwelton]

> Ok, I figured out how to get it to work. The Geronimo transaction was
> timing out, and that TRANSACTION_TIMEOUT parameter doesn't seem to
> effect that. I went into the GeronimoTransactionFactory.java file and
> changed the default transaction timeout to 30 min and it imported the
> data without any problems. There is probably a more elegant way to fix
> this, but it seems to be working for now.

Perhaps it's something that's not quite integrated yet with the new
Geronimo stuff.  Thanks in any case for reporting it back!

--
David N. Welton
 - http://www.dedasys.com/davidw/

Linux, Open Source Consulting
 - http://www.dedasys.com/
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[rohit]

hi,

i got the same problem and could import just 100 products, when i adjusted the  TRANSACTION_TIMEOUT in GeronimoTransactionFactory.java.

rohit