Re: Users - XML data import fails when importing more than ~10000 products

Posted by Nick Olson on
URL: http://ofbiz.116.s1.nabble.com/Users-XML-data-import-fails-when-importing-more-than-10000-products-tp140028p140033.html

Ok, I figured out how to get it to work. The Geronimo transaction was
timing out, and that TRANSACTION_TIMEOUT parameter doesn't seem to
effect that. I went into the GeronimoTransactionFactory.java file and
changed the default transaction timeout to 30 min and it imported the
data without any problems. There is probably a more elegant way to fix
this, but it seems to be working for now.

-Nick

BJ Freeman wrote:

> I know command line would not work for me.
> I import from emails (CSV, Tab, XML) websites (Txt using | delimiter,
> XmL) FTP sites(zip, CSV, XML).
> Some file are 8-100mb
> These are done daily sometimes every four hours.
>
> thanks for the tip where to look will see what I can do, if someone does
> not pick up on it before then.
>
>
> Not sure on second thought that widgets would effect a service, but will
> look anyway.
>
>
> Jacopo Cappellato sent the following on 6/20/06 9:44 PM:
>> Nick, all,
>>
>> I probably know the cause of this issue (but maybe not how to resolve
>> this): this issue has been introduced when that import page has been
>> converted to from jsp to screen widget; in fact 2 minutes is the default
>> transaction timeout set for screens.
>> In the "EntityImport" screen definition there is the following entry:
>>
>> <set field="parameters.TRANSACTION_TIMEOUT" value="7200"/>
>>
>> that should set the transaction timeout to 2 hours; if this doesn't work
>> it is probably too late to set the timeout there (or the timeout is
>> overriden by one of the screen's decorator)... any hints?
>>
>> Jacopo
>>
>> PS: Nick, you can always import your products using the command line:
>>
>> http://ofbizwiki.go-integral.com/Wiki.jsp?page=SeedData
>>
>>
>>
>> Nick Olson wrote:
>>
>>> It takes about 2 minutes before it comes back with this error. I've
>>> tried increasing the timeout in the services.xml file, but it doesn't
>>> seem to make a difference.
>>>
>>> David Welton wrote:
>>>
>>>> On 6/20/06, Nick Olson <[hidden email]> wrote:
>>>>
>>>>> I am trying to import 40k products into the database (I'm running
>>>>> postgres), but when I try to import, I get an error message that says:
>>>>> ERROR parsing Entity Xml file: Commit transaction failed.
>>>> How long is it taking?  No idea if this is the cause, but there is a
>>>> transaction timeout.
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>>
>>
>>  
>> _______________________________________________
>> Users mailing list
>> [hidden email]
>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users
>
>
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users