Re: Users - StoreAll crash on big import files ?

Posted by Jacopo Cappellato on Mar 30, 2006; 2:23pm
URL: http://ofbiz.116.s1.nabble.com/Users-StoreAll-crash-on-big-import-files-tp138244p138253.html

Hi Tibor,

I don't think that the problem is the size of your data (100000 are not
so many...) so my guess is that there are some problem in the data itself...

Jacopo


tibor katelbach wrote:

> thanks for the quick answer
>
> we are not using directly the webtools but the same methods as in the
> import api.
> we are on Sequoia 0.8,
> it's quite hard to say if the time before failure is constant.
> BJ how would you increase the streams memory ? this seems be a good
> solution.
> is there a limit to this stream  memory size, cause ultematly we will
> have an xml file with 100000 insert tags. Right now we are still testing
> with 10000 ?
>
> offfcourse the cut up will be our last way to go.
> Thanks a lot
>
>
> On 3/30/06, *BJ Freeman* <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     not sure what version you have, or if they have changed the import since
>     5,000 reves, but the import is a stream, therefore memory.
>     you can try increasing the memory, or cutting up the import.
>
>
>     tibor katelbach sent the following on 3/30/06 5:48 AM:
>      > Hi everyone
>      > we have managed to import dynamically small xml files coming from
>     a CSV then
>      > transformed to a valid entity engine XML format. That worked
>     great. We then
>      > tested with big files and unfortunetly it crashed on us ? has
>     anobody
>      > noticed this ?
>      > The crash seems to be on the StoreAll.
>      > is there a limit to the size of files that can be imported ?
>      >
>      >
>      > regards
>      > Tibor
>      >
>      >
>      >
>      >
>     ------------------------------------------------------------------------
>
>      >
>      >
>      > _______________________________________________
>      > Users mailing list
>      > [hidden email] <mailto:[hidden email]>
>      > http://lists.ofbiz.org/mailman/listinfo/users
>
>     _______________________________________________
>     Users mailing list
>     [hidden email] <mailto:[hidden email]>
>     http://lists.ofbiz.org/mailman/listinfo/users
>
>
>
> ------------------------------------------------------------------------
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users