[jira] Created: (OFBIZ-3558) Out of memory when importing large files

[Nicolas Malin (Jira)]

Out of memory when importing large files
----------------------------------------

                 Key: OFBIZ-3558
                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
             Project: OFBiz
          Issue Type: Bug
          Components: framework
    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
            Reporter: BJ Freeman
            Priority: Blocker
             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk


importing a tab delimited file 8megs in size

javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

n

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845618#action_12845618 ]

Adam Heath commented on OFBIZ-3558:
-----------------------------------

Are there lots of columns, and few lines, or lots of lines, and few columns?  this problem might be related to the number of lines, as it is probably trying to do it all in a single transaction, and the parsed strings end up sharing the raw character array data in some places, but duplicating it in others.

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

BJ Freeman updated OFBIZ-3558:
------------------------------

    Description:
importing a tab delimited file 8megs(discontinued products) in size

to me this is poor design since most file are even bigger when importing inventory from suppliers.


javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

this is a typical record

2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"


  was:
importing a tab delimited file 8megs in size

javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

n

        Summary: Out of memory when importing large files  /framework/datafile  (was: Out of memory when importing large files)

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

BJ Freeman updated OFBIZ-3558:
------------------------------

    Attachment: Copy of dobraimportTemplate.xml

this is the datafile definitions.
since the data is a large file I will not put in the Jira.
I don't know if it is legal to put a url to the 8meg file.

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845633#action_12845633 ]

BJ Freeman commented on OFBIZ-3558:
-----------------------------------

The import does not go directly into the database.
it creates xml files that are then imported via code or webtools.

http://cwiki.apache.org/confluence/display/OFBENDUSER/OFBiz%27s+Data+File+Tools

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845760#action_12845760 ]

Jacques Le Roux commented on OFBIZ-3558:
----------------------------------------

I don't think anything prevent to have an URL to a 8meg file.

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-3558:
-----------------------------------

    Description:
importing a tab delimited file 8megs(discontinued products) in size

to me this is poor design since most file are even bigger when importing inventory from suppliers.


javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

this is a typical record

{code}
2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's
Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the
impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many
occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of
21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00
181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products
/569/images_inventory_23320.jpg 250 250"
{code}

  was:
importing a tab delimited file 8megs(discontinued products) in size

to me this is poor design since most file are even bigger when importing inventory from suppliers.


javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

this is a typical record

{code}
2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"
{code}

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> {code}
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's
> Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the
> impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many
> occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of
> 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00
> 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products
> /569/images_inventory_23320.jpg 250 250"
> {code}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacques Le Roux updated OFBIZ-3558:
-----------------------------------

    Description:
importing a tab delimited file 8megs(discontinued products) in size

to me this is poor design since most file are even bigger when importing inventory from suppliers.


javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

this is a typical record

{code}
2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"
{code}

  was:
importing a tab delimited file 8megs(discontinued products) in size

to me this is poor design since most file are even bigger when importing inventory from suppliers.


javax.servlet.ServletException: Servlet execution threw an exception
        org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)

root cause

java.lang.OutOfMemoryError: Java heap space
        java.util.Arrays.copyOfRange(Unknown Source)
        java.lang.String.<init>(Unknown Source)
        java.lang.StringBuffer.toString(Unknown Source)
        java.io.StringWriter.toString(Unknown Source)
        org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
        org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
        org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
        org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
        org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
        org.apache.log4j.Category.callAppenders(Category.java:206)
        org.apache.log4j.Category.forcedLog(Category.java:391)
        org.apache.log4j.Category.log(Category.java:856)
        org.ofbiz.base.util.Debug.log(Debug.java:166)
        org.ofbiz.base.util.Debug.log(Debug.java:149)
        org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
        org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
        org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
        org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
        org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        java.lang.reflect.Method.invoke(Unknown Source)
        org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
        groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
        groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
        org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
        org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
        viewdatafile.run(viewdatafile.groovy:65)
        org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
        org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
        org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)

this is a typical record

2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> {code}
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products/569/images_inventory_23320.jpg 250 250"
> {code}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12845901#action_12845901 ]

BJ Freeman commented on OFBIZ-3558:
-----------------------------------

http://www.businessesnetwork.com/customerSupport/ofbiz/download/outofstock.txt
is the datafile.

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> {code}
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's
> Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the
> impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many
> occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of
> 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00
> 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products
> /569/images_inventory_23320.jpg 250 250"
> {code}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12846433#action_12846433 ]

BJ Freeman commented on OFBIZ-3558:
-----------------------------------

still researching.
I have used creating temp table Idea in my Database applications.
Currently I am looking at storing temporarily the list of records in persistence layer.
i have seen code where an entities is created in java code.
what I don't see is how to implement the entity(create table) in database on the fly.
I am sure that can be done.
next is to find a way to remove the entity, from the db at the end of the process.

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>            Priority: Blocker
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> {code}
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's
> Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the
> impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many
> occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of
> 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00
> 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products
> /569/images_inventory_23320.jpg 250 250"
> {code}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[Nicolas Malin (Jira)]

     [ https://issues.apache.org/jira/browse/OFBIZ-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Scott Gray updated OFBIZ-3558:
------------------------------

    Priority: Major  (was: Blocker)

> Out of memory when importing large files  /framework/datafile
> -------------------------------------------------------------
>
>                 Key: OFBIZ-3558
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-3558
>             Project: OFBiz
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>            Reporter: BJ Freeman
>             Fix For: Release Branch 9.04, Release Candidate Branch 10.04, SVN trunk
>
>         Attachments: Copy of dobraimportTemplate.xml
>
>
> importing a tab delimited file 8megs(discontinued products) in size
> to me this is poor design since most file are even bigger when importing inventory from suppliers.
> javax.servlet.ServletException: Servlet execution threw an exception
> org.ofbiz.webapp.control.ContextFilter.doFilter(ContextFilter.java:259)
> root cause
> java.lang.OutOfMemoryError: Java heap space
> java.util.Arrays.copyOfRange(Unknown Source)
> java.lang.String.<init>(Unknown Source)
> java.lang.StringBuffer.toString(Unknown Source)
> java.io.StringWriter.toString(Unknown Source)
> org.apache.log4j.spi.LocationInfo.<init>(LocationInfo.java:114)
> org.apache.log4j.spi.LoggingEvent.getLocationInformation(LoggingEvent.java:247)
> org.apache.log4j.AsyncAppender.append(AsyncAppender.java:160)
> org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
> org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
> org.apache.log4j.Category.callAppenders(Category.java:206)
> org.apache.log4j.Category.forcedLog(Category.java:391)
> org.apache.log4j.Category.log(Category.java:856)
> org.ofbiz.base.util.Debug.log(Debug.java:166)
> org.ofbiz.base.util.Debug.log(Debug.java:149)
> org.ofbiz.base.util.Debug.logInfo(Debug.java:258)
> org.ofbiz.datafile.RecordIterator.getNextLine(RecordIterator.java:117)
> org.ofbiz.datafile.RecordIterator.next(RecordIterator.java:163)
> org.ofbiz.datafile.DataFile.readDataFile(DataFile.java:126)
> org.ofbiz.datafile.DataFile.readFile(DataFile.java:63)
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> java.lang.reflect.Method.invoke(Unknown Source)
> org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:86)
> groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:230)
> groovy.lang.MetaClassImpl.invokeStaticMethod(MetaClassImpl.java:1105)
> org.codehaus.groovy.runtime.InvokerHelper.invokeMethod(InvokerHelper.java:749)
> org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodN(ScriptBytecodeAdapter.java:170)
> viewdatafile.run(viewdatafile.groovy:65)
> org.ofbiz.base.util.GroovyUtil.runScriptAtLocation(GroovyUtil.java:117)
> org.ofbiz.widget.screen.ModelScreenAction$Script.runAction(ModelScreenAction.java:408)
> org.ofbiz.widget.screen.ModelScreenAction.runSubActions(ModelScreenAction.java:118)
> this is a typical record
> {code}
> 2010-03-15 16:56:15,421 (http-0.0.0.0-8443-1) [ RecordIterator.java:117:INFO ] br.readLine()="569 0.00 Britney 5473236 23320:302273:P "14K Yellow Gold Kansas City Chief's
> Enamel Helment Pendant" "Reflecting the flawless elegance from a long-gone age of gentility, this luminescent pendant blends the sublime majestic aesthetics of the past with the
> impeccable advanced metallurgical technology of the modern day. This 14k yellow gold pendant makes a great addition to any jewelry collection and can be worn on many
> occasions. The total metal weight of this pendant is 2.30 grams. This pendant is part of our sports pendants collection of jewelry. This pendant has a length of 21 mm, a width of
> 21.25 mm." new Britney Britney 0 "2009-11-06 16:41:38" 5910653 23320:302273:P 23320:302273:P "14K Yellow Gold Kansas City Chief's Enamel Helment Pendant" 0 0 0.00
> 181.94 176.72 599.98 10 in-stock 0 "2009-11-06 15:02:07" "Catalog||Apparel, shoes & jewelry||Jewelry & watches||Pendants" http://images.doba.com/products
> /569/images_inventory_23320.jpg 250 250"
> {code}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.