[jira] [Commented] (OFBIZ-11534) Error in uploading large files

[Nicolas Malin (Jira)]

    [ https://issues.apache.org/jira/browse/OFBIZ-11534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17084865#comment-17084865 ]

Jacques Le Roux commented on OFBIZ-11534:
-----------------------------------------

I had another look. I preferred to keep things as is and simple (KISS way) because my patch was possibly raising other issues.

Since we can upload ZIP files, simply converting ubuntu-18.04.4-desktop-amd64.iso to ubuntu-18.04.4-desktop-amd64.iso.zip works with the code as is.

But in order to avoid

{noformat}
SEVERE: Failed to complete processing of a request
java.lang.OutOfMemoryError: Java heap space
        at org.apache.commons.fileupload.disk.DiskFileItem.get(DiskFileItem.java:308)
        at org.apache.ofbiz.base.util.UtilHttp.getMultiPartParameterMap(UtilHttp.java:278)
{noformat}

for a 2GB+ file you need to set Xmx to more than at least the double of the file size, eg: {{gradlew ofbiz -PjvmArgs="-Xms1024M -Xmx5048M"}}
 
You then get another small issue (only in log) when redirecting to send the partyId parameter.

{noformat}
2020-04-16 10:58:15,039 |jsse-nio-8443-exec-7 |UtilObject                    |E| null
java.io.NotSerializableException: org.apache.commons.fileupload.disk.DiskFileItem
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_202]
        at java.util.ArrayList.writeObject(ArrayList.java:766) ~[?:1.8.0_202]
        at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_202]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_202]
        at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_202]
        at java.util.HashMap.internalWriteEntries(HashMap.java:1793) ~[?:1.8.0_202]
        at java.util.HashMap.writeObject(HashMap.java:1363) ~[?:1.8.0_202]
        at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_202]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_202]
        at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178) ~[?:1.8.0_202]
        at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348) ~[?:1.8.0_202]
        at org.apache.ofbiz.base.util.UtilObject.getBytes(UtilObject.java:47) [main/:?]
        at org.apache.ofbiz.webapp.control.RequestHandler.callRedirect(RequestHandler.java:886) [main/:?]
{noformat}

This is because, in this case at least, DiskFileItem is a temporary Object with a null value contained in "fileItems" attribute. It can't be detected by UtilMisc::makeMapSerializable and that makes "fileItems" not serializable. So it must be removed from reqAttrMap.

I'll soon commit the final change which is quite simple:

{code:java}
         if (reqAttrMap.size() > 0) {
+            // fileItems is not serializable.
+            // It contains a temporary DiskFileItem with a null value than can't be detected by UtilMisc::makeMapSerializable
+            // So it must be removed from reqAttrMap. See OFBIZ-11534
+            reqAttrMap.remove("fileItems");
             byte[] reqAttrMapBytes = UtilObject.getBytes(reqAttrMap);
{code}

> Error in uploading large files
> -------------------------------
>
>                 Key: OFBIZ-11534
>                 URL: https://issues.apache.org/jira/browse/OFBIZ-11534
>             Project: OFBiz
>          Issue Type: Bug
>          Components: ALL APPLICATIONS
>    Affects Versions: Trunk
>            Reporter: Chandan Khandelwal
>            Assignee: Jacques Le Roux
>            Priority: Major
>         Attachments: OFBIZ-11534.patch, OFBIZ-11534.patch
>
>
> There is an issue in uploading large files, I am able to upload files up to 2 GB successfully but getting an error when trying to upload a file larger than 2GB.
> Example - 
> 1. Go to party profile -
> [https://demo-trunk.ofbiz.apache.org/partymgr/control/viewprofile?partyId=admin]
> 2. Try to create a party content with a file larger than 2 GB
> 3. Error on console
> ============================
> java.lang.NegativeArraySizeException: nulljava.lang.NegativeArraySizeException: null at org.apache.commons.fileupload.disk.DiskFileItem.get(DiskFileItem.java:308) ~[commons-fileupload-1.4.jar:1.4] at org.apache.ofbiz.base.util.UtilHttp.getMultiPartParameterMap(UtilHttp.java:275) ~[main/:?] at
> =============================
>  

--
This message was sent by Atlassian Jira
(v8.3.4#803005)