[jira] Created: (OFBIZ-453) XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files

[Nicolas Malin (Jira)]

XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
------------------------------------------------------------------------------------------------------------

                 Key: OFBIZ-453
                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
             Project: OFBiz (The Open for Business Project)
          Issue Type: Bug
          Components: framework
    Affects Versions: SVN trunk
         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
Apache OFBiz (SVN Trunk)
            Reporter: Fabian Gorsler
         Attachments: patch.txt

Hi,

if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.

The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.

Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.

HTH and go for the release. :-)

For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

     [ http://issues.apache.org/jira/browse/OFBIZ-453?page=all ]

Fabian Gorsler updated OFBIZ-453:
---------------------------------

    Attachment: patch.txt

This is the patch. HTH!

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449583 ]
           
Jacopo Cappellato commented on OFBIZ-453:
-----------------------------------------

Thanks Fabian,

for working at this.
However I'm not sure that adding xml specific treatment in the GenericEntity.java class is a good idea since that file should not contain output specific code.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449594 ]
           
Fabian Gorsler commented on OFBIZ-453:
--------------------------------------

Hi Jacopo,

GenericEntity has already code for XML-specific issues. My changes were in the existing method writeXmlText() and there are five other XML-specific methods.

Best regards,
Fabian.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449595 ]
           
Jacopo Cappellato commented on OFBIZ-453:
-----------------------------------------

Fabian,

you are completely correct... I just had a cursory review at your patch without looking at the existing code (sorry).
Well, so I think that Fabian's patch can go in svn, am I wrong?

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449597 ]
           
Jacques Le Roux commented on OFBIZ-453:
---------------------------------------

+1

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449602 ]
           
Fabian Gorsler commented on OFBIZ-453:
--------------------------------------

Hi Jacopo,

no problem. :)

Best regards,
Fabian.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449608 ]
           
Hans Bakker commented on OFBIZ-453:
-----------------------------------

+1 i had several problems with accented characters...deleted them out of the xml file...this much better,

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449670 ]
           
David E. Jones commented on OFBIZ-453:
--------------------------------------

This looks like a good idea and solves a long time problem... even if it would be nice to have the UTF-8 stuff working properly.

This patch looks fine, though a few code cleanups would be good:

1. ""+ curChar is not a very efficient way to create a String from a char
2. the Debug.logInfo line should probably be changed to logVerbose and have a if(Debug.verboseOn()) prefix added to the line for performance reasons to avoid the time/memory required to create that String when it's not in use
3. the replacement=null; line isn't necessary

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449830 ]
           
Fabian Gorsler commented on OFBIZ-453:
--------------------------------------

Hi David,

2) was added, 3) was a mistake (I uploaded the wrong file...:/), but I don't now how I could solve 1) more efficient. I did it always that way - how would you do it?

Nevertheless I will attach a new patch.

Best regards,
Fabian.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

     [ http://issues.apache.org/jira/browse/OFBIZ-453?page=all ]

Fabian Gorsler updated OFBIZ-453:
---------------------------------

    Attachment: patch-v2.txt

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch-v2.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12449988 ]
           
Jacopo Cappellato commented on OFBIZ-453:
-----------------------------------------

Fabian,

your last patch contains tab-based formatting.
About the 1), I'm not sure, but maybe something like this could be better?

String replacement = "&#" + Character.codePointAt(new char[]{curChar}, 0) + ";";

Just my two cents.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch-v2.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12450119 ]
           
Fabian Gorsler commented on OFBIZ-453:
--------------------------------------

Hi Jacopo,

thanks for the advice. This should be quite faster. Tomorrow I will upload a new version. Before I will test a few possibilities to solve the problem and profile them. Perhaps someone else is interested in the results, too.

Best regards,
Fabian.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch-v2.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12450493 ]
           
Fabian Gorsler commented on OFBIZ-453:
--------------------------------------

Sorry for my delay, but here is the patched (and tested) patch. :)

PS: Jacopo, your advice is correct. There are some performance improvements when using the elementary char[]. Thanks again!

Best regards,
Fabian.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch-v2.txt, patch-v3.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

     [ http://issues.apache.org/jira/browse/OFBIZ-453?page=all ]

Fabian Gorsler updated OFBIZ-453:
---------------------------------

    Attachment: patch-v3.txt

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Attachments: patch-v2.txt, patch-v3.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

     [ http://issues.apache.org/jira/browse/OFBIZ-453?page=all ]

David E. Jones closed OFBIZ-453.
--------------------------------

    Fix Version/s: SVN trunk
       Resolution: Fixed
         Assignee: David E. Jones

Thanks Fabian. This is in SVN rev 476469, with one major exception: I changed it to not use the codePointAt method because it was introduced in Java 5 and won't work in a 1.4 series JVM. The change I made _should_ work, but could you please test it?

Also, as a side note, was there a particular reason that you were using the codePointAt method, especially with a single character?

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Assigned To: David E. Jones
>             Fix For: SVN trunk
>
>         Attachments: patch-v2.txt, patch-v3.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12450988 ]
           
David E. Jones commented on OFBIZ-453:
--------------------------------------

Fabian: please note some details for future contributions:

1. use 4 spaces for indentation (not tabs)
2. put spaces after keywords like if, while, etc and around most + signs
3. when creating a patch, do it from the ofbiz directory (this one was 4 directories above)

Note that these and more details are described here and I highly recommend reviewing them:

http://docs.ofbiz.org/x/r

Thanks,
-David

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Assigned To: David E. Jones
>             Fix For: SVN trunk
>
>         Attachments: patch-v2.txt, patch-v3.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[Nicolas Malin (Jira)]

    [ http://issues.apache.org/jira/browse/OFBIZ-453?page=comments#action_12451147 ]
           
Fabian Gorsler commented on OFBIZ-453:
--------------------------------------

Hi David,

I'm sorry. I'm quite stressed at the moment and didn't think about more details than "Just works". Sorry for that, I should have waited. And I'm asking me why I didn't have the idea to use just a casting to int...
Next time it'll be better, for sure! :)

Thanks for the patience (to all others, too ;)) and best regards,
Fabian.

> XML import from Webtools Entity Export fails when (e.g.) umlauts are in the via Webtools generated XML files
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: OFBIZ-453
>                 URL: http://issues.apache.org/jira/browse/OFBIZ-453
>             Project: OFBiz (The Open for Business Project)
>          Issue Type: Bug
>          Components: framework
>    Affects Versions: SVN trunk
>         Environment: Linux 2.6.18.1, x86_64 with UTF-8 environment and file system / The problem exists on Windows XP, too.
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_08-b03)
> Apache OFBiz (SVN Trunk)
>            Reporter: Fabian Gorsler
>         Assigned To: David E. Jones
>             Fix For: SVN trunk
>
>         Attachments: patch-v2.txt, patch-v3.txt, patch.txt
>
>
> Hi,
> if you have entities filled up with (e.g.) German umlauts, you won't be able to import them via Webtools. The characters are writtend directly to the XML file and the parser can't read them while import. In order to import the files you would have to change all umlauts to their corresponding XML entities (e.g. Ä --> ä). Files with XML entites can be imported w/o any problems and the data can be stored and represented correctly.
> The attached patch fixes the issue. The patch replaces each char with an value bigger than 0x7F with &#NNN; - this means that there will just the ASCII chars will appear in the XML exports. CDATA doesn't need to be exported this way. Their content will be "untouched". All changes to characters will be logged to the INFO-facility - this could perhaps be switched to the DEBUG-facility. Only org.ofbiz.entity.GenericEntity was changed, no other files were modified for this patch.
> Please note that I justed tested this case with German umlauts. Please check the behavior with other special characters, too.
> HTH and go for the release. :-)
> For more information see this thread on ofbiz-user: http://mail-archives.apache.org/mod_mbox/incubator-ofbiz-user/200611.mbox/%3c455040E2.4090306@...%3e

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira