UTF-8 input

[V.S.]

Why after trunk update 2010-06-03 i can't input any UTF8 text in any form field? Only Latin characters display correctly after input. Any other (for example russian) characters stores in DB and displays wrong.

[BJ Freeman]

I don't know the exact answer.
I do know we have had problems importing UTF8 via webtools since version
4.0.
so lets start with what version your are running and any error in the
logs you may find that relate to this.
then use the appropriate demo server and perform the steps and give a
link to the pages you work with.
this will give us clearer picture to help you.

note the demo server get rebuilt nightly so any thing you do will be
lost if you do it close to that time and before one of us can look at it.
thanks

=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>


V.S. sent the following on 6/14/2010 7:55 AM:
>
> Why after trunk update 2010-06-03 i can't input any UTF8 text in any form
> field? Only Latin characters display correctly after input. Any other (for
> example russian) characters stores in DB and displays wrong.

[V.S.]

Tested on demo server now
https://demo-trunk.ofbiz.apache.org/partymgr/control/editpartygroup?create_new=Y
input:
Group Name   : testutf8
Group Name Local: Тест ввода русских символов

after click "save" i see:
Party Id   10005
Group Name testutf8
Group Name Local Ð¢ÐµÑ Ñ Ð²Ð²Ð¾Ð´Ð° Ñ Ñ Ñ Ñ ÐºÐ¸Ñ Ñ Ð¸Ð¼Ð²Ð¾Ð»Ð¾Ð²

[V.S.]

input


result

[BJ Freeman]

looks like got stored as hex bytes instead of Unicode hex words.
\xd0\xa2\xd0\xb5\xd1 \xd1 \xd0\xb2\xd0\xb2\xd0\xbe\xd0\xb4\xd0\xb0 \xd1
\xd1 \xd1 \xd1 \xd0\xba\xd0\xb8\xd1 \xd1
\xd0\xb8\xd0\xbc\xd0\xb2\xd0\xbe\xd0\xbb\xd0\xbe\xd0\xb2
https://demo-trunk.ofbiz.apache.org/webtools/control/ViewGeneric?entityName=PartyGroup&partyId=10005&enableEdit=true

V.S. sent the following on 6/14/2010 12:50 PM:


=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>

>
> Tested on demo server now
> https://demo-trunk.ofbiz.apache.org/partymgr/control/editpartygroup?create_new=Y
> input:
> Group Name   : testutf8
> Group Name Local: Тест ввода русских символов
>
> after click "save" i see:
> Party Id   10005
> Group Name testutf8
> Group Name Local Ð¢ÐµÑ Ñ Ð²Ð²Ð¾Ð´Ð° Ñ Ñ Ñ Ñ ÐºÐ¸Ñ Ñ Ð¸Ð¼Ð²Ð¾Ð»Ð¾Ð²
>

[BJ Freeman]

I pasted
Тест ввода русских символов
into the record and it was changed to bytes.
hopefully someone will have time to look into this.
I will think on it.

=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>


BJ Freeman sent the following on 6/14/2010 2:26 PM:
> Тест ввода русских символов

[Sumit Porwal]

Hi VS & BJ,

Below details may help you -

Command to get character set settings for MySQL - show variables like
'char%';
Snapshot: http://screencast.com/t/OTVmNzEwNjYt
I think for database default character set to use is set to latin1, You can
set these variables according to your requirement. For your problem you can
set "utf8" at database level or entity level or column level.

Ex: ALTER TABLE <myTableName> MODIFY <myColumnName> VARCHAR(255) CHARACTER
SET utf8;

Link -
http://confluence.atlassian.com/display/DOC/Configuring+Database+Character+Encoding


Thanks,
Sumit Porwal
Email: [hidden email]
Contact: [phone]


On Tue, Jun 15, 2010 at 3:09 AM, BJ Freeman <[hidden email]> wrote:

>
> I pasted
> Тест ввода русских символов
> into the record and it was changed to bytes.
> hopefully someone will have time to look into this.
> I will think on it.
>
>
> =========================
> BJ Freeman
> http://bjfreeman.elance.com
> Strategic Power Office with Supplier Automation  <
> http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> Specialtymarket.com  <http://www.specialtymarket.com/>
>
> Systems Integrator-- Glad to Assist
>
> Chat  Y! messenger: bjfr33man
> Linkedin <
> http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro
> >
>
>
> BJ Freeman sent the following on 6/14/2010 2:26 PM:
>
>> Тест ввода русских символов
>>
>
>

[V.S.]

My own installation on PostgreSQL. Encoding UTF8
It works well until trunk update 2010-06-03. I did not do any changes and did not create new base. Input has ceased to work right after updating trunk.

trunk update 2010-06-02 worked correctly. Something have broken 2010-06-03.

[Jacques Le Roux]

From you information I understand that *you* updated the 2010-06-03
If you could give the revision you used before and the one you updated for it would help.

Jacques

From: "V.S." <[hidden email]>
> My own installation on PostgreSQL. Encoding UTF8
> It works well until trunk update 2010-06-03. I did not do any changes and
> did not create new base. Input has ceased to work right after updating
> trunk.
> --
> View this message in context: http://ofbiz.135035.n4.nabble.com/UTF-8-input-tp2254499p2255633.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
>

[V.S.]

More precisely
svn  co -r {2010-06-02} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
works normally, and in
svn  co -r {2010-06-04} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
already there is an error

[Jacques Le Roux]

OK, this is a bit better but we have 83 commits between these 2 dates...

Jacques

From: "V.S." <[hidden email]>
> More precisely
> svn  co -r {2010-06-02} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
> works normally, and in
> svn  co -r {2010-06-04} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
> already there is an error
> --
> View this message in context: http://ofbiz.135035.n4.nabble.com/UTF-8-input-tp2254499p2255660.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
>

[V.S.]

I have checked up now.
svn  co -r {2010-06-03} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
too works correctly
The error appears after
svn  co -r {2010-06-04} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz

[BJ Freeman]

there were 27 commits on 06/03
it would depend on the time of day you did your co.
there were 9 commits on 06/04
to save us time we need the exact time of day you did theses so we can
narrow the range of commit to review.

thanks


=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>


V.S. sent the following on 6/15/2010 4:15 AM:
>
> I have checked up now.
> svn  co -r {2010-06-03} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
> too works correctly
> The error appears after
> svn  co -r {2010-06-04} http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz

[BJ Freeman]

see what you think of r951553
done on 06/04
I don't see anything in any other commit that comes close to causing
this problem
I have not read 951553 commit yet. will when I get back.



=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>


Jacques Le Roux sent the following on 6/15/2010 4:03 AM:

> OK, this is a bit better but we have 83 commits between these 2 dates...
>
> Jacques
>
> From: "V.S." <[hidden email]>
>> More precisely svn co -r {2010-06-02}
>> http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
>> works normally, and in svn co -r {2010-06-04}
>> http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
>> already there is an error
>> --
>> View this message in context:
>> http://ofbiz.135035.n4.nabble.com/UTF-8-input-tp2254499p2255660.html
>> Sent from the OFBiz - User mailing list archive at Nabble.com.
>>
>
>

[V.S.]

Very strange. An error in revision 951119. That is
svn  co -r 951118 http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
works correctly, and
svn  co -r 951119 http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
it is wrong.
I do not understand, why. There only one change which like should not so to influence.

ofbiz/trunk/framework/webapp/src/org/ofbiz/webapp/control/ContextFilter.java
-        httpRequest.getSession().setAttribute("webSiteId", config.getServletContext().getAttribute("webSiteId"));
+        httpRequest.getSession().setAttribute("webSiteId", WebSiteWorker.getWebSiteId(httpRequest));

Now I have updated to current trunk revision  and have manually edited ContextFilter.java. All works correctly.

[BJ Freeman]

that indeed strange.
I just looked at the revision history for ContextFilter.java
and don't see see any revision 951118 or 951119
it does show 939941` on 5/1/10 next one is 9539918 on 6/11/10
and the code shows
         // set the webSiteId in the session
         httpRequest.getSession().setAttribute("webSiteId",
config.getServletContext().getAttribute("webSiteId"));


V.S. sent the following on 6/16/2010 12:45 AM:


=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>

>
> Very strange. An error in revision 951119. That is
> svn  co -r 951118 http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
> works correctly, and
> svn  co -r 951119 http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
> it is wrong.
> I do not understand, why. There only one change which like should not so to
> influence.
>
> ofbiz/trunk/framework/webapp/src/org/ofbiz/webapp/control/ContextFilter.java
> -        httpRequest.getSession().setAttribute("webSiteId",
> config.getServletContext().getAttribute("webSiteId"));
> +        httpRequest.getSession().setAttribute("webSiteId",
> WebSiteWorker.getWebSiteId(httpRequest));
>
> Now I have updated to current trunk revision  and have manually edited
> ContextFilter.java. All works correctly.

[V.S.]

http://ofbiz.135035.n4.nabble.com/svn-commit-r951119-ofbiz-trunk-framework-webapp-src-org-ofbiz-webapp-control-ContextFilter-java-td2242192.html#a2242192

[BJ Freeman]

sorry was looking at 10.4 release.
I do see 951119 in the trunk.

=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>


BJ Freeman sent the following on 6/16/2010 1:17 AM:

> that indeed strange.
> I just looked at the revision history for ContextFilter.java
> and don't see see any revision 951118 or 951119
> it does show 939941` on 5/1/10 next one is 9539918 on 6/11/10
> and the code shows
> // set the webSiteId in the session
> httpRequest.getSession().setAttribute("webSiteId",
> config.getServletContext().getAttribute("webSiteId"));
>
>
> V.S. sent the following on 6/16/2010 12:45 AM:
>
>
> =========================
> BJ Freeman
> http://bjfreeman.elance.com
> Strategic Power Office with Supplier Automation
> <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
> Specialtymarket.com <http://www.specialtymarket.com/>
>
> Systems Integrator-- Glad to Assist
>
> Chat Y! messenger: bjfr33man
> Linkedin
> <http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>
>
>
>>
>> Very strange. An error in revision 951119. That is
>> svn co -r 951118 http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
>> works correctly, and
>> svn co -r 951119 http://svn.apache.org/repos/asf/ofbiz/trunk ofbiz
>> it is wrong.
>> I do not understand, why. There only one change which like should not
>> so to
>> influence.
>>
>> ofbiz/trunk/framework/webapp/src/org/ofbiz/webapp/control/ContextFilter.java
>>
>> - httpRequest.getSession().setAttribute("webSiteId",
>> config.getServletContext().getAttribute("webSiteId"));
>> + httpRequest.getSession().setAttribute("webSiteId",
>> WebSiteWorker.getWebSiteId(httpRequest));
>>
>> Now I have updated to current trunk revision and have manually edited
>> ContextFilter.java. All works correctly.
>

[BJ Freeman]

btw good work here is the comment
A patch from Sascha Rodekamp "Set webside Id in context filter"
(https://issues.apache.org/jira/browse/OFBIZ-3801) - OFBIZ-3801

=========================
BJ Freeman
http://bjfreeman.elance.com
Strategic Power Office with Supplier Automation  <http://www.businessesnetwork.com/automation/viewforum.php?f=52>
Specialtymarket.com  <http://www.specialtymarket.com/>

Systems Integrator-- Glad to Assist

Chat  Y! messenger: bjfr33man
Linkedin
<http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro>


V.S. sent the following on 6/16/2010 1:23 AM:
>
> http://ofbiz.135035.n4.nabble.com/svn-commit-r951119-ofbiz-trunk-framework-webapp-src-org-ofbiz-webapp-control-ContextFilter-java-td2242192.html#a2242192
>

[Jacques Le Roux]

Thanks Valentin,

I have reopened https://issues.apache.org/jira/browse/OFBIZ-3801

Jacques

From: "V.S." <[hidden email]>
>
> http://ofbiz.135035.n4.nabble.com/svn-commit-r951119-ofbiz-trunk-framework-webapp-src-org-ofbiz-webapp-control-ContextFilter-java-td2242192.html#a2242192
>
> --
> View this message in context: http://ofbiz.135035.n4.nabble.com/UTF-8-input-tp2254499p2257052.html
> Sent from the OFBiz - User mailing list archive at Nabble.com.
>