Externalizing strings - need some feedback please - PEASE READ.

[Gerrie Myburg [ MTN - Innovation Centre ]]

PLEASE READ! Not the end of the world, but if I start booking in the
code a lot of people can be very pissed off if they don't agree with
what I need to do.

Externalizing strings.
I can now externalize all the strings in the ofbiz code base. I used the
antlr parser generator and a JAVA 1.5 grammar to write a program to do
it.

The problem I now have is that there is more than one resource file per
module and that the format for the key strings differs from one module
property file to the other module property file.

The first question is the number of property files per module. Can we
not have one property file per module? I understand that the different
files create a namespace effect, but this same effect can be gotten by
encoding the namespace in the property key. Eg

error.ui.get.lost=You made a mistake. Go to the start and try again.

If we do this then all the property strings can be in one file. I would
prefer one property file per module with the namespace encoded as part
of the key.

The second issue is the key string format. I need to now what is the
preferred key string format please? There are 2 formats in general

Example

1. Name1Name2Name3    -  like a java id with the first letter in caps
2. name1.name2.name3  - '.' in place of the spaces

Some of the keys also contains contextual information eg

error.module.name1.name2  

I can try to place contextual information in the generated keys for a
message when I run the program that extract the strings, but don't hold
your breath as this is VERY hard to do in principle.

If I can get some consensus on the format of the key strings then I will
go and change all the key string and property files to the same format.
If nobody responds to this mail then I will change all the key strings
as in format 2 above. It is more readable for me. THIS CHANGE AFFECTS
ALL THE FILES IN ofbiz!

There are more problems with the message string constructions that I
never foresaw and cannot find a simple way to resolve. This relates to
constructs like

String message = "this " + ( isThere == null ? " is " :
isThere.toString) + ".";

I need to go and rewrite the message by hand in such a way that it can
be translated to other languages. The above example does not happen
often. I cannot guarantee that there are other messages that are
constructed in a similar manner. We will just have to go through the
code looking for other strange examples.

One disadvantage of programmatically restructuring code is that it is
hard to deal with exceptions. One example of exceptions is the
construction of strings contacting

1. SQL
2. html

String.  If no one complains about this then I will extract these
strings as well.


NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/default.aspx?pid=34411

[cjhowe]

I generally have very little opinion on the
externalizing of strings.  However, because these
variables are primarily used in a freemarker syntax, I
would prefer it remain Name1Name2Name3 format.  I'm
not sure if I would call it a naming convention or a
naming convenience, but the "." in freemarker
distinguishes a node.  I think it is possible that it
creates issues with the creation of the label maps.

Also, the externalized string shouldn't necessarily be
"readable".  Translation is in context of it's usage,
not translating the words that are present.  While it
may make quick work of tranlating strings, those
translations can be quite innacurate, meaningless,
embarrasing, or offensive if not regarded in their
context.

--- "Gerrie Myburg  [ MTN - Innovation Centre ]"
<[hidden email]> wrote:

> PLEASE READ! Not the end of the world, but if I
> start booking in the
> code a lot of people can be very pissed off if they
> don't agree with
> what I need to do.
>
> Externalizing strings.
> I can now externalize all the strings in the ofbiz
> code base. I used the
> antlr parser generator and a JAVA 1.5 grammar to
> write a program to do
> it.
>
> The problem I now have is that there is more than
> one resource file per
> module and that the format for the key strings
> differs from one module
> property file to the other module property file.
>
> The first question is the number of property files
> per module. Can we
> not have one property file per module? I understand
> that the different
> files create a namespace effect, but this same
> effect can be gotten by
> encoding the namespace in the property key. Eg
>
> error.ui.get.lost=You made a mistake. Go to the
> start and try again.
>
> If we do this then all the property strings can be
> in one file. I would
> prefer one property file per module with the
> namespace encoded as part
> of the key.
>
> The second issue is the key string format. I need to
> now what is the
> preferred key string format please? There are 2
> formats in general
>
> Example
>
> 1. Name1Name2Name3    -  like a java id with the
> first letter in caps
> 2. name1.name2.name3  - '.' in place of the spaces
>
> Some of the keys also contains contextual
> information eg
>
> error.module.name1.name2  
>
> I can try to place contextual information in the
> generated keys for a
> message when I run the program that extract the
> strings, but don't hold
> your breath as this is VERY hard to do in principle.
>
> If I can get some consensus on the format of the key
> strings then I will
> go and change all the key string and property files
> to the same format.
> If nobody responds to this mail then I will change
> all the key strings
> as in format 2 above. It is more readable for me.
> THIS CHANGE AFFECTS
> ALL THE FILES IN ofbiz!
>
> There are more problems with the message string
> constructions that I
> never foresaw and cannot find a simple way to
> resolve. This relates to
> constructs like
>
> String message = "this " + ( isThere == null ? " is
> " :
> isThere.toString) + ".";
>
> I need to go and rewrite the message by hand in such
> a way that it can
> be translated to other languages. The above example
> does not happen
> often. I cannot guarantee that there are other
> messages that are
> constructed in a similar manner. We will just have
> to go through the
> code looking for other strange examples.
>
> One disadvantage of programmatically restructuring
> code is that it is
> hard to deal with exceptions. One example of
> exceptions is the
> construction of strings contacting
>
> 1. SQL
> 2. html
>
> String.  If no one complains about this then I will
> extract these
> strings as well.
>
>
> NOTE: This e-mail message is subject to the MTN
> Group disclaimer see
> http://www.mtn.co.za/default.aspx?pid=34411
>

[David E Jones-2]

Gerrie,

This is an interesting effort, but I think it would cause serious  
problems if we externalized ALL strings. I'm all for externalizing  
error messages to support localization efforts. But for many things  
externalizing strings would make the code more complicated, error  
prone, and difficult to maintain, and that is a big problem for a  
project like OFBiz. Developer efficiency and avoiding complexity is  
very important in order to be able to handle a large volume of  
business logic with as little code as possible, and among a small  
group of people.

On the property name issue: I'd prefer names that do not use dots for  
word separators because in some code, like in FreeMarker ${} and in  
the various OFBiz XML file attributes that support the Flexible  
String Expander with the ${} syntax, it causes problems because dots  
are interpreted as meaning the symbol to the left of the dot  
represents a Map name and the one to the right of the dot represents  
a Map entry key. Actually, we really should change all of the  
existing ones that have dots to use capital letters to separate the  
words instead.

I hope that helps... though I recognize that especially the first  
comment doesn't really make it easier...

-David


On Oct 6, 2006, at 11:40 AM, Gerrie Myburg [ MTN - Innovation  
Centre ] wrote:

> PLEASE READ! Not the end of the world, but if I start booking in the
> code a lot of people can be very pissed off if they don't agree with
> what I need to do.
>
> Externalizing strings.
> I can now externalize all the strings in the ofbiz code base. I  
> used the
> antlr parser generator and a JAVA 1.5 grammar to write a program to do
> it.
>
> The problem I now have is that there is more than one resource file  
> per
> module and that the format for the key strings differs from one module
> property file to the other module property file.
>
> The first question is the number of property files per module. Can we
> not have one property file per module? I understand that the different
> files create a namespace effect, but this same effect can be gotten by
> encoding the namespace in the property key. Eg
>
> error.ui.get.lost=You made a mistake. Go to the start and try again.
>
> If we do this then all the property strings can be in one file. I  
> would
> prefer one property file per module with the namespace encoded as part
> of the key.
>
> The second issue is the key string format. I need to now what is the
> preferred key string format please? There are 2 formats in general
>
> Example
>
> 1. Name1Name2Name3    -  like a java id with the first letter in caps
> 2. name1.name2.name3  - '.' in place of the spaces
>
> Some of the keys also contains contextual information eg
>
> error.module.name1.name2
>
> I can try to place contextual information in the generated keys for a
> message when I run the program that extract the strings, but don't  
> hold
> your breath as this is VERY hard to do in principle.
>
> If I can get some consensus on the format of the key strings then I  
> will
> go and change all the key string and property files to the same  
> format.
> If nobody responds to this mail then I will change all the key strings
> as in format 2 above. It is more readable for me. THIS CHANGE AFFECTS
> ALL THE FILES IN ofbiz!
>
> There are more problems with the message string constructions that I
> never foresaw and cannot find a simple way to resolve. This relates to
> constructs like
>
> String message = "this " + ( isThere == null ? " is " :
> isThere.toString) + ".";
>
> I need to go and rewrite the message by hand in such a way that it can
> be translated to other languages. The above example does not happen
> often. I cannot guarantee that there are other messages that are
> constructed in a similar manner. We will just have to go through the
> code looking for other strange examples.
>
> One disadvantage of programmatically restructuring code is that it is
> hard to deal with exceptions. One example of exceptions is the
> construction of strings contacting
>
> 1. SQL
> 2. html
>
> String.  If no one complains about this then I will extract these
> strings as well.
>
>
> NOTE: This e-mail message is subject to the MTN Group disclaimer  
> see http://www.mtn.co.za/default.aspx?pid=34411

[Jacques Le Roux]

Gerrie,

Is this aspect there is still some work to do. For instance I will soon drop all Common labels that are in *_fr.properties files and
not in other *.properties files (default properties file being the reference). It seems indeed that some cleaning have been done and
that some needs
always to be done.

I don't want to stop your effort but from my point of view even if it's "not the end of the world" your work could have a great
impact
and we have to watch at it carefully.

Normalising labels names sounds good to me. For the dot separator I agree with what was already said.

And last but not least I agree with David's 1st paragraph, not an easy task for sure...

Jacques

PS : sorry I guess this message is not clear. Let me ask a question : why did you choose to make a such task in OFBiz ?

>
> Gerrie,
>
> This is an interesting effort, but I think it would cause serious
> problems if we externalized ALL strings. I'm all for externalizing
> error messages to support localization efforts. But for many things
> externalizing strings would make the code more complicated, error
> prone, and difficult to maintain, and that is a big problem for a
> project like OFBiz. Developer efficiency and avoiding complexity is
> very important in order to be able to handle a large volume of
> business logic with as little code as possible, and among a small
> group of people.
>
> On the property name issue: I'd prefer names that do not use dots for
> word separators because in some code, like in FreeMarker ${} and in
> the various OFBiz XML file attributes that support the Flexible
> String Expander with the ${} syntax, it causes problems because dots
> are interpreted as meaning the symbol to the left of the dot
> represents a Map name and the one to the right of the dot represents
> a Map entry key. Actually, we really should change all of the
> existing ones that have dots to use capital letters to separate the
> words instead.
>
> I hope that helps... though I recognize that especially the first
> comment doesn't really make it easier...
>
> -David
>
>
> On Oct 6, 2006, at 11:40 AM, Gerrie Myburg [ MTN - Innovation
> Centre ] wrote:
>
> > PLEASE READ! Not the end of the world, but if I start booking in the
> > code a lot of people can be very pissed off if they don't agree with
> > what I need to do.
> >
> > Externalizing strings.
> > I can now externalize all the strings in the ofbiz code base. I
> > used the
> > antlr parser generator and a JAVA 1.5 grammar to write a program to do
> > it.
> >
> > The problem I now have is that there is more than one resource file
> > per
> > module and that the format for the key strings differs from one module
> > property file to the other module property file.
> >
> > The first question is the number of property files per module. Can we
> > not have one property file per module? I understand that the different
> > files create a namespace effect, but this same effect can be gotten by
> > encoding the namespace in the property key. Eg
> >
> > error.ui.get.lost=You made a mistake. Go to the start and try again.
> >
> > If we do this then all the property strings can be in one file. I
> > would
> > prefer one property file per module with the namespace encoded as part
> > of the key.
> >
> > The second issue is the key string format. I need to now what is the
> > preferred key string format please? There are 2 formats in general
> >
> > Example
> >
> > 1. Name1Name2Name3    -  like a java id with the first letter in caps
> > 2. name1.name2.name3  - '.' in place of the spaces
> >
> > Some of the keys also contains contextual information eg
> >
> > error.module.name1.name2
> >
> > I can try to place contextual information in the generated keys for a
> > message when I run the program that extract the strings, but don't
> > hold
> > your breath as this is VERY hard to do in principle.
> >
> > If I can get some consensus on the format of the key strings then I
> > will
> > go and change all the key string and property files to the same
> > format.
> > If nobody responds to this mail then I will change all the key strings
> > as in format 2 above. It is more readable for me. THIS CHANGE AFFECTS
> > ALL THE FILES IN ofbiz!
> >
> > There are more problems with the message string constructions that I
> > never foresaw and cannot find a simple way to resolve. This relates to
> > constructs like
> >
> > String message = "this " + ( isThere == null ? " is " :
> > isThere.toString) + ".";
> >
> > I need to go and rewrite the message by hand in such a way that it can
> > be translated to other languages. The above example does not happen
> > often. I cannot guarantee that there are other messages that are
> > constructed in a similar manner. We will just have to go through the
> > code looking for other strange examples.
> >
> > One disadvantage of programmatically restructuring code is that it is
> > hard to deal with exceptions. One example of exceptions is the
> > construction of strings contacting
> >
> > 1. SQL
> > 2. html
> >
> > String.  If no one complains about this then I will extract these
> > strings as well.
> >
> >
> > NOTE: This e-mail message is subject to the MTN Group disclaimer
> > see http://www.mtn.co.za/default.aspx?pid=34411
>

[Gerrie Myburg [ MTN - Innovation Centre ]]

Jacques, David

Not all strings are been externalized. Only strings that contain a " "
are considered for externalization. These are the ones that are
typically that is meant to be read by a person. There are other strings
that are also meant be read like: value + "::" value, but to take these
strings out will just make the developers job a lot harder in the long
run (complexity).

To be able to do better than this implies that I will have to make the
program understand what is a message. This can be done by looking at the
context in which the string is used. It does not guarantee that the
string is a message; it only gives a high probability that the string is
a message.

I will transform all the string keys to Name1Name2Name3... I will also
use a spell checker to make sure that the names are at least spelled
correctly. In the case where I don't know if the name is spelled ok then
I will flag the key and check it.

I will split the property files into 2 groups per module. Firstly the
strings that goes into the logs. These have a low priority for been
translated. Then all the other strings that need translation are placed
in the other group.

Jacques you are correct that these changes can stuff everything up. This
is why I do not want to do the work piecemeal, but all in one go. This
is why I am building programs to do the changes. Modifying these
programs is not hard and a change to the way the externalization is done
is relatively inexpensive.

Jacques to answer your question as to why:
I want to make the externalization changes as I want to localize ofbiz
for Afrikaans. I have noticed some other problems in ofbiz that I think
needs cleaning up, but I will get to this later. From looking at the
changes that are committed it seems that most relate to business
functionality been fixed or added. This is nice, but it does not address
the issue of overall code quality decreasing with time due to entropy.
It is the old thing, the older a system becomes the more messy things
become unless you do some house cleaning now and then.

Gerrie



NOTE: This e-mail message is subject to the MTN Group disclaimer see http://www.mtn.co.za/default.aspx?pid=34411