Users - Configuration management?

[bjc]

Folks,

I'm curious about what people are doing for configuration  
management?  For instance, deployment and/or development to different  
IPs, ports, etc. all on the same box, or spread across different  
machines.  We're setting up to do ofbiz development, and work with  
multiple developers on a single beefy box.  In addition, we would  
like to, say, run regression tests on the same box while developing  
(hey, it's beefy).  Either of these requires some means of keeping  
things separated, such that we don't conflict with one another.

This brings up the issue of maintaining many separate files with the  
same set of IP or port values used in them.  There are quite a few  
files in which a clean checkout of ofbiz currently has either  
"localhost", "127.0.0.1", or "8080" in them.  If we do this for our  
situation, this would means that a developer must change several  
files after checking out, and remember not to commit those changes  
back to our local repository.  Likewise, when deploying to a  
production environment, those values may have to be changed, possibly  
with different values for various components in a high-volume setting.

My question is how do people deal with different configurations of  
host and port, database parameters, etc. today?  Is there some single  
change point that controls all others?  Perhaps variables or  
properties resolvers so I can use a $primaryHost variable in several  
files, rather than "hard-coding" localhost into all of those files?

Or does everyone just use localhost:8080 for everywhere?

Any insight will be deeply appreciated,

   Ben
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[David E. Jones]

Ben,

In general there shouldn't be any such references in code (and if there are they should be fixed...). This still leaves a fun mix of configuration files to deal with and for those I usually recommend just using patches for each deployment.

We run into this a fair amount with larger deployments of multiple instances of OFBiz. I've mostly worked with this in a server farm environment, but the same pattern would apply to running multiple instances on the same box.

The general idea is to have no manual editing of anything in the deployment process. This improves reliability quite a bit, especially in production environments where you generally want to keep the deployment process as simple as possible and have everything come straight from your code repository. This makes it easier and less error prone to test a deployment candidate, and if something does happen it is easier to track down and fix, and in the mean time back up to and deploy a previous revision.

This would involve a set of patches for each target system. Sometimes if certain deployments have more variations it is helpful to have branches in your local SVN repository (or whatever you are using) that can be synchronized as needed and deployed from there.

Usually the patches would include one for each test, staging, and production deployment. It depends a lot on what your environment is like of course. In most cases development deployments are the "no patch" settings and these are mostly run locally on developers machines. OFBiz runs fine on plain old (but modern...) desktops and for development that is the way to go.

-David



Benjamin Cox wrote:

> Folks,
>
> I'm curious about what people are doing for configuration  
> management?  For instance, deployment and/or development to different  
> IPs, ports, etc. all on the same box, or spread across different  
> machines.  We're setting up to do ofbiz development, and work with  
> multiple developers on a single beefy box.  In addition, we would  
> like to, say, run regression tests on the same box while developing  
> (hey, it's beefy).  Either of these requires some means of keeping  
> things separated, such that we don't conflict with one another.
>
> This brings up the issue of maintaining many separate files with the  
> same set of IP or port values used in them.  There are quite a few  
> files in which a clean checkout of ofbiz currently has either  
> "localhost", "127.0.0.1", or "8080" in them.  If we do this for our  
> situation, this would means that a developer must change several  
> files after checking out, and remember not to commit those changes  
> back to our local repository.  Likewise, when deploying to a  
> production environment, those values may have to be changed, possibly  
> with different values for various components in a high-volume setting.
>
> My question is how do people deal with different configurations of  
> host and port, database parameters, etc. today?  Is there some single  
> change point that controls all others?  Perhaps variables or  
> properties resolvers so I can use a $primaryHost variable in several  
> files, rather than "hard-coding" localhost into all of those files?
>
> Or does everyone just use localhost:8080 for everywhere?
>
> Any insight will be deeply appreciated,
>
>    Ben
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[bjc]

David, 

That sounds like a good start for most people, but still a tad brittle.  We would do developer instances on our own boxes as you suggest, except that we're scattered around the country and do a whole lot of pair programming.  VNC makes that work - we've found it easier and cheaper to maintain one multi-processor unix box for several developers.  Being a screamer, on a fat pipe, and capable of running several instances of it's nice lightweight window manager (with almost no color) - it saves us oodles of VNC Hell :-)

Even if we weren't doing development like that, we'd still want to share integration test boxes for multiple users on multiple projects, to ensure a similar environment to our hosting env (i.e. not Windows).  It sounds like you're suggesting that each developer/instance maintain a diff file and run patch to alter various files throughout the tree... is that right?  

This scares me for a few reasons:

   * developers must manually change multiple relevant files for each value to create their personal patch - this is likely error-prone, and doesn't allow for composing or reusing values (a la Ant properties)

   * individual developers must remember to apply the patch every time they check out fresh - better would be an externalized collection of their property values automatically picked up by the build process

   * developers have to remember not to check in their patched files - or edit them 'back' in order to check in other parts of these files

   * I haven't done enough patching recently, but I seem to remember it barfing on a file that didn't look sufficiently like the version from which the patch was produced.  So once I've applied it, how do I build a production copy?  A test copy?  Do I need a patch for every permutation?

   * someone adding a value that requires 'patching in' could wreak havoc on the team.  Developers would either have to blow away their sandboxes and reapply the patch (possibly forcing them to check-in incomplete changes), or else we'd have to maintain incremental patch files (probably perpetuating the previously portrayed permutations problem! ;-)).

One big difference between ofbiz and other libraries, app servers, or frameworks I've known is that it carries a development structure.  It's modular is another.  To my newbie eyes, it looks hard to separate ofbiz from the build and deployment process it lives in.  Thus, configuration management really seems to be the province of the framework - moreso than I'm used to, anyway.

What do you guys think - is this something ofbiz should address more thoroughly?  If not, what is a developer to do if his needs are similar to mine?  Some of you have to be working in team environments in order to accomplish more than a bit of new stuff in a short period of time!  Where does this patching process break down in practice?

Thanks again,

  Ben

---

HotWax Media (www.hotwaxmedia.com)

Ben Cox

Director of Technology

"Reality is that which, when you stop believing in it, does not go away."  - Philip K. Dick

On Mar 29, 2006, at 5:48 AM, David E. Jones wrote:

Ben,

In general there shouldn't be any such references in code (and if there are they should be fixed...). This still leaves a fun mix of configuration files to deal with and for those I usually recommend just using patches for each deployment.

We run into this a fair amount with larger deployments of multiple instances of OFBiz. I've mostly worked with this in a server farm environment, but the same pattern would apply to running multiple instances on the same box.

The general idea is to have no manual editing of anything in the deployment process. This improves reliability quite a bit, especially in production environments where you generally want to keep the deployment process as simple as possible and have everything come straight from your code repository. This makes it easier and less error prone to test a deployment candidate, and if something does happen it is easier to track down and fix, and in the mean time back up to and deploy a previous revision.

This would involve a set of patches for each target system. Sometimes if certain deployments have more variations it is helpful to have branches in your local SVN repository (or whatever you are using) that can be synchronized as needed and deployed from there.

Usually the patches would include one for each test, staging, and production deployment. It depends a lot on what your environment is like of course. In most cases development deployments are the "no patch" settings and these are mostly run locally on developers machines. OFBiz runs fine on plain old (but modern...) desktops and for development that is the way to go.

-David

Benjamin Cox wrote:

Folks,

I'm curious about what people are doing for configuration  

management?  For instance, deployment and/or development to different  

IPs, ports, etc. all on the same box, or spread across different  

machines.  We're setting up to do ofbiz development, and work with  

multiple developers on a single beefy box.  In addition, we would  

like to, say, run regression tests on the same box while developing  

(hey, it's beefy).  Either of these requires some means of keeping  

things separated, such that we don't conflict with one another.

This brings up the issue of maintaining many separate files with the  

same set of IP or port values used in them.  There are quite a few  

files in which a clean checkout of ofbiz currently has either  

"localhost", "127.0.0.1", or "8080" in them.  If we do this for our  

situation, this would means that a developer must change several  

files after checking out, and remember not to commit those changes  

back to our local repository.  Likewise, when deploying to a  

production environment, those values may have to be changed, possibly  

with different values for various components in a high-volume setting.

My question is how do people deal with different configurations of  

host and port, database parameters, etc. today?  Is there some single  

change point that controls all others?  Perhaps variables or  

properties resolvers so I can use a $primaryHost variable in several  

files, rather than "hard-coding" localhost into all of those files?

Or does everyone just use localhost:8080 for everywhere?

Any insight will be deeply appreciated,

   Ben

_______________________________________________

Users mailing list

[hidden email]

http://lists.ofbiz.org/mailman/listinfo/users

_______________________________________________

Users mailing list

[hidden email]

http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[cjhowe]

Along with patching in the local environment, creating
this improvement in Jira might make it easier for
updating this scenario...

http://jira.undersunconsulting.com/browse/OFBIZ-775

It focuses on making the configuration files that are
actually used (config.xml) not be under version
control, but rather have a template (config.tmpl) be
under version control.  Then in the build routine, if
the configuration file does not exist, have
config.tmpl be copied to config.xml so that the
default scenario will exist.


=========David Jones wrote:
Ben,

In general there shouldn't be any such references in
code (and if there are they should be fixed...). This
still leaves a fun mix of configuration files to deal
with and for those I usually recommend just using
patches for each deployment.

We run into this a fair amount with larger deployments
of multiple instances of OFBiz. I've mostly worked
with this in a server farm environment, but the same
pattern would apply to running multiple instances on
the same box.

The general idea is to have no manual editing of
anything in the deployment process. This improves
reliability quite a bit, especially in production
environments where you generally want to keep the
deployment process as simple as possible and have
everything come straight from your code repository.
This makes it easier and less error prone to test a
deployment candidate, and if something does happen it
is easier to track down and fix, and in the mean time
back up to and deploy a previous revision.

This would involve a set of patches for each target
system. Sometimes if certain deployments have more
variations it is helpful to have branches in your
local SVN repository (or whatever you are using) that
can be synchronized as needed and deployed from there.

Usually the patches would include one for each test,
staging, and production deployment. It depends a lot
on what your environment is like of course. In most
cases development deployments are the "no patch"
settings and these are mostly run locally on
developers machines. OFBiz runs fine on plain old (but
modern...) desktops and for development that is the
way to go.

-David



Benjamin Cox wrote:
> Folks,
>
> I'm curious about what people are doing for
configuration  
> management?  For instance, deployment and/or
development to different  
> IPs, ports, etc. all on the same box, or spread
across different  
> machines.  We're setting up to do ofbiz development,
and work with  
> multiple developers on a single beefy box.  In
addition, we would  
> like to, say, run regression tests on the same box
while developing  
> (hey, it's beefy).  Either of these requires some
means of keeping  
> things separated, such that we don't conflict with
one another.
>
> This brings up the issue of maintaining many
separate files with the  
> same set of IP or port values used in them.  There
are quite a few  
> files in which a clean checkout of ofbiz currently
has either  
> "localhost", "127.0.0.1", or "8080" in them.  If we
do this for our  
> situation, this would means that a developer must
change several  
> files after checking out, and remember not to commit
those changes  
> back to our local repository.  Likewise, when
deploying to a  
> production environment, those values may have to be
changed, possibly  
> with different values for various components in a
high-volume setting.
>
> My question is how do people deal with different
configurations of  
> host and port, database parameters, etc. today?  Is
there some single  
> change point that controls all others?  Perhaps
variables or  
> properties resolvers so I can use a $primaryHost
variable in several  
> files, rather than "hard-coding" localhost into all
of those files?
>
> Or does everyone just use localhost:8080 for
everywhere?
>
> Any insight will be deeply appreciated,
>
>    Ben
>  
 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Daniel Kunkel]

Hi

It seems like more and more users are starting to want to setup multiple
instances of OFBiz on the same box for testing, and even production.

What about changing the system so more, or even almost all of the
configuration settings come from within the database? Of course the
database connections will always have to be external, but most of the
other configuration files could be stored in a database table.

Actually, I would like to see a single instance of OFBiz support
multiple virtual domains, just switching the backend database for each
page depending on the request's header ie.

www.Visual-Illusion.net --> database1
www.SatelliteRadioZone.com --> database2
www.WizCity.com --> database3

Thanks

Daniel



On Wed, 2006-03-29 at 08:58 -0800, Chris Howe wrote:

> Along with patching in the local environment, creating
> this improvement in Jira might make it easier for
> updating this scenario...
>
> http://jira.undersunconsulting.com/browse/OFBIZ-775
>
> It focuses on making the configuration files that are
> actually used (config.xml) not be under version
> control, but rather have a template (config.tmpl) be
> under version control.  Then in the build routine, if
> the configuration file does not exist, have
> config.tmpl be copied to config.xml so that the
> default scenario will exist.
>
>
> =========David Jones wrote:
> Ben,
>
> In general there shouldn't be any such references in
> code (and if there are they should be fixed...). This
> still leaves a fun mix of configuration files to deal
> with and for those I usually recommend just using
> patches for each deployment.
>
> We run into this a fair amount with larger deployments
> of multiple instances of OFBiz. I've mostly worked
> with this in a server farm environment, but the same
> pattern would apply to running multiple instances on
> the same box.
>
> The general idea is to have no manual editing of
> anything in the deployment process. This improves
> reliability quite a bit, especially in production
> environments where you generally want to keep the
> deployment process as simple as possible and have
> everything come straight from your code repository.
> This makes it easier and less error prone to test a
> deployment candidate, and if something does happen it
> is easier to track down and fix, and in the mean time
> back up to and deploy a previous revision.
>
> This would involve a set of patches for each target
> system. Sometimes if certain deployments have more
> variations it is helpful to have branches in your
> local SVN repository (or whatever you are using) that
> can be synchronized as needed and deployed from there.
>
> Usually the patches would include one for each test,
> staging, and production deployment. It depends a lot
> on what your environment is like of course. In most
> cases development deployments are the "no patch"
> settings and these are mostly run locally on
> developers machines. OFBiz runs fine on plain old (but
> modern...) desktops and for development that is the
> way to go.
>
> -David
>
>
>
> Benjamin Cox wrote:
> > Folks,
> >
> > I'm curious about what people are doing for
> configuration  
> > management?  For instance, deployment and/or
> development to different  
> > IPs, ports, etc. all on the same box, or spread
> across different  
> > machines.  We're setting up to do ofbiz development,
> and work with  
> > multiple developers on a single beefy box.  In
> addition, we would  
> > like to, say, run regression tests on the same box
> while developing  
> > (hey, it's beefy).  Either of these requires some
> means of keeping  
> > things separated, such that we don't conflict with
> one another.
> >
> > This brings up the issue of maintaining many
> separate files with the  
> > same set of IP or port values used in them.  There
> are quite a few  
> > files in which a clean checkout of ofbiz currently
> has either  
> > "localhost", "127.0.0.1", or "8080" in them.  If we
> do this for our  
> > situation, this would means that a developer must
> change several  
> > files after checking out, and remember not to commit
> those changes  
> > back to our local repository.  Likewise, when
> deploying to a  
> > production environment, those values may have to be
> changed, possibly  
> > with different values for various components in a
> high-volume setting.
> >
> > My question is how do people deal with different
> configurations of  
> > host and port, database parameters, etc. today?  Is
> there some single  
> > change point that controls all others?  Perhaps
> variables or  
> > properties resolvers so I can use a $primaryHost
> variable in several  
> > files, rather than "hard-coding" localhost into all
> of those files?
> >
> > Or does everyone just use localhost:8080 for
> everywhere?
> >
> > Any insight will be deeply appreciated,
> >
> >    Ben
> >  
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

--
Daniel

*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-
Have a GREAT Day!

Daniel Kunkel           [hidden email]
BioWaves, LLC           http://www.BioWaves.com
14150 NE 20th St. Suite F1
Bellevue, WA 98007
800-734-3588    425-895-0050
http://www.Apartment-Pets.com  http://www.Focus-Illusion.com
http://www.Brain-Fun.com       http://www.ColorGlasses.com
*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-.,,.-*"*-

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[David E. Jones]

Ben,

You might already be familiar with this issue, but configuration management varies as much as configurations do. You may not think your particular needs are very creative, but you might be surprised at how many variations and strange things come up, and different this might be than what others are doing.

The patch idea probably wouldn't be too helpful in a development environment. Like I mentioned before: it is mainly for reproducibility in testing, staging, and production environments.

It sounds like you have something in mind that you'd like to see, and perhaps have seen elsewhere... is there anything you could share along those lines?

Some of this should perhaps be part of OFBiz, and some of it already is with the container stuff since you can have multiple startup properties files and multiple configurations and such. This is not applied to everything in OFBiz and is not really meant to handle what you describe.

>From experience with other application servers I've never seen anything that would handle this nicely. The only way it could be easier was if the application and infrastructure were smaller, then you wouldn't have so many things to configure. It's a bummer that things usually are not so simple...

-David



Benjamin Cox wrote:

> David,
>
> That sounds like a good start for most people, but still a tad brittle.  
> We would do developer instances on our own boxes as you suggest, except
> that we're scattered around the country and do a whole lot of pair
> programming.  VNC makes that work - we've found it easier and cheaper to
> maintain one multi-processor unix box for several developers.  Being a
> screamer, on a fat pipe, and capable of running several instances of
> it's nice lightweight window manager (with almost no color) - it saves
> us oodles of VNC Hell :-)
>
> Even if we weren't doing development like that, we'd still want to share
> integration test boxes for multiple users on multiple projects, to
> ensure a similar environment to our hosting env (i.e. not Windows).  It
> sounds like you're suggesting that each developer/instance maintain a
> diff file and run patch to alter various files throughout the tree... is
> that right?  
>
> This scares me for a few reasons:
>    * developers must manually change multiple relevant files for each
> value to create their personal patch - this is likely error-prone, and
> doesn't allow for composing or reusing values (a la Ant properties)
>    * individual developers must remember to apply the patch every time
> they check out fresh - better would be an externalized collection of
> their property values automatically picked up by the build process
>    * developers have to remember not to check in their patched files -
> or edit them 'back' in order to check in other parts of these files
>    * I haven't done enough patching recently, but I seem to remember it
> barfing on a file that didn't look sufficiently like the version from
> which the patch was produced.  So once I've applied it, how do I build a
> production copy?  A test copy?  Do I need a patch for every permutation?
>    * someone adding a value that requires 'patching in' could wreak
> havoc on the team.  Developers would either have to blow away their
> sandboxes and reapply the patch (possibly forcing them to check-in
> incomplete changes), or else we'd have to maintain incremental patch
> files (probably perpetuating the previously portrayed permutations
> problem! ;-)).
>
> One big difference between ofbiz and other libraries, app servers, or
> frameworks I've known is that it carries a development structure.  It's
> modular is another.  To my newbie eyes, it looks hard to separate ofbiz
> from the build and deployment process it lives in.  Thus, configuration
> management really seems to be the province of the framework - moreso
> than I'm used to, anyway.
>
> What do you guys think - is this something ofbiz should address more
> thoroughly?  If not, what is a developer to do if his needs are similar
> to mine?  Some of you have to be working in team environments in order
> to accomplish more than a bit of new stuff in a short period of time!  
> Where does this patching process break down in practice?
>
> Thanks again,
>
>   Ben
>
> ---
> HotWax Media (www.hotwaxmedia.com <http://www.hotwaxmedia.com>)
> Ben Cox
> Director of Technology
>
> "Reality is that which, when you stop believing in it, does not go
> away."  - Philip K. Dick
>
>
> On Mar 29, 2006, at 5:48 AM, David E. Jones wrote:
>
>>
>> Ben,
>>
>> In general there shouldn't be any such references in code (and if
>> there are they should be fixed...). This still leaves a fun mix of
>> configuration files to deal with and for those I usually recommend
>> just using patches for each deployment.
>>
>> We run into this a fair amount with larger deployments of multiple
>> instances of OFBiz. I've mostly worked with this in a server farm
>> environment, but the same pattern would apply to running multiple
>> instances on the same box.
>>
>> The general idea is to have no manual editing of anything in the
>> deployment process. This improves reliability quite a bit, especially
>> in production environments where you generally want to keep the
>> deployment process as simple as possible and have everything come
>> straight from your code repository. This makes it easier and less
>> error prone to test a deployment candidate, and if something does
>> happen it is easier to track down and fix, and in the mean time back
>> up to and deploy a previous revision.
>>
>> This would involve a set of patches for each target system. Sometimes
>> if certain deployments have more variations it is helpful to have
>> branches in your local SVN repository (or whatever you are using) that
>> can be synchronized as needed and deployed from there.
>>
>> Usually the patches would include one for each test, staging, and
>> production deployment. It depends a lot on what your environment is
>> like of course. In most cases development deployments are the "no
>> patch" settings and these are mostly run locally on developers
>> machines. OFBiz runs fine on plain old (but modern...) desktops and
>> for development that is the way to go.
>>
>> -David
>>
>>
>>
>> Benjamin Cox wrote:
>>> Folks,
>>>
>>> I'm curious about what people are doing for configuration  
>>> management?  For instance, deployment and/or development to different  
>>> IPs, ports, etc. all on the same box, or spread across different  
>>> machines.  We're setting up to do ofbiz development, and work with  
>>> multiple developers on a single beefy box.  In addition, we would  
>>> like to, say, run regression tests on the same box while developing  
>>> (hey, it's beefy).  Either of these requires some means of keeping  
>>> things separated, such that we don't conflict with one another.
>>>
>>> This brings up the issue of maintaining many separate files with the  
>>> same set of IP or port values used in them.  There are quite a few  
>>> files in which a clean checkout of ofbiz currently has either  
>>> "localhost", "127.0.0.1", or "8080" in them.  If we do this for our  
>>> situation, this would means that a developer must change several  
>>> files after checking out, and remember not to commit those changes  
>>> back to our local repository.  Likewise, when deploying to a  
>>> production environment, those values may have to be changed, possibly  
>>> with different values for various components in a high-volume setting.
>>>
>>> My question is how do people deal with different configurations of  
>>> host and port, database parameters, etc. today?  Is there some single  
>>> change point that controls all others?  Perhaps variables or  
>>> properties resolvers so I can use a $primaryHost variable in several  
>>> files, rather than "hard-coding" localhost into all of those files?
>>>
>>> Or does everyone just use localhost:8080 for everywhere?
>>>
>>> Any insight will be deeply appreciated,
>>>
>>>    Ben
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email] <mailto:[hidden email]>
>>> http://lists.ofbiz.org/mailman/listinfo/users
>>
>> _______________________________________________
>> Users mailing list
>> [hidden email] <mailto:[hidden email]>
>> http://lists.ofbiz.org/mailman/listinfo/users
>
>
> ------------------------------------------------------------------------
>
>  
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[BJ Freeman]

The configuration setup I use is a little different.
The Development source is on one machine and this I use the Ant to copy
the files to each instance. Since I can specify what files, I can not
send the two major configuration files, entityengine.xml and Base.
I have created extra folders with each instance names, where I store the
configuration specific files.

for other distribution to customers I install WAR_FTP server, if they do
not run one, already. I have two folders, one for current and one for new.
the ant runs the ftp just like the copy.

as far as the multiple databases in one instance, (there needs to be
more than this) the coded needs to be changed so the Store name is used
for which dB is used. Then in the entityengine.xml you define the
storename as the Delegator Name.

for the others like base/config/loadcontainers.xml where a delegator
element is added for each instance.

Now who has the time to implement this..
LOL.


Daniel Kunkel sent the following on 3/29/06 9:47 AM:

> Hi
>
> It seems like more and more users are starting to want to setup multiple
> instances of OFBiz on the same box for testing, and even production.
>
> What about changing the system so more, or even almost all of the
> configuration settings come from within the database? Of course the
> database connections will always have to be external, but most of the
> other configuration files could be stored in a database table.
>
> Actually, I would like to see a single instance of OFBiz support
> multiple virtual domains, just switching the backend database for each
> page depending on the request's header ie.
>
> www.Visual-Illusion.net --> database1
> www.SatelliteRadioZone.com --> database2
> www.WizCity.com --> database3
>
> Thanks
>
> Daniel
>
>
>
> On Wed, 2006-03-29 at 08:58 -0800, Chris Howe wrote:
>
>>Along with patching in the local environment, creating
>>this improvement in Jira might make it easier for
>>updating this scenario...
>>
>>http://jira.undersunconsulting.com/browse/OFBIZ-775
>>
>>It focuses on making the configuration files that are
>>actually used (config.xml) not be under version
>>control, but rather have a template (config.tmpl) be
>>under version control.  Then in the build routine, if
>>the configuration file does not exist, have
>>config.tmpl be copied to config.xml so that the
>>default scenario will exist.
>>
>>
>>=========David Jones wrote:
>>Ben,
>>
>>In general there shouldn't be any such references in
>>code (and if there are they should be fixed...). This
>>still leaves a fun mix of configuration files to deal
>>with and for those I usually recommend just using
>>patches for each deployment.
>>
>>We run into this a fair amount with larger deployments
>>of multiple instances of OFBiz. I've mostly worked
>>with this in a server farm environment, but the same
>>pattern would apply to running multiple instances on
>>the same box.
>>
>>The general idea is to have no manual editing of
>>anything in the deployment process. This improves
>>reliability quite a bit, especially in production
>>environments where you generally want to keep the
>>deployment process as simple as possible and have
>>everything come straight from your code repository.
>>This makes it easier and less error prone to test a
>>deployment candidate, and if something does happen it
>>is easier to track down and fix, and in the mean time
>>back up to and deploy a previous revision.
>>
>>This would involve a set of patches for each target
>>system. Sometimes if certain deployments have more
>>variations it is helpful to have branches in your
>>local SVN repository (or whatever you are using) that
>>can be synchronized as needed and deployed from there.
>>
>>Usually the patches would include one for each test,
>>staging, and production deployment. It depends a lot
>>on what your environment is like of course. In most
>>cases development deployments are the "no patch"
>>settings and these are mostly run locally on
>>developers machines. OFBiz runs fine on plain old (but
>>modern...) desktops and for development that is the
>>way to go.
>>
>>-David
>>
>>
>>
>>Benjamin Cox wrote:
>>
>>>Folks,
>>>
>>>I'm curious about what people are doing for
>>
>>configuration  
>>
>>>management?  For instance, deployment and/or
>>
>>development to different  
>>
>>>IPs, ports, etc. all on the same box, or spread
>>
>>across different  
>>
>>>machines.  We're setting up to do ofbiz development,
>>
>>and work with  
>>
>>>multiple developers on a single beefy box.  In
>>
>>addition, we would  
>>
>>>like to, say, run regression tests on the same box
>>
>>while developing  
>>
>>>(hey, it's beefy).  Either of these requires some
>>
>>means of keeping  
>>
>>>things separated, such that we don't conflict with
>>
>>one another.
>>
>>>This brings up the issue of maintaining many
>>
>>separate files with the  
>>
>>>same set of IP or port values used in them.  There
>>
>>are quite a few  
>>
>>>files in which a clean checkout of ofbiz currently
>>
>>has either  
>>
>>>"localhost", "127.0.0.1", or "8080" in them.  If we
>>
>>do this for our  
>>
>>>situation, this would means that a developer must
>>
>>change several  
>>
>>>files after checking out, and remember not to commit
>>
>>those changes  
>>
>>>back to our local repository.  Likewise, when
>>
>>deploying to a  
>>
>>>production environment, those values may have to be
>>
>>changed, possibly  
>>
>>>with different values for various components in a
>>
>>high-volume setting.
>>
>>>My question is how do people deal with different
>>
>>configurations of  
>>
>>>host and port, database parameters, etc. today?  Is
>>
>>there some single  
>>
>>>change point that controls all others?  Perhaps
>>
>>variables or  
>>
>>>properties resolvers so I can use a $primaryHost
>>
>>variable in several  
>>
>>>files, rather than "hard-coding" localhost into all
>>
>>of those files?
>>
>>>Or does everyone just use localhost:8080 for
>>
>>everywhere?
>>
>>>Any insight will be deeply appreciated,
>>>
>>>   Ben
>>>
>>
>>
>>_______________________________________________
>>Users mailing list
>>[hidden email]
>>http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Andrew Sykes]

Ben,

On some multiple deployment systems, we use patch files where the values
are replaced with variables e.g.

+ password="@@@password@@@"

We then use ant to do string replaces from .properties files on the
patch files before applying the patches themselves.

I know it sounds a little convoluted and it may not suit your specific
scenario, but it's another option to add to the armoury of tricks.
--
Kind Regards
Andrew Sykes <[hidden email]>
Sykes Development Ltd
http://www.sykesdevelopment.com

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[bjc]

Sure, I can share...  I don't want to assume my previous pattern will  
necessarily apply, but I suppose I'm asking for features that it  
provides, so I ought to lay it out and let you all decide for  
yourselves.

This is kind of long... if you'd prefer, you can just check out JFig  
(http://jfig.sourceforge.net/), whose homepage explains our  
motivations better than I probably could.  We've never used that,  
however, and I don't know if it's a good fit here at all, so I'll go  
ahead and explain our previous approach.

We devised something where we maintained a couple of files per  
developer or instance which were never committed.  Though similar to  
what Chris Howe suggested in that sense, we resolved the properties  
file from a hidden ".projectname" directory in the user's home by  
default.  That way we could work with different branches and/or blow  
away our sandbox and get a fresh start without having to keep track  
of copying these files around.  We could build a completely different  
instance configuration by passing a different properties file - no  
source code mods required.

We wanted the most oft-repeated configuration parameters in a common  
location, so we only had templates checked in for one or two  
properties files, rather than for all of the configuration files.  
Developers were expected to inform the team if they changed these, so  
others would know to go grab the new, necessary, values when they  
updated their sandbox to that rev.  In these properties files, we  
composed several values where we could.  An excerpt might look like:

server.hostname=tools.hotwaxmedia.com
instance.hostname=${user.name}.${server.hostname}

url.server.devel.jboss=${instance.hostname}:8080
url.server.devel.websphere=testsphere.${server.hostname}:9080

userdata.datasource.name=UserDataDS
userdata.datasource.jndi.name=jdbc/${userdata.datasource.name}
userdata.datasource.driver.class=oracle.jdbc.driver.OracleDriver
userdata.datasource.username=${user.name}
userdata.datasource.password=${user.name}
userdata.datasource.connection.url=jdbc:oracle:thin:$
{userdata.datasource.username}/${userdata.datasource.password}@
(description=(address_list=(address=(protocol=tcp)(host=$
{server.hostname})(port=1521)))(source_route=yes)(connect_data=
(sid=hotwax${user.name})))


You can see how the server.hostname, instance.hostname, and user.name  
(from the env) properties are used to compose other properties,  
including the SID (name) of the database.

In the configuration files throughout the tree that would require  
such values, you'd take something like (from ofbiz-containers.xml)
   <property name="default.host" value="0.0.0.0"/>
and replace it with:
   <property name="default-host" value="${instance.hostname}"/>

[[ An aside - actually, in this instance we'd have told our container  
his host (along with the java.rmi.server.hostname) on the command  
line, from the scripts we used to start and stop our containers.  But  
that was the typical J2EE world where the container was not  
configured from within the codebase, just the web or ejb context ]]

Now that we have a properties file and a bunch of Ant-property values  
scattered throughout the real configuration files, we have to have  
Ant fold in the properties.  We would do so in a <copy from="$
{source.dir}" to="${build.dir}"><filterchain expandproperties="true"/
 ></copy> Ant task and then use the files in our special build.dir  
instead of the original files to jar or run things.  This sounds  
similar to what BJ suggested here as I was writing the first version  
of this.  By the way, BJ, I'm not sure I completely understand your  
approach, but it may be a better fit for ofbiz ]].

Pros:

    * fairly pushbutton
    * new developers could easily tweak the most commonly-tweaked  
settings
    * when faced with a ${property} in a configuration file, there  
was one small group of files in which to look for their resolution
    * I could change the hostname property there and kick off a build  
and run from Ant, in which the EJB tier, Web apps, and GUI client all  
communicate properly without further intervention.  And, I could  
build and deploy an ear or any component thereof with the appropriate  
Ant task and "-Ddeploy.to.prod=true".

Cons:

    * it definitely breaks down if you are trying to maintain  
different sub-configurations, as illustrated with the  
url.server.devel.jboss and url.server.devel.websphere properties.  We  
would resolve those conditionally in the Ant file, and had to do so  
explicitly and so produce a little manual code to handle it
    * there was a step of copying the real configuration files, so  
we'd be unable to hot-deploy them without touching Ant (though a  
small, separate task could spare a rebuild)
    * developers would occasionally forget to either let others know  
or check their email before freaking out when they updated their  
sandbox and things broke.
    * other stuff I've probably forgotten

So that's it - what we've had in the fairly recent past, anyway.  As  
for doing something similar in ofbiz, I think that the container's  
presence in the codebase may complicate such an approach - we're not  
just creating war files to deploy!  But you all would know better  
than I...

Where to from here, people?  I could definitely devote a bit of time  
to such a thing, but I'm a little overwhelmed at the newness of it  
all as it is, so I'd not want to try that alone.  Besides, as David's  
valid warning about the one-size-doesn't-fit-all nature of  
configuration management suggests, it may not fit enough people's  
needs, so I'd really want some sense of agreement from people about it.

   Ben

On Mar 29, 2006, at 10:13 AM, BJ Freeman wrote:

> The configuration setup I use is a little different.
> The Development source is on one machine and this I use the Ant to  
> copy
> the files to each instance. Since I can specify what files, I can not
> send the two major configuration files, entityengine.xml and Base.
> I have created extra folders with each instance names, where I  
> store the
> configuration specific files.
>
> for other distribution to customers I install WAR_FTP server, if  
> they do
> not run one, already. I have two folders, one for current and one  
> for new.
> the ant runs the ftp just like the copy.
>
> as far as the multiple databases in one instance, (there needs to be
> more than this) the coded needs to be changed so the Store name is  
> used
> for which dB is used. Then in the entityengine.xml you define the
> storename as the Delegator Name.
>
> for the others like base/config/loadcontainers.xml where a delegator
> element is added for each instance.
>
> Now who has the time to implement this..
> LOL.
>
>
> Daniel Kunkel sent the following on 3/29/06 9:47 AM:
>> Hi
>>
>> It seems like more and more users are starting to want to setup  
>> multiple
>> instances of OFBiz on the same box for testing, and even production.
>>
>> What about changing the system so more, or even almost all of the
>> configuration settings come from within the database? Of course the
>> database connections will always have to be external, but most of the
>> other configuration files could be stored in a database table.
>>
>> Actually, I would like to see a single instance of OFBiz support
>> multiple virtual domains, just switching the backend database for  
>> each
>> page depending on the request's header ie.
>>
>> www.Visual-Illusion.net --> database1
>> www.SatelliteRadioZone.com --> database2
>> www.WizCity.com --> database3
>>
>> Thanks
>>
>> Daniel
>>
>>
>>
>> On Wed, 2006-03-29 at 08:58 -0800, Chris Howe wrote:
>>
>>> Along with patching in the local environment, creating
>>> this improvement in Jira might make it easier for
>>> updating this scenario...
>>>
>>> http://jira.undersunconsulting.com/browse/OFBIZ-775
>>>
>>> It focuses on making the configuration files that are
>>> actually used (config.xml) not be under version
>>> control, but rather have a template (config.tmpl) be
>>> under version control.  Then in the build routine, if
>>> the configuration file does not exist, have
>>> config.tmpl be copied to config.xml so that the
>>> default scenario will exist.
>>>
>>>
>>> =========David Jones wrote:
>>> Ben,
>>>
>>> In general there shouldn't be any such references in
>>> code (and if there are they should be fixed...). This
>>> still leaves a fun mix of configuration files to deal
>>> with and for those I usually recommend just using
>>> patches for each deployment.
>>>
>>> We run into this a fair amount with larger deployments
>>> of multiple instances of OFBiz. I've mostly worked
>>> with this in a server farm environment, but the same
>>> pattern would apply to running multiple instances on
>>> the same box.
>>>
>>> The general idea is to have no manual editing of
>>> anything in the deployment process. This improves
>>> reliability quite a bit, especially in production
>>> environments where you generally want to keep the
>>> deployment process as simple as possible and have
>>> everything come straight from your code repository.
>>> This makes it easier and less error prone to test a
>>> deployment candidate, and if something does happen it
>>> is easier to track down and fix, and in the mean time
>>> back up to and deploy a previous revision.
>>>
>>> This would involve a set of patches for each target
>>> system. Sometimes if certain deployments have more
>>> variations it is helpful to have branches in your
>>> local SVN repository (or whatever you are using) that
>>> can be synchronized as needed and deployed from there.
>>>
>>> Usually the patches would include one for each test,
>>> staging, and production deployment. It depends a lot
>>> on what your environment is like of course. In most
>>> cases development deployments are the "no patch"
>>> settings and these are mostly run locally on
>>> developers machines. OFBiz runs fine on plain old (but
>>> modern...) desktops and for development that is the
>>> way to go.
>>>
>>> -David
>>>
>>>
>>>
>>> Benjamin Cox wrote:
>>>
>>>> Folks,
>>>>
>>>> I'm curious about what people are doing for
>>>
>>> configuration
>>>
>>>> management?  For instance, deployment and/or
>>>
>>> development to different
>>>
>>>> IPs, ports, etc. all on the same box, or spread
>>>
>>> across different
>>>
>>>> machines.  We're setting up to do ofbiz development,
>>>
>>> and work with
>>>
>>>> multiple developers on a single beefy box.  In
>>>
>>> addition, we would
>>>
>>>> like to, say, run regression tests on the same box
>>>
>>> while developing
>>>
>>>> (hey, it's beefy).  Either of these requires some
>>>
>>> means of keeping
>>>
>>>> things separated, such that we don't conflict with
>>>
>>> one another.
>>>
>>>> This brings up the issue of maintaining many
>>>
>>> separate files with the
>>>
>>>> same set of IP or port values used in them.  There
>>>
>>> are quite a few
>>>
>>>> files in which a clean checkout of ofbiz currently
>>>
>>> has either
>>>
>>>> "localhost", "127.0.0.1", or "8080" in them.  If we
>>>
>>> do this for our
>>>
>>>> situation, this would means that a developer must
>>>
>>> change several
>>>
>>>> files after checking out, and remember not to commit
>>>
>>> those changes
>>>
>>>> back to our local repository.  Likewise, when
>>>
>>> deploying to a
>>>
>>>> production environment, those values may have to be
>>>
>>> changed, possibly
>>>
>>>> with different values for various components in a
>>>
>>> high-volume setting.
>>>
>>>> My question is how do people deal with different
>>>
>>> configurations of
>>>
>>>> host and port, database parameters, etc. today?  Is
>>>
>>> there some single
>>>
>>>> change point that controls all others?  Perhaps
>>>
>>> variables or
>>>
>>>> properties resolvers so I can use a $primaryHost
>>>
>>> variable in several
>>>
>>>> files, rather than "hard-coding" localhost into all
>>>
>>> of those files?
>>>
>>>> Or does everyone just use localhost:8080 for
>>>
>>> everywhere?
>>>
>>>> Any insight will be deeply appreciated,
>>>>
>>>>   Ben
>>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> [hidden email]
>>> http://lists.ofbiz.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[bjc]

Definitely more good grist for the mill...  thanks for that, Andrew.  
It doesn't sound a whole lot more convoluted than what I described.

A couple of questions:  does that get around the "unapply the patch"  
issues from my earlier post, or is there no real concern with that  
(and my 'patchy' memory was faulty - yuk yuk... er...  sorry)?  And  
also, I'm wondering whether you're satisfied with what you're doing  
at this point - does that meet your needs sufficiently?

Any more yeahs or nays about whether this is a sane or worthwhile  
effort?

Cheers,

   Ben

On Mar 29, 2006, at 10:53 AM, Andrew Sykes wrote:

> Ben,
>
> On some multiple deployment systems, we use patch files where the  
> values
> are replaced with variables e.g.
>
> + password="@@@password@@@"
>
> We then use ant to do string replaces from .properties files on the
> patch files before applying the patches themselves.
>
> I know it sounds a little convoluted and it may not suit your specific
> scenario, but it's another option to add to the armoury of tricks.
> --
> Kind Regards
> Andrew Sykes <[hidden email]>
> Sykes Development Ltd
> http://www.sykesdevelopment.com
>
>
> _______________________________________________
> Users mailing list
> [hidden email]
> http://lists.ofbiz.org/mailman/listinfo/users

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users

[Andrew Sykes]

Ben,

I don't really have any requirement to unpatch, but I don't see why it
couldn't be used in reverse.

I am reasonably happy with this situation because I'm a bit sceptical
about how much better it can get, to give an example...

In entityengine.xml we have one configuration (development) where we
simply change the password in another config though, we actually have to
add several new datasource blocks.

Adding the datasource node is quite a good use for the patch, we have a
patch to add a single datasource and we pass in the specific parts from
a properties file we then execute this patch multiple times. This is
really great when we want to add a new datasource, because we simply add
a few properties and ant automatically does an extra iteration of the
patch.

I think this is a good example of where simply abstracting a few one-
liners to a .properties file would fail miserably on it's own, but with
the two step process life is made much easier.

One thing that would probably be a lot more efficient/flexible than this
would be to somehow use FTL to transform the config, but I can't imagine
how that could be easily achieved. Perhaps Al Byers could say more about
this.

I also considered using something like eXist to store config, but again
this would probably be quite painful to setup. If eXist could also be
used to store service definitions and such like it would make the effort
much more viable, although this would probably have more of a
detrimental effect on performance than anything else...
--
Kind Regards
Andrew Sykes <[hidden email]>
Sykes Development Ltd
http://www.sykesdevelopment.com

 
_______________________________________________
Users mailing list
[hidden email]
http://lists.ofbiz.org/mailman/listinfo/users